Though I haven’t posted for a while, I’ve done quite a bit of work on climate recently, though it hasn’t been the sort of work that lends itself readily to blog posts.
I made a presentation at a workshop session in Erice in the third week of August, which, at Chris Essex’ request, was entitled “Year in Review”, focusing on developments in proxy reconstructions. Ross McKitrick made a presentation with an identical title, covering other topics. I spent much of my time on the section on proxy reconstructions in the forthcoming IPCC report, as presaged in the Second Draft, a document which I’ve had for some time, but which I hadn’t yet parsed. In carrying out my own internal review, I re-examined the voluminous literature on individual proxies: ice cores, speleothems, ocean sediments as well as tree rings and “multi-proxy” reconstructions.
The review reminded me of conversations that I had with two prominent though then relatively early/mid-career climate scientists shortly before the announcement of AR4 in January 2007 (mentioned in passing here). The occasion was the AGU conference in December 2006, where there had been a Union session on the NAS panel report on reconstructions, which was then very fresh. Both scientists said that they were convinced by our criticism of the data and methods used by Mann and similar articles and agreed that there would be little progress in the field without the development of new and better data. The more optimistic of the two estimated that this would take 10 years; the more pessimistic estimated 20 years. Although both scientists were tenured, neither was willing to be identified publicly and both required that I keep their identities confidential.
It is now seven years later – 70% of the way through the 10 year process contemplated by the more “optimistic” of the two scientists. This ought to be enough time to see first fruits of any improvement in the proxies themselves. My re-examination of literature on proxies was done with this in mind.
As CA readers are aware, I have been very critical of the repeated use in IPCC studies of proxies with known attributes (e.g. Graybill’s bristlecone chronologies and Briffa’s Yamal). Such “data snooping” – a term used in wider statistical literature – poisons standard statistical tests. In my opinion, real progress in the field will only come when performance and consistency can be demonstrated with out-of-sample proxies of the same “type”.
Conversely, I see little purpose whatever in the application of increasingly complicated and increasingly poorly understood multivariate methods to snooped datasets (e.g. ones with Graybill bristlecones and/or Briffa Yamal). Nor to datasets with gross contamination, such as the Tiljander or Igaliku lake sediments. Nor do I see much chance of progress when specialists are unable to specify the orientation of a proxy ex ante. Or when they use multivariate methods that permit ex post flipping or screening.
Over the past seven years, one sees both tendencies at work.
The production of temperature reconstructions using increasingly complicated multivariate methods on snooped datasets has continued unabated. Mann et al 2008 is the most prominent such article, but there have been others applying very complicated methods to what ought to be a simple problem.
New multiproxy reconstuctions cited by IPCC (e.g. PAGES2K) have also increasingly incorporated lake sediment data, especially since Kaufman et al 2009. Lake sediments have much higher accumulation than ocean sediments and in principle ought to yield higher resolution. However, ocean specialists have focused on a few proxies (O18, Mg-Ca, alkenones) and thereby developed populations that increasingly permit analysis for consistency. In contrast, lake sediment specialists have reported a bewildering variety of proxies (varve thickness, grain size, chironomids, XRay fluorescence, greyscale, pigments, pollen flux, …as well as occasional O18, Mg-Ca and alkenone) without seemingly making any effort at consistency. Lake sediments are also vulnerable to human disturbance e.g. agricultural activity at Korttajarvi (Tiljander) and Igaliku, rendering modern contamination a very real problem. Specialists have rushed to incorporate this still inchoate data into multiproxy reconstructions (Mann et al 2008; Kaufman et al 2009; PAGES2K; Tingley and Huybers 2013), without adequate precautions to ensure that the data is actually a proxy for temperature.
On the other hand, there has also been steady though less publicized progress by specialists on other fronts.
Antarctic ice cores are an “old” proxy, and, in deeper time, O18 data from Antarctic and Greenland ice cores have been perhaps the most important proxy. However, prior to 2013, no annually-resolved Antarctic core with data to the medieval period had been archived. (IPCC AR4 authors refused to show a relatively high (4-year) resolution Law Dome isotope series.) In 2013, three series became available: Law Dome, Steig’s new data from West Antarctic and, miracle of miracles, even an Ellen Mosley-Thompson series from the 1980s – to my knowledge, the first known sighting of Ellen Mosley-Thompson isotope data. (Despite the importance of Antarctica in the Southern Hemisphere, Mann et al 2008 did not use any Antarctic isotope data before the 13th century, instead reconstructing SH temperature with NH data such as bristlecones and contaminated Finnish sediments, as discussed below.) Unfortunately, no Greenland isotope data more recent than 1995 has been archived – the dead hand of Ellen Mosley-Thompson chilling progress in this area.
On a positive note, I am particularly struck by the progress in high-resolution O18 speleothem data, especially in China. This development has passed mostly under the radar, but offers several lines of real progress.
First, some speleothems directly connect the past two millennia to the Holocene. Placement of one- to two-millennium reconstructions in a Holocene context seems to me to be an important somewhat-behind-the-scenes trend in the field. Esper et al 2012, an under-discussed article, has an extremely interesting discussion in respect to tree rings – a topic that I’ve meant to discuss for ages and promise to get to. There has been interesting progress on treelines, including at (of all places) Yamal, again a topic that I’ve meant to discuss and promise to get to.
Second, speleothems provide information on the tropics and subtropics, areas which were abysmally covered by AR4 reconstructions. The interesting population of Chinese speleothems provide important insight into the East Asian monsoon, an insight that is much enhanced by a perspective that extends back through the Holocene to the LGM – a perspective that offers the opportunity to orient the data on a more rational basis that 20th century correlation.
Third, some speleothems are located relatively close to Lonnie Thompson’s tropical ice core data and permit a much clearer perspective on this data – particularly when placed in a Holocene context.
There has also been considerable progress in high-resolution ocean sediment series. Ocean specialists have tended to focus on “deep time”, but in the past decade, “high resolution” data has become more available. Somewhat counterintuitively, the recovery of recent sediments is a serious problem with many ocean cores: uppermost sediments are poorly recovered by piston corers and thus data on the past one-to-two millennia is surprisingly spotty and 20th century data is relatively uncommon.
So there was a both an opportunity and a need for an insightful assessment of work in the field since AR4 by the IPCC AR5 authors. In a recent post, Judy Curry mentioned that a young scientist of her acquaintance had complained of the tension in AR5 between incoming scientists with a fresh perspective and holdovers who were primarily concerned with vindicating/protecting AR4.
The Lead Author roster for the paleoclimate chapter gave grounds for both optimism and pessimism as to whether the section on recent paleoclimate would be insightful.
Giving some grounds for optimism was that Valerie Masson-Delmotte had succeeded Jonathan Overpeck as Coordinating Lead Author. Although committed to IPCC conclusions, Masson-Delmotte is more ecumenical in perspective. Plus, in her own right, she had written competent, interesting and detailed assessments and reconciliations of Antarctic ice cores – precisely the sort of thing that needs to be done with lake sediments and tree rings. For example, Masson et al 2001 canvassed all Holocene Antarctic ice cores, carefully analysing inconsistencies between the information. Some coastal ice cores showed strong recent increases in d18O that were inconsistent with declines in inland sites. Rather than relying on Mannian statistics to sort out inconsistent data, she concluded that the coastal ice cores had been affected by ice flow – with the lower part of the core coming originally from higher elevations. A careful assessment of the proxy literature in accordance with Masson-Delmotte’s own style and standards would have been a welcome change from, for example, Mann’s paean to the Hockey Stick (and Briffa’s obsequious defence).
However, the Lead Author of the section on recent paleoclimatology was Tim Osborn of CRU, a prominent Climategate correspondent. It is impossible to imagine an author more closely allied to Mann, Jones and Briffa and less qualified to provide an independent assessment of their work. In the wake of Climategate, Hans von Storch (among others) had urged CRU authors to step down from IPCC assessment roles, but instead a CRU author was placed in charge of the section that had occasioned the primary past controversy.
In addition, although Briffa, Mann and Jones have attracted most of the attention, Osborn had a personal role in some of the most notorious Climategate incidents and has undeservedly flown beneath the radar. It was Osborn who physically deleted the post-1960 data that hid the decline in the IPCC 2001 report. Osborn was also co-author of the first article that originally deleted the declining data (with the result that the inconsistency between the Briffa and Mann reconstructions was disguised.) It was Osborn who Mann asked not to reveal his “dirty laundry” to the wider community. In a lesser known AR4 hide the decline incident, Osborn suggested that AR4 authors withhold any visual display of the declining Law Dome (Antarctica) isotope series, even though this left the AR4 illustration with only two long Southern Hemisphere proxies (both tree ring series used by Gergis). Osborn was also one of the participants in the delete-all-emails incident. Remarkably, despite his central role in these incidents, Osborn is not listed as having been interviewed by any of the “inquiries” and has sailed pretty much under the radar.
Osborn’s publication record consists almost entirely of joint articles with Briffa – articles which have been rightly reviled on skeptic blogs both for internal inconsistency and assertions that can only be reasonably characterized as cargo cult “science”. I and others have focused on Briffa, but Osborn was also responsible.
Worse, given Osborn’s unrepented complicity in Hide the Decline and similar incidents, no reader can reasonably presume that Osborn has disclosed results adverse to the “message” or that the form of presentation has not been tailored to the advantage of the “message”. As too often, one has to watch the pea.
On balance, the IPCC section on recent paleoclimate mostly lives up to what one would expect from Osborn. Its focus is mainly on multiproxy reconstructions, rather than the data within the reconstructions. Unfortunately, many of these “new” reconstructions use snooped datasets and it takes time to determine what, if anything, is “new” in the reconstruction (or whether it depends on Graybill’s bristlecones or other “old” data). In fairness, AR5 conceded that NH reconstructions prior to AD1200 were heavily dependent on limited data (carefully avoiding the term “bristlecone”) lowering their confidence in reconstructions prior to AD1200, while still displaying them. Remarkably, their NH spaghetti graph ecumenically includes the Loehle and McCulloch 2008 reconstruction. (I haven’t endorsed this reconstruction as being “right” but see no reason why it is unworthy of being in a spaghetti graph.)
Their handling of the Southern Hemisphere is very curious – a topic that I’ll address in detail. However, as a quick preview, readers will recall that Briffa once famously wrote:
I am sick to death of Mann stating his reconstruction represents the tropical area just because it contains a few (poorly temperature representative ) tropical series. He is just as capable of regressing these data again any other “target” series, such as the increasing trend of self-opinionated verbage he has produced over the last few years
Despite presumed awareness of this absurd aspect of Mann reconstructions, IPCC’s Southern Hemisphere disagram relies on Mann et al 2008 reconstructions, which suffer from and exacerbate the defect observed years earlier by Briffa: Mann et al 2008 did not only use bristlecones and contaminated Tiljander sediments for its NH reconstruction, but for its Southern Hemisphere reconstruction. While IPCC did not place great confidence in Mann’s SH reconstruction, it is not clear that it contains any usable scientific information on SH temperature history.
Their section on the tropics is also curious: in the Zero Order Draft, they had observed that there was “nothing unusual” about recent drought or floods in a 1000-year context. At the time, I thought that there was zero chance that this (true) observation would survive the editorial slant. It’s interesting to see how this has evolved. An important new emphasis for recent paleoclimate in AR5 are model-proxy comparisons arising out of the PMIP3 Last Millennium project. I haven’t discussed this interesting enterprise in the past, but it is an important new direction and quite interesting.
In all, there’s lots of fresh material which lends itself to many individual posts. I’ve got much material in inventory and I’ll see what overall conclusions result from the exercise of writing them up.
However, to the extent that I might have hoped that IPCC AR5 would shed fresh insight into the development of recent paleoclimatology, it is mostly very disappointing. Too much rationalization of questionable multiproxy studies and not nearly enough assessment of actual progress on the data front. In a phrase: too much Osborn and not enough Masson-Delmotte.
Climate Audit 
73 Comments
Thanks Mr McIntyre, we need you.
Dear Steve,
as always good to hear from you! I think we all looking forward to see your take on the new data, but I also enjoyed your update of what is going on in the paleo-IPCC-world..
Thank you very much and keep up the blogging!
LoN
P.S.: there is a double “that” in the 3rd last paragraph..
Welcome back, Steve.
I always wondered about the Osborn factor – ever since he teamed up with Briffa, to “respond” to Holland’s submission to Muir Russell with what struck me as a colour-coded, cherry-picking exercise – based on their, uh, reconstruction of Boulton’s butchery.
With Stocker as Co-Chair and Osborn as designated “hockey-stick” preserver, we may find that AR5 is a candidate for tossing Into the Dustbin 😉
P.S. A typo you might want to fix: “I that that there was zero chance ….”
Less publicized until now. Thank you and welcome back.
“IPCC’s Southern Hemisphere disagram relies on Mann et al 2008 reconstructions”
*Disagram* Is that like disinformation, only in pictures…
Steve: good question.
Steve,
Glad you’re back, and up to good things.
Osborn, Briffa and Mann, when was the last time any of them had a real field season? I don’t know, but I’ve formed the opinion [maybe in ignorance] that these characters prefer to sit behind the desk and play with their old and other peoples’ data rather than going out and getting new or better data. Why get new data when you can get away with torturing the old data forever? Some of their reluctance seems to be a basic unwillingness to risk: heat, cold, hunger, fatigue or seasickness to bringing back the new data. I’ll try not to be too cynical and suggest this is the preferred strategy because it gives a better return-on-investment not to go out into the field – the grant pie usually has to be cut in a lot more slices out in the field.
The Thompsons, to their, credit seem to like it out in the field, they just don’t seem to know what to do with the data once they get back to the office to recharge the grant account in preparation for the next campaign.
I’m the pessimist about institutional behavior. Institutions like the IPCC do not change, or change their positions willingly. Most institutions will willingly: explode, implode, or slide into complete irrelevance rather than change their fundamental position. A few have learned to appear to tap-dance around what is currently expedient [maybe you can think of some], but that is merely a symptom a deeper and degenerate institutional behavior – loss of clear operating principles.
In so far as most science seems to have been co-opted by large bureaucratic institutions: academic and governmental, there will be no improvement in the situation until either the institution, or the people running them, who ever that may actually be [and I don’t necessarily mean that in the conspiratorial sense], expire. I wait breathlessly for news of them meeting the yawning grave.
W^3
Steve A review of candidate proxy data reconstructions and the written record of climate suggests that at this time the most useful 2000 year reconstruction is that of Christiansen and Ljungqvist 2012 (Fig 5)
Click to access cp-8-765-2012.pdf
Are you familiar with this and does it appear at all in AR5?
That paper has a number of issues, and the authors don’t seem keen to fix them. For example, their stated requirement for proxies to be included in their long reconstruction was that the series have decadal resolution. I e-mailed one of the authors to inform him the 25th series in their paper, which was used back to 505 AD, did not have decadal resolution back to 505. This was made obvious by the fact the table listing their proxies said that series starts in 1505. I suggested it was inadvertently included back to 505 it had decadal resolution to 1505, but 30 year resolution back to 505. The author acknowledged I was right, but that’s it.
Similarly, I pointed out their 21st series (Dulan) was said by its originators to be a precipitation proxy, not a temperature proxy. There was a brief exchange in which both authors said precipitation and temperatures had a particular correlation in the region, thus it was fine to use the Dulan series as either (I’m sure S&B would love to hear this). Only, when I read the paper sent to me to support this supposed correlation, the paper didn’t show the signal they claimed existed. I pointed this out, and I didn’t get a response.
There were several other issues I raised that the authors indicated they were aware of, but they made no effort to address. My impression is they wanted to just throw a bunch of series together without giving any thought to what that data was. The fact other people used the series seemed to be enough for them.
And that‘s never happened before in paleoclimatology.
I guess we should just be grateful that’s all it took. They could have been like other people and just picked series they knew would give them answers they’d like.
Whoo hoo, welcome back.
More of Masson-Delmotte? Please, not!
Sue
It might be agnotography, if there is such a word.
Welcome back Mr McIntyre.
I have understood little of what you have written here but
it does not stop me from being amazed by your ability to
bring disparate aspects of a topic together.
Either you have a photographic memory or the most amazing
information retreval system.
Re: pesadia (Sep 11 16:42),
Not sure about either; but there is a wonderful adherence to logical thought and subsequently allowing only the logic to pervade.
Thanks Mr McIntyre for making me aware that thought is a discipline.
The moving hand writes and having writ
Gets itself encased in ice and sits.
====================
I can think of a useful word in the singular to replace “sits”
good to hear from you again Steve.
hope all is is well with you & the family.
when you say above – “treelines, including at (of all places) Yamal.
do you have a link to latest sudies or is this the best link to use ?
http://www.eoearth.org/view/article/151226/
oops – or studies even
from above link last line –
“Finally, environmental complexity and novelty in an altered, warmer world is likely to produce unexpected vegetation patterns at a local or regional scale.”
had to add that I suppose, or they wouldn’t sleep that night.
“unexpected vegetation patterns”
I guess, being a layman, I would be laughed at for expecting that a warmer world would lead to more vegetation in general. Especially if the warmth was due to higher levels of CO2.
But of course I’m not a climate scientist.
The summer drought breaks! First Donna LaFramboise returns and now Steve.
Welcome back!
Hi Steve,
Very valuable update. You’re right that Tim Osborn has been largely under the radar in the debates despite his key role, but his data mining rarely gets called out. There is one exception (besides your reviews) – his 2006 Science commentary (with Briffa, see http://www.sciencemag.org/content/311/5762/841.abstract), which concluded “the spatial extent of recent warmth to be of greater significance than that during the medieval period”. Surprisingly, Science allowed a comment by Gerd Bürger (see
http://www.sciencemag.org/content/316/5833/1844.1.abstract ). Prof. Bürger pointed out that Osborn’s study “ignores the effect of proxy screening on the corresponding significance levels. After appropriate correction, the significance of the 20th-century warming anomaly disappears”. In a very fine comment, he continues ” the reported anomalous warmth of the 20th century is at least partly based on a circularity of the method, and similar results could be obtained for any proxies, even random-based proxies. This is not reflected in the reported significance levels”.
And yet, Osborn’s comment has been cited more than 20 times, while Prof.Bürger’s clear unravelling of the Osborn paper has been cited only once (by Phil Jones!) (and then ignored).
The State of Climate Science in a nutshell (so to speak).
Steve: good spotting. Burger did some excellent work. I don’t recall seeing anything from Burger for a number of years. Wonder why?
Hi Steve,
I don’t know exactly what happened to Prof. Bürger and why he disappeared. I like to think he killed a man. It’s the romantic in me.
An alternative explanation may be that he got fed up with the unscientific responses to his work on these issues (for example see http://www.clim-past-discuss.net/2/357/2006/cpd-2-357-2006-discussion.html and read his final comment on “lessons learned”). Perhaps also the combined attack by Mann and Littlemore (see http://www.ecowho.com/foia.php?file=0035.txt ) and other insiders was too much to take. After all, he was lead chapter author for the IPCC TAR.
He seems to have worked for three years in Canada at the Pacific Climate Impacts Consortium in Victor, BC, but now seems to be back at the University of Potsdam. There do not seem to have been many papers in the meantime but the latest paper on downscaling extremes is at the Journal of Climate at http://journals.ametsoc.org/doi/abs/10.1175/JCLI-D-12-00249.1 .
We’re eventually going to need a concordance/index for CA, but the earlier comment on this case is available at https://climateaudit.org/2007/06/30/burger-comment-on-osborn-and-briffa-2006/ .
Give Tim Osborn credit. He was aware of the limitations of his own work. Here’s a Climategate classic – from memory I think the backdrop was something to do with Mann spinning an exculpatory yarn to Andy Revkin, and needing Osborn to warrant that O&B2006 was unaffected by Steve’s earlier Yamal revelations. It wasn’t.
“I wouldn’t say we were immune to the issue – results are similar for these leave 1, 2 or 3 out cases, but they certainly are not as strong as the case with all 14 proxies. Certainly in figure S6, there are some cases with 3 omitted (i.e. some sets of 11) where modern results are comparable with intermittent periods between 800 and 1100. Plus there is the additional uncertainty, discussed on the final page of the supplementary information, associated with linking the proxy records to real temperatures (remember we have no formal calibration, we’re just counting proxies-I’m still amazed that Science agreed to publish something where the main analysis only involves counting from 1 to 14!)”
…not to mention the amazement that a scientist could think counting from 1 to 14 was worth a Science submission in the first place!
No points for figuring out who reviewer #2 was in Bürger’s paper. Reviewer #2 even gets smacked down by the editor when the paper is finally rejected. I hope Prof. Bürger worked out his calculations somewhere else.
Burger published some controversial papers when he was at the Free University of Berlin. That stopped when he moved to the University of Potsdam. He may have tired of the team abuse, or he may have been told that such publications were not good for his employment prospects. In any case, his focus has been in model downscaling and extreme precipitation… about as far from controversial as you can get in climate science.
Killed a man? No, he is at the University of Potsdam, at the institute for Earth- and enviromental-sciences:
http://www.geo.uni-potsdam.de/mitarbeiterdetails/show/353/Gerd_B%C3%BCrger.html
(In case anybody wonders: The PIK in Potsdam is as far it seems unrelated to the University of Potsdam)
Geoff, sorry for the late response, but I am well and, as far as I know, did not kill anybody.
Yes, climate reconstruction can be a dirty business, but I mainly moved away from it as I just got bored.
Steve: nice to hear from you, Gerd.
The following on Gerd Burger from the University of Potsdam web site:
http://www.geo.uni-potsdam.de/mitarbeiterdetails/show/353/Gerd_B%C3%BCrger.html
Biografie
1985
Diplom (Mathematik), Freiburg
1989
Dissertation (Mathematische Logik), Freiburg
1990 – 1992
Max Planck Institut für Meteorologie, Hamburg
1992 – 1994
Lamont-Doherty Earth Observatory of Columbia University, New York
1995 – 2003
Potsdam Institut für Klimafolgenforschung, Potsdam
2003 – 2004
Meteo Service Weather Research, Berlin
2004 – 2007
Freie Universität, Berlin
2007 – 2009
Universität Potsdam
2009 – 2012
Pacific Climate Impacts Consortium, Victoria
2012 –
Universität Potsdam
Contact info. is on the web site.
I wonder if Burger’s papers made their way into the UEA email treasure trove? Moshpit… did you ever see ‘Burger’ referenced?
Steve, I just took a look at the Climategate 1+2 emails on EchoWho. Burger was treated respectfully at East Anglia. They asked each other to look at his work, cited him sometimes, and discussed his comment on their paper in courteous terms. Richard Littlemore, on the other hand, had no such scruples. He sent an extremely combative email about a post on this site, like so:
Not really surprising, except for the amusing admission.
Yes, I did the same. Osbourn in particular seems to be trying to keep the relationship collegial. Mike Mann seems a lot less than collegial toward Burger. Geoff’s link to Burger’s ‘lessons learned’ statement suggests that he was discouraged and frustrated by the treatment his last ‘controversial’ paper on reconstructions got (especially from reviewer #2, AKA ‘the Mann’), and he did not try to publish anything else on reconstructions after that episode.
Did I miss the link to Steve’s and Ross’ presentations at Erice?
Steve: I didn’t post a link. I was working on an annotation of my speaking notes, but got distracted by some points that I wished to further clarify.
Roger. All in good time.
Thompson & Thompson keep on drilling but delay publishing data they don’t like. Mann looks like professor Calculus but acts more like captain Haddock in his treatment of others data. Tim Osborn used to look like Tintin himself but lost the plot in his eagerness to play with the big shots at the IPCC.
/humor off
Glad to see your latest post. We have been disappointed daily for a while.
I think we will need to wait to at least AR6 before there is real reform, assuming that the whole corrupt facade does not collapse in the meantime.
What has happened to the observation about recent drought or floods? I felt I was on the edge of my seat there but maybe I missed it. Or is this one of many subjects for future AR5-related posts?
Excellent – Welcome Back, Steve!
No one quite like you for shining a light into the murky depths of climate ‘science’.
Following Climate Audit McIntyre’s pioneering lead, Ian Boyd addresses the need for science audits.
Research: A standard for policy-relevant science. Nature doi:10.1038/501159a. See discussion at Climate Etc.
Readers need to realise that Ian Boyd is the current Chief Scientific Advisor to the UK Department of Environment Food and Rural Affairs. This statement may therefore exert some influence at least on UK science.
Very interesting AD. Saw the thread on CE earlier but didn’t know that angle. Ministers asking the right questions at last?
Let us not forget Osborn’s role in efforts to prevent the release of Yamal measurement data because he and Briffa “have none to give out!”
Steve, do you ever feel guilty for killing climate change?
Tomas, he is doing no such thing. He is providing analysis that helps to explain it honestly and with proper science.
Besides the fact that Steve cannot change the weather, he has done no such thing. The existence of the hockey stick does not strengthen the case for global warming, it weakens it. Michael Mann has said that the non-global nature of the Medieval Warm Period, suggests a long-term negative feedback to warming, which means that climate models ‘vastly overstate’ warming.
Sorry to be a pest, but is there a relatively easy definition of “snooped data” for someone, like myself, who has a minimal amount of statistical training?
Aka, cherry picking, i.e., choosing data prior to actual analysis that looks like the desired result. Also known as self deception if you’re being kind and cheating otherwise.
“Data dredging (data fishing, data snooping, equation fitting) is the use of data mining to uncover relationships in data.”
“The process of data mining involves automatically testing huge numbers of hypotheses about a single data set by exhaustively searching for combinations of variables that might show a correlation. Conventional tests of statistical significance are based on the probability that an observation arose by chance, and necessarily accept some risk of mistaken test results, called the significance. When large numbers of tests are performed, some produce false results, hence 5% of randomly chosen hypotheses turn out to be significant at the 5% level, 1% turn out to be significant at the 1% significance level, and so on, by chance alone. When enough hypotheses are tested, it is virtually certain that some falsely appear statistically significant, since every data set with any degree of randomness contains some spurious correlations. Researchers using data mining techniques if they are not careful can be easily misled by these apparently significant results, even though they are mere artifacts of random variation.”
http://en.wikipedia.org/wiki/Data-snooping_bias
There have been notable refinements to the art of data snooping in recent years, as exemplified by Ammann & Wahl’s calibration/verification RE statistic. I believe the main innovation here is that the failure to achieve the desired significance level using conventional statistical tests on any combination of relevant variables and datasets at all should prompt the creation of one’s own statistic. Advanced users – such as Ammann & Wahl again – are even able to assert an arbitrary benchmark for simulation run rejection, thus ensuring statistical significance.
Re: igsy (Sep 15 12:25),
++++++111111111!!!!!!
brilliant.
There are a variety of different types of data manipulation sins – some venial, most mortal. Another form is dust-bowl empiricism. Steve’s emphasis on replication is crucial for ensuring that “findings” are simply not a matter of chance.
An important new emphasis for recent paleoclimate in AR5 are model-proxy comparisons
There is a nice plot of Mann08 (and another one that i forgot) vs a modele in the SO2
Who put it in AR5 ? Osborn ?
Who, with guilty feeling, insisted tu put tiny cross in the corner of the graph showing that others reconstructions were not all “consistent” (or whatever word they used) ?
Or, an alternate question, who was furious that his reconstruction wasn’t in the graph ?
But the best one, because we can always expect the best from the team:
Who find the amazing idea to use a 1800 years baseline ?
Model output is a natural CO2 hockey stick. Here we need a master of trick, someone unable to refrain himself to use trick even if there is no need.
Excellent work, Steve. I had to look up the word ‘ecumenical’. LOL.
Dr. Fred Singer has said that he believes temperatures will continue their gradual rise for the next several hundred years until the height of the Medieval Warm Period is reached.
Is it possible to determine with any reasonable accuracy what global mean temperature was at the height of the MWP, and when it occurred, without doing a full blown multi-century paleoclimate reconstruction?
Regarding snooped data sets, I don’t understand the value to the Paleo scientists of including them in their studies. For arguments sake, let’s assume the snooped data sets don’t make a difference to the results. It would make sense to exclude them, simply to remove the objection. What’s the harm in that? You remove a significant objection, get the same result.
On the other hand, if the snooped data sets do make a difference, and Paleo folks are aware of the issues, then one has to question the integrity of the truth seeking. As in, perhaps there is identifiable bias.
In this case, it seems some enterprising paleo scientist ought to redo the studies, without the snooped data, and without the poor statistical methods, and republish. It would seems what is at stake ought to be high enough to warrant checking, and it seems there is enough interest that some scientist/grad student ought to want to. So, why hasn’t it happened?
From my perspective, having read your notes for a number of years now, though I’m not a statistician, it seems there is enough evidence to cast doubt on the results themselves.
A key component to the scientific method is replication of results. Identifying data and or method issues ought to encompass that. In short, one possible results here is the scientific method is not being adhered to, for reasons I don’t understand.
Money.
Do you mean it costs money? Even if it cost $10M, an unbiased recreation of paleo reconstructions is worth it.
No. I mean they will not adhere to the proper scientific methods because they are afraid that the results will totally undermine their case and put their funding at risk. These people are being paid to support AGW and will continue to conduct themselves accordingly.
I also believe that a lot of these individuals are not statistically or mathematically competent or trained and therefore do not understand how to properly structure their data or their analysis. For years they have steadfastly resisted enlisting competent expert statisticians and mathematicians to assist in their endeavors. Why? One can only speculate.
Snooping is not so easily corrected. You can’t simply remove “snooped data” from a study as you suggest. It is practically impossible to determine which data in the study was included because when “snooped” it gave the “right” answer. More seriously, how are you going to fix the problem of data missing from the study because when “snooped” it gave the “wrong” answer.
The amount of effort required to try to correct this type of bias is so great that it would be simpler to just redo everything properly from scratch.
More on IPCC at Judith Curry’s place:
Looking forward to the results of your investigation of whether there have been any improvements in the proxies. This does appear to be a neglected area.
Much attention is currently being given to the import of new, current data, i.e. “The pause” But I am not seeing much attention given to Development of better, more accurate, historical data. And an update on the Thompson’s disclosures of data, or lack thereof, is always interesting.
So how’s your squash game? 🙂
I much appreciated SteveM’s thread introduction where he has reiterated throughout his review of the temperature reconstructions and proxies the problem of data snooping and the limitations that that problem presents for these published temperature reconstructions. Data snooping or selection of proxies for use in temperature reconstructions based on after the fact measurements in my mind is a not only a major impediment to validating proxy thermometers but is a show stopper. Without those working in this field recognizing the dangers of data snooping and the limitations of in-sample results, I do not see a path for this field progressing and no matter how much new proxy data are presented or what methods – as pretty as they may be – are used to grind the data.
Many of the problems, that the field either hand waves away or ignores, can be shown to be related to the problems that can occur from using proxies that are not responding reasonably well to temperature changes but rather from finding spurious relationships between two variables that have high levels of auto correlations. Those types of spurious relationships, where the physical or other independent means of supporting a relationship are tenuous or do not exist and linear trends in the series being correlated can occur not infrequently by chance, are well known in the field of statistics. The divergence problem with TRW proxies is an example of evidence for a spurious relationship. Remember also that non dendro proxies can suffer from divergence problems as noted in Mann (08).
I have for some time now thought that the lack of appreciation of some scientists working in hard or near hard sciences to the problems of data snooping results from the fact that if one can preliminarily snoop the data before running confirming experiments, the confirming and controlled experiments provide out-of-sample testing that is truly a valid method. With climate and economics the opportunities to use controlled experiments is very limited to nonexistent and thus the hard scientist delving into these areas of research may well forget that statistical testing must take that limitation into account before doing analyses.
I also strongly suspect that many interested laypersons on all sides of the AGW issue do not appreciate the nuanced difference between the requirements for a hard science versus softer science testing and application of proper statistics. The problem with that lack of understanding is that policy makers and reviewers of climate science findings, like the IPCC, take the peer reviewed results as valid without ever broaching the problem of data snooping and how that might well invalidate the findings in published papers. For example, for my approval, the IPCC would have to devote an entire section on the problems of data snooping before I would be convinced that the problem was understood and appreciated.
I would suppose that there are alternative routes to showing the weaknesses of proxies and temperature reconstructions but those come in my judgment to be at secondary and at times tertiary level and would ignore the basic problem presented by data snooping.
Ways around the data snooping problem with temperature reconstructions are: (1) providing a physically based criteria for selecting proxies and then using all the data from the selected proxies, (2) very scrupulously separating the data used into two parts for calibration and validation (this method can work but is fraught with problems associated with peeking at all the data), (3) testing the conclusions from the original data with data that could not have been available to those using the original data, i.e. using out-sample-testing and (4) using proxy thermometers that can be tested against temperatures much in the way that instrumental thermometers are currently tested and used. On this last point I would think that some of the proxy methods that are more directly based in physics than TRWs, like isotope ratios, would be favored candidates.
Even with something as well based in physics as instrumental thermometers can have problems outside the domain of physics. Think of the differences between the uncorrected and corrected trends in the US based on the instrumental temperature records. The other problems with proxies is that the data is limited both spatially and temporally when compared with the instrumental record. I have often heard when comparing proxy estimated temperatures that have large differences regionally that those differences may well be real and thus not indicative of the problems with the proxy responses to temperatures. Unfortunately if one assumes those differences as real then one must immediately address the problem that presents as expanded CIs on global mean temperatures given the lack spatial and temporal data.
I just finished perusing the Gerd Burger paper linked above and while I think that he pushes all the right buttons with regards to the problems of selecting proxies after the fact, I have not been able to understand completely how he is able to analyze results that were determined using the wrongheaded and after the fact selection criteria. He is correct in showing that this wrongheaded approach greatly expands the CIs, but determining that expansion is no easy task given that approach and what you cannot know with regards to the number of degrees of freedom lost. A Bonferroni correction can in theory adjust the CIs but that adjustment requires knowing all the trials that were used for the final selection. That would be difficult to do without the original experimenters scrupulously recording all alternative proxies considered and even those that were rejected without formal testing. If those experimenters did not mention the problem that the Bonferroni addresses you can be rather certain that they have no understanding of it or made no attempts to correct for it.
In the manner of peer-review and in replying to it, Burger was obviously limited in his criticism to only the one paper in question. On the other hand, a review process, like the AR5 by way of the IPCC, could well apply such a criticism across the board. The fact that the IPCC does not even address the issue in any depth, leads me to conclude that as an organization they are either horribly deficient in understanding of the statistics involved and/or have an agenda that requires that organization to ignore the problem.
Kenneth,
Another big problem, in my small view, is the use of statistics to estimate errors, when the methods provide information on the scatter of precision, but ignore bias.
One can more easily estimate precision type errors, as when comparing replicated data, than accuracy or bias errors. As a simple example, two independent studies of the work using Tiljander sediments could readily yield results for precision testing; but the estimation of bias depends in part on values assigned to the known effects of sediment disruption.
It is not adequate, when publishing error bars, to simply include a statistical estimate of precision. There needs to be an actual calculation of bias, rather than merely a mention of effects that could cause it.
I suspect that the omission of bias estimates causes problems with CMIP work.
A bit off topic but, I have been very pleasantly surprised and impressed with the comments Judith Curry has made about the IPCC. While I have had my disagreements with Dr. Curry in the past here at CA concerning primarily tropical storm propensity past, present and future, I give her much credit for discussing issues about the IPCC that no other climate scientist would appear to want to initiate. Her comments on the IPCC’s attempts to put formal and objective appearing uncertainty limits on the reviewed findings and evidence for climate effects have been spot-on in my opinion. She is the only climate scientist, who to my knowledge, has pointed to the AR4 attempts of publishing uncertainty limits and requesting the lead authors to document the means by which these limits were reached and then never following through on enforcement or producing these means for public consumption. I made my requests several times to the IPCC for this information and never even received a reply.
She also has seen the disadvantage and biasness of the IPCC presenting evidence as a lawyer would for his client in an adversarial court case, i.e. only one side of the issue. She has recommended that the IPCC AR5 be the last session and replaced by authors/reviewers presenting their cases for both/all sides to the AGW issue. She has noted that the IPCC (and others in the field) have weakened their cases by attempting show and use the concept of a consensus on AGW. I agree whole heartedly with Dr. Curry on these issues.
Thank you.
+
‘He was a tremendous fighter, but he never started fights. I don’t believe that he liked to get into them, despite the fact that he came from a line of fighters. He never went for another dog’s throat but for one of its ears (that teaches a dog a lesson), and he would get his grip, close his eyes, and hold on. He could hold on for hours. His longest fight lasted from dusk until almost pitch-dark, one Sunday.’
……..
‘Rex’s joy of battle, when battle was joined, was almost tranquil. He had a kind of pleasant expression during fights, not a vicious one,his eyes closed in what would have seemed to be sleep had it not been for the turmoil of the struggle.’
H/t James Thurber, ‘Snapshot of a Dog’.
==================
With Peter Clark on board, you’d think that the Super Hickey Stick has to appear somewhere.
Maybe last minute so the criticisms don’t rain it out?
On rereading the Gerd Burger paper and SI linked above, I see where he uses Monte Carlo simulations to obtain CIs for the correlations of proxies to CRU instrumental temperatures. He does this to show that the original significance level of 0.5 used by Osborne/Briffa for a merely positive correlation goes to a ridiculous and useless 0.99 where the selection and location product of proxies is 10. A significance level for correlation of 0.05 would go to 0.4 and for one of 0.01 would go to 0.1 given the selection/location product of 10.
All of these calculations made by Burger are well and good in showing a theoretical limitation of using a selection criteria for proxies after the fact. I, however, would have to disagree with the implications left by Burger in his SI that a selection process after the fact might work within these theoretical limitations if the selection were based on a much higher certainty of correlation of the proxy series with the instrumental temperatures – taking into account the persistence of the series.
Using proxies as thermometers requires an initial reasonable physical connection to a consistent and predictable response to temperature and that requirement has to have a basis that goes beyond a momentarily significant (and not necessarily high) correlation of a proxy with temperature in the instrumental period.
I noticed that Burger looks at the annual correlations and decadal correlations in his paper. Doing this means he is aware of the issue of higher versus lower frequency correlations and which should be used in an after the fact selection criteria. I would say that after the fact selections should not be used, but if used the limitations should be pointed to. Proxies in temperature reconstructions are used primarily to compare trends of the current warming period to those in historical times. Trends are best determined using low frequency correlations as it can shown rather easily that two series could have a good high frequency correlation and contain very different trending tendencies. Using lower frequency correlations has its own problem which is manifested in more readily obtaining spurious low frequency correlations when the series have a high level of persistence.
Steve – long-time lurker and I’ve been waiting for an update after your couple months absence. A nice summary article.
You mention [some quotation snips] “…prior to 2013, no annually-resolved Antarctic core with data to the medieval period had been archived … In 2013, three series became available: Law Dome, Steig’s new data from West Antarctic and … an Ellen Mosley-Thompson series from the 1980s.”
Is there a location where one can get (to do a comparison plot, say) the data for these 3 series (as you state, newly available in 2013), so compare the records back to the medieval period? If you are planning to do this and publish on your blog in the future, let us know.
Thanks.
Steve: PAGES2K data is in spreadsheet at http://hurricane.ncdc.noaa.gov/pls/paleox/f?p=519:2:0::::P1_study_id:12621
Test: CA assistant blanking comments?
2 Trackbacks
[…] More: https://climateaudit.org/2013/09/11/ipcc-and-the-end-of-summer/ […]
[…] https://climateaudit.org/2013/09/11/ipcc-and-the-end-of-summer/ […]