On Sep 2, I was getting ready to report on my trip to Italy and my presentation to the World Federation of Scientists seminar in Erice, Sicily, when I was rudely interrupted by the publication of Mann et al 2008. It will take time to fully parse the situation, but the main framework is pretty clear. The addiction of Mann and the paleoclimate community to problematic Graybill bristlecone chronologies has not been cured. Indeed, Mann et al 2008 has made things worse, using even more fanciful data sets, such as serial (4 time) use of disturbed Kortajarrvi lake sediments.
I found the Erice conference very interesting. The topics were much more policy-oriented than I try to cover at the blog (climate was only one of many topics, others included energy supply, nuclear security, computer security). Over the next few months, I’ll relax these policies for some individual threads and pick up some of the themes at the conference, as these will undoubtedly be of interest to many readers – indeed, most of the themes will be of more interest to readers than the technical issues that I like to discuss.
But for today, you’ll have to content yourselves with my presentation. I had to submit a written paper and had a 20-minute talk. The presentation immediately prior to mine had used the HS (the IPCC 2001 version) as an unannotated “fact”, which was a good set-up for my presentation. The Erice conferences have a distinguished history of promoting openness in science even (and perhaps especially) in nuclear topics and I think that it was disappointing to many of them that climate scientists, of all people, should be anything less than 100% forthcoming with the provision of data and methods, and the idea of a climate scientist using personal Intellectual Property Rights as a pretext for refusing data and methods definitely did not sit well with scientists from other disciplines who are concerned, as citizens, about quantifying climate change.
Here is the written submission. Most readers here are familiar with the history of the dispute, but third parties aren’t and, points of detail such as different versions of Tornetrask, Polar Urals and Sheep Mountain obviously make no sense except in the context of an ongoing dialogue.
I then proceeded to discuss two problems which between the two of them, pretty much eviscerate all the IPCC AR4 reconstructions: (1) the divergence problem; and (2) differences between updated and IPCC versions of key sites (Tornetrask, Polar Urals and bristlecones). CA readers are familiar with these points, but I made new graphics in a consistent format illustrating the argument, which is pretty simple.
New versions of three important sites (Grudd’s Tornetrask, the unpublished Polar Urals update and Ababneh’s Sheep Mountain) have materially different MWP-modern differentials than the older versions used in the IPCC spaghetti graph (Briffa’s Tornetrask and Yamal; Graybill’s Sheep Mt/Mann’s PC1). Because so many purportedly “independent” studies are not actually “independent”, changes at only three sites cause a reversal of the MWP-modern relationship in 9 of 10 studies in the IPCC spaghetti graph and make the specific IPCC claim of AR4 unsupportable within their likelihood definitions. Here is the conclusion of my paper:
Although the statistical problems of the Mann et al (1998, 1999) reconstruction are by no means conceded within the reconstruction community, they have nonetheless been
identified for some time. Two blue ribbon U.S. panels have acknowledged these
criticisms, but IPCC 2007 did not.Updated versions of Tornetrask, Urals and Sheep Mountain have opposite medievalmodern
differentials to the IPCC versions. Because virtually all of the IPCC reconstructions rely on these three sites and because the framework of MWP proxies in IPCC reconstructions is so limited, changes in only 3 site versions turn out to have a knock-on impact on 9 of 10 reconstructions, an issue which also affects the Mann et al 1999 reconstruction additional to all the other problems. IPCC failed to provide any accounting or reconciliation of the discrepant versions.Adding to the problems of the IPCC 2007 reconstructions is the “Divergence Problem” – ring widths going down in the last half of the 20th century, while temperatures go up. In the absence of any such explanation and reconciliation, IPCC could not state within its probability definitions that: “[It is] likely that this 50-year period was the warmest Northern Hemisphere
period in the last 1.3 kyr.”Verification of paleoclimate studies has been made far more onerous than necessary, by
the failure of authors to archive data and to properly document methodological procedures. Econometrics journals have dealt with similar problems by requiring authors to archive data (as used with accurate data citations to the precise version) and source code as a condition of reviewing. This procedure is recommended for paleoclimate journals as well. In addition, the U.S. National Science Foundation (NSF) has essentially abandoned its duties to ensure that paleoclimate authors comply with existing U.S. data archiving policies and many problems could be averted merely by NSF carrying out its duties.
Climate Audit 
37 Comments
Did you get some feedback as to how your talk was received?
That’s what I want to know, too: how did the audience react to your presentation?
It has come to my attention that Willie Soon have this paper up on his “website”;
Click to access Nordell03.pdf
any one know more about it? It’s peer revived so it seams a bit strange that I haven’t seen it before…
Steve,
Did the prior presenter see your presentation? Did he/she comment on your presentation?
Also I keep hearing that Mann’s HS doesn’t matter since it is “verified” by many “independent” studies. Obviously this refers to the other incestuous IPCC spaghetti graphs that you show are dependent on the same dubious series as MBH 98.
Did you have a sense that individual scientists understood the ramifications of your presentation?
@Steve
I noted that the collaborative effort on the Almagre bristlecone data set appears completed – I hope there is an article for publication being built?
Steve,
Have you considered re-writing this for publication in a popular science magazine?
If you ever do, may I suggest that you first submit it to New Scientist… just to see how they phrase their letter of rejection 🙂
Nice presentation again! I also wonder how it was receieved and what folks said to you afterwards.
Here’s 3 things I noticed.
1) On page 2, the caption has ‘presents hockey stick’ It’s actually a supposed to be an independent recreation of the hockey stick that validates Mann’s which is actually Mann’s in new clothing with an addition. Maybe that should be mentioned somehow in passing.
2) On page 8 you have a bit of a cut and paste error it seems, and the plurals are a bit confusing as to how it got mixed up:
You should also mention where this is later discussed; it appears all four in the 2) section of The IPCC 2007 Saghetti Graph.
3) On page 9, you quote how difficult it’s supposed to be to update the proxies and on page 12 you show how easy it is to do with little expense (relatively). You could mention this in one or the other or both.
Hope that helps.
Steve:
Could you describe the size and composition of the audience ?
Please resist the temptation to “fully parse the situation” in public. GS would pounce.
Well, I’ve said it once and I’ll say it again. It’s useful poking holes in Mann, etc. It’s way, way more useful to present an alternative. If I had the capability to do a temperature reconstruction, I would. But I don’t. Until someone else does, Mann is the only one that really exists.
Build one, use all the best data, put all the code and data online, publish it. And then let the chips fall where they may. Until then, most people won’t care about some obscure problems with statistical analysis.
–t
Tim G comments: “Well, I’ve said it once and I’ll say it again. It’s useful poking holes in Mann, etc. It’s way, way more useful to present an alternative.”
I am sure many here are perfectly comfortable with an accurate statement of the fact that we have not yet developed a reliable pre-instrument temperature record. You can put lipstick on a pig, but it is still a pig.
Steve McIntyre writes:
What was this previous presentation about?
Come on Steve you’re teasing us. Are you trying to to sell seats for the denouement of the event or something?
“which was a good set up for my presentation…”
and then??
TonyB
Very nice summary, Steve. The statistical parts made my head hurt, but the message was clear. Thanks.
The alternative presented is “we don’t really know what climate was like way back when”.
Re: nanny_govt_sucks (#15), Except that we do have a large volume of historical texts done by careful and reputable historians and scientists that indicate the existence of an MWP in many parts of the world. When, if ever, has the Team, or the IPCC or the “thousand” concensus scientists, really, ever, dealt directly with these texts? It is as if we have swallowed Mann’s magic, even though we know it’s all smoke and mirrors, and that’s it: the texts don’t count. Don’t believe, don’t trust, their phony proxies, huh? Well, tough. That’s all you got, baby. Science, even fake science, triumphs. Get a life.
Loehle, C. 2007. A 2000-year global temperature reconstruction based on non-treering proxies. Energy & Environment 18(7-8): 1049-1058.
I am frustrated. I have been following the arguments for some years now. Principaly due to the great work of Steve, I am convinced that the Hockey Stick is nonsense. As a scientist myself, I can’t conceive how any scientist can follow the arguments and still believe the HS. The arguments remain strong, but I am afraid too abstract for the public or journalists. They remain focused on a comparison of ‘statistical methods’. However, I don’t see that statistical arguments are necessary for the ‘outside world’. Mann el al 2008 can be dismissed on much simpler grounds. I know the following is clear to all of you, but I think the problem is that it is not explained in terms that are clear and simple enough for the outside world.
Mann et al 2008 follows the old well-worn trick of laying thick red instrumental data on top of recent proxies. It implies that because recent instrumental data are higher than any historical proxies, we have record temperatures. But it is misleading to compare instrumental data directly with proxies. Proxies are, at best, smoothed representations of real temperature. At what period in history would we expect good proxies to best correlate with instrumental data? Today and in recent years – when we have the most numerous, accurate and most verifiable data, measurements, etc. However, we see with the ‘divergence problem’, this is not the case. Mann et al 2008 even aknowledges the dirvergence problem: “we observed evidence for a systematic bias in the underestimation of recent warming….the observed warming rises above the error bounds of the estimates during the 1980s decade, consistent with the known ‘‘divergence problem’ “. They conclude the problem is reduced through “elimination of all tree-ring data from the proxy dataset”. The best result is achieved with the EIV approach which uses “nonlocal and NON-TEMPERATURE RELATED proxy forrmation …thereby AVOIDING RELIANCE ON PURE TEMPERATURE PROXIES that may exhibit a low-biased sensitivity to recent temperature change.” [my capitals].
Thus, Mann et al state they have no explanation of why the instrumental data are so much higher than the proxy data in the most recent times. Without this explanation, a logical conclusion from the proxy graph is that the MWP was at least as warm as now and very possibly warmer.
Take the difference between instrumental and proxy for the most recent time: approx 0.7 deg C (in Mann et al, ignoring for now why they chose ‘land based, NH’ T data). Then add this to the highest proxy result in the MWP (0.2 degC). If there were relaible thermometers then, it is perfectly possible that they would have recorded up to 0.9 degC (on the given scale). Mann et al 2008 does nothing to demonstrate current temperatures are unprecedented in 2000 years. In fact their results seem to demonstrate that it was probably warmer in the past.
I would like to pick up on somethintg you said, Steve.
Currently, I am not aware of anything freely available to the general reader of the sort they you carry out here i.e. careful auditing of significant data sources with day by day explanations of what you are doing, all open to critical evaluation. Like a terrier after a rat describes your approach but you maintain a sense of humour with it. I do not claim to understand the detail of what you are doing but there are enough who apparently do to reassure me that what you are saying is of major importance. The fact that you are largely ignored by “The Team” says volumes.
If you want to broaden things to cover other topics then this is your site and many others will no doubt welcome it. Please consider how best you might so this without loosing out on the major strength. I would hate to see critics using a change of approach to try to undermine the validity of your core work.
However, you are probably right and I am probably wrong and its your site! QED
best wishes
Steve: No, I’m not going to broaden the focus of the site other than, if and when I report on the Erice conference, I may allow a thread or two on topics from the conference.
PHE,
I am also frustrated. This alleged science of temperature reconstruction is so broken now, it has no basis in reality. I just figured out today that the proxy data was extended at the endpoints using a predictive algorithm in Mann08. Something, probably half the readers here already know. I had assumed it was based on new measurments in the area or some kind of addition based on data which did exist in the calibration range, but no….Jeff don’t be ridiculous.
I can’t beleive they can take a proxy, totally unproven to have anything to do with temperature and use an algorithm to extend the endpoints, calibrate and correlate to the result and then brag to the world about temperature. The “predicted” endpoint shot the data up of nearly every data set by a huge amount. I just plotted the ‘added’ values on my blog last night. I can’t work on it full time so it was this afternoon before I found out how this data was added. How can this get through peer review, it’s insane.
My graphs are of over 1000 modified proxies most of the dataset, not just the used portion in the hockey stick graph, yet the modified series added together had a huge uptick in value. I’m no rocket surgeon but I’m pretty sure you can’t predict the endpoint of a proxy and use it for correlation analysis to find out if it might be temperature!! If Mann can pull it off accurately he should be in stocks.
Here’s my post, I linked to it on another thread
You only need to see the last graph.
I am putting it here because I read Steve McIntyre’s presentation and saw his discussion about using data which wasn’t updated in other papers and that simply updating the data increased the MWP above the current temp. When taken in the context of the ‘divergence’ (trees won’t do what I want) problem (which just happens to be the same timeframe as the “prediction”) this predictive algorithm becomes critically important to the result.
I have to ask, I hope someone will answer. Are these series in the latest 08 Mann paper the latest updated series or are they intentionally old series with predicted, (fake, imagined, colored in with a crayon) free of divergence data in place of the new, real, measured data??
Really amazingly frustrating.
Re: Jeff Id (#19),
Rather easily, I’m afraid. That is bcause the prevailing culture is the belief that the science is “self-correcting”. And it is. But not efficiently so. It takes too long, and this is a problem when policy people are promoting the science before it’s had a chance to ripen.
#16. A lot of people don’t understand why my approach has been critical without presenting an alternative. One of the reasons is partly background in financial markets. I view some of my work as being similar to that of an analyst at a securities firm who recommended a sell order on (say) Enron, or perhaps to be more current, Lehmann Bros. There’s lots of value in the sell recommendation even if you have no buy recommendations in mind. A recommendation that combined a sell on Lehmann with a buy on something that went up would, of course, be a better overall recommendation. However, if your mandate as an analyst is financial stocks, I don’t think that any of your clients would have minded a recommendation to stay on the sidelines.
#19. Jeff, in the case of series ca534, which is the most prominent HS shaped series in the dendro roster going back to the MWP, the Mann 2008 version (allproxy1209) shows this series carried forward to 1998 complete with supposed counts of trees right off to 1998. The counts are completely fictitious and are merely the count of trees in the last year of the Graybill record (1990 in this case, 1984 or earlier in other cases). In this case, there was an actual update of the series (Ababneh) and one of the Mann et al 2008 coauthors (Hughes) was on Ababneh’s thesis committee so they knew about the data, but elected to use invented data. We had discussions of the Ababneh data last fall. A CA reader telephoned her and she said that she had received legal advice not to release the data. (She’s young and untenured so I don’t blame her, but it was all quite weird.)
I think that another reason for not providing an alternative temperature reconstruction from the work on this site is that this site is, simply, an audit of climate science work.
Auditors shouldn’t, in general, advise on differing methods as this would impair their independence – instead their job is to check everything and decide (or allow other people to decide) if the approach is reasonable.
If an auditor suggests a different approach then it could be that they find themselves in a position to be unable to audit the different approach fairly (or at least they won’t be seen to be acting fairly – which is probably more important).
That said if somebody other than SM came up with an alternative reconstruction I’m sure that it wouldn’t be long before it found its way through the Climate Audit review process!
Steve: Quite so. A company can’t say to an auditor – if you don’t like the way we do our accounting, then start your own damn business. Doesn’t work that way. But scientists frequently don’t understand this approach – there’s a bit of a cultural divide – and say that science is “self-correcting”. Well, there’s no such thing as something being “self-correcting”; people correct things. In my opinion, audits improve the efficiency of the correction process. Markets are also “self-correcting” but audits have an important role in ensuring that information in the market is as accurate as possible. They don’t eliminate problems but the cover-ups of problems would go on much longer without them and the problems would be far worse without them.
17 –PHE,
I have given a lot of thought to the way I would make a presentation about AGW to my son’s high school science class, if given the opportunity. I think the best way to make the case to the general public (about AGW generally and the hockey stick specifically) is to point out how the methodology used by Mann and other climate scientists departs from the public’s assumption of how science should be conducted. First, as for the team, they make up some of their data and they throw out real data that doesn’t agree with them. Because standard statistics won’t support their conclusions, they make up new statistical tests. They abuse stat techniques by using them improperly. Second, as for the rest of climate scientists, no one ever bothers to check anyone else’s work. In fact, they make it incredibly hard for other scientists to do so.
I would start out by explaining what science has to be done to get a new drug approved or for one of
Steve’s geological securities offerings. Then I’d contrast all the checks and balances and double checks with the complete absence of such in climate science.
If you get bogged down in the science, eyes glaze over. However, everyone understands that their grade school teachers expected them to check their arithmetic before it was turned in. If you tell them that no one ever bothered to see if the temperature sites are properly sited, they can understand why that’s a huge problem. If you show that climate scientists all embraced the hockey stick without ever “checking their math”, they understand how reckless that is. They understand careful. They understand the need for an audit. They understand “making stuff up”.
I do understand. And from the perspective of science, it is a very admirable (and extremely useful) thing to be doing. My comment was coming from a more “Realpolitik” perspective. As much as I hate to see politics intermingled with science, this is exactly what’s happening.
People don’t come to this site because they care what the temperature was 800 years ago in Greenland. The only reason anyone cares about Mann or the Hockey Stick Graph is because it affects public policy. I’m pretty sure that’s the only reason you’ve taken the time to care.
And since this really is an issue of public policy, it really is less about science. The people that make the decisions in the world (the voters and their representative) are really the target of all this work (both yours and Mann’s). Mann and his crew recognize this. That’s why they make sure their graph says something to non-scientists. It practically hits you over the head with its conclusions.
Normal people are not scientists. They need to understand the issue without understanding obscure statistics. This is why Gore made his (in)famous movie. They have no reason to believe Gore more than you. But since Gore is the only one talking to them, that’s all they hear. And, by extension, that’s all they know.
In a perfect world, the critiques from this site would be heeded by other scientists. Those scientists would then create their own reconstructions and hopefully challenge Mann, et al. But that doesn’t seem to be happening. If Nature (and everyone else) is declaring that the Hockey Stick is alive and well, then it’s clear this site has had (essentially) no affect on the public dialog. You are, unfortunately, just tilting at windmills.
Don’t get me wrong. I personally have the highest respect for what you are doing. I think you are a very honest, reasonable and hard-working scientist. I’d personally like to see your work get a larger audience. But since it isn’t written for one, it can’t.
Thanks,
tim
Bender,
I am really really surprised by this. Unless I’m missing something, they filled in predicted data in a large part of the correlation section. I keep thinking I’m missing something but I can’t find it. I’ve read that paper a bunch of times and I can’t figure out what I’m missing. How can they add that huge chunk to the end of the data right in the calibration range. I don’t care what statistical prediction process you use, it doesn’t make sense. – Did you see the last curve I plotted on the link above? Two weeks ago, I was a nice happy skeptic not too worried about a chunk of ice melting but this craziness is pushing me further than that.
It makes it hard to be open minded.
Just wow!
Re: Jeff Id (#27),
RegEM is a method they developed to (legitimately) interpolate missing data. Mann has taken to using it (or something like it?) for extrapolating missing (i.e. future) data. Has this change in application ever been documented or validated in the literature? I do not think so. Correct me if I am wrong.
#27. Jeff, I’m looking through their code to see how and when their correlations were calculated – before or after infilling. Right now it looks like they were done after infilling. Which makes the whole thing an even worse mess than is on the table right now.
While this may seem opaque to you now, it’s NOTHING like trying to parse MBH when I started. This time, Mann has archived a lot of code. Now it’s lousy code and no one’s been able to make it work just yet, but it gives a clue as to what he did. With MBH, everything was much more of a mystery and it was hard to even get a foothold.
Availability of code makes assessment far more efficient – which is the rationale for archiving code and data in econometrics journals.
It’s also a method used originally to interpolate like data e.g. missing gridcell values in a gridcell network. Not exactly the same thing as going from bristlecones and disturbed Finnish sediments to instrumental temperatures.
Re: Steve McIntyre (#30),
Quite so.
The key proxy values for the most recent years are extrapolated…wow. Is this like projected earnings for a company? Of course, you don’t get to base a proof of something on projected earnings…
I can understand the use of an algorithm like this to “fill in” data between knowns. It probably would do an ok job, but prediction is unreasonable. I think it shows from the strong net positive in the (primarily tree ring) signal I graphed whereas the same tree rings are experiencing “divergence” in reality.
I love that word divergence.
The grid cell thing I read about in Mann08 makes more sense to me than extension. Its accuracy would depend on the density and proximity of real values. While its accuracy might be off, at least it makes sense.
Re: Jeff Id (#33), The “divergence problem” is a real and well-known problem in dendroclimatology. An area of active research.
hi
the ERICE documentation is more like a paper, than the presentation. Have I got this wrong, and will Steve be putting the presentation (ppt ?) up ?
I think that most scientists are quite horrified by some of the behaviour unearthed in the course of this fiasco. That is why it is very good to see Steve having the standing to be able to present (credibly) to other scientists.
I do think that there is one very important difference practically in how science works. In business, there is often a legal obligation to provide information; so it doesn’t matter how rude you are, it has to be given to you. In science, there is no statutory obligation, and it goes on people’s good will. In these circumstances, if you are abrasive, people will occasionally refuse you just for personal reasons. None of this is exculpatory; just a comment.
per
Bender #34,
I know. If you look at what I found in the data modifications, a forecast extrapolated spike was added to the recent end of the Mann08 proxies. From what I can tell, (still a rookie) it is a nice way to correct for “divergence”. I probably wasn’t clear enough.
This was done despite the availability of better, newer data in at least one case. (our host’s point in #21)
This is why Steve McIntyre was looking at the software to see if it really was used in the correlation calcs.
If the theory doesn’t fit the data, change the data.
If the darn trees won’t grow, we’ll grow em with statistical fertilizer. BS has more than one use. (I couldn’t resist)
Steve, the Erice manuscript, at least the one I downloaded, needs a serious proofing. Will you do it, or would you like a volunteer?