In September 2008, Mann et al reported a “significant development” in paleoclimate reconstructions – a “skillfull” reconstruction without tree ring data for over 1300 years.
A skillful EIV reconstruction without tree-ring data is possible even further back, over at least the past 1,300 years, for NH combined land plus ocean temperature (see SI Text). This achievement represents a significant development relative to earlier studies with sparser proxy networks (4) where it was not possible to obtain skillful long-term reconstructions without tree-ring data.
The story was widely covered at the time and the result has been relied upon to marginalize criticism of the reliance of IPCC multiproxy studies on strip bark bulges or tree ring chronologies developed by CRU. Now it turns out that the much vaunted claim to have a “validated” no-dendro reconstruction for the past 1300 years was merely an illusion.
Not only was it an illusion, but recent admissions by Gavin Schmidt show that it foundered on Mann’s much criticized use of the Tiljander sediments – a topic on which the seeming obtuseness of the climate science community to the simplest of issues (e.g. contamination by bridge and agricultural sediments) has mystified third parties over the past two years. Only last month, Schmidt had re-assured readers at Keith Kloor’s that Mann’s misuse of the Tiljander sediments didn’t “matter”. It turns out that it did.
Mann et al 2008
Let’s go back to Mann et al 2008. Claims for its no-dendro reconstruction featured prominently not just in the abstract and conclusions of the article itself, but in associated press releases and promotion.
The abstract to Mann et al 2008 stated:
Recent warmth appears anomalous for at least the past 1,300 years whether or not tree-ring data are used.
The running text stated:
For both methods, we perform reconstructions both with and without dendroclimatic proxies to address any potential sensitivity of our conclusions to issues that have been raised with regard to the reliability of tree-ring data on multicentury time scales (4, 11, 16, 19, 33, 34)…
A skillful EIV reconstruction without tree-ring data is possible even further back, over at least the past 1,300 years, for NH combined land plus ocean temperature (see SI Text). This achievement represents a significant development relative to earlier studies with sparser proxy networks (4) where it was not possible to obtain skillful long-term reconstructions without tree-ring data…
We place greatest confidence in the EIV reconstructions, particularly back to A.D. 700, when a skillful reconstruction as noted earlier is possible without using tree-ring data at all…
Recent warmth exceeds that reconstructed for at least the past 1,800 years in the EIV reconstructions, and this conclusion extends back at least 1,500 years without using tree-ring data.
The point was re-iterated in the caption to its Figure S6.
News Releases for Mann et al 2008
The no-dendro claim of Mann et al 2008 was heavily promoted by Penn State and by the media.
The Penn State news release stated
Results of this study without tree-ring data show that for the Northern Hemisphere, the last 10 years are likely unusually warm for not just the past 1,000 as reported in the 1990s paper and others, but for at least another 300 years going back to about A.D. 700 without using tree-ring data. The same conclusion holds back to A.D. 300 if the researchers include tree-ring data.
“Ten years ago, we could not simply eliminate all the tree-ring data from our network because we did not have enough other proxy climate records to piece together a reliable global record,” said Mann. “With the considerably expanded networks of data now available, we can indeed obtain a reliable long-term record without using tree rings.“
The no-dendro reconstruction was breathlessly reported at realclimate (a post subsequently used as authority by the recent Tamino post):
Now though, the Northern hemisphere land temperature reconstructions without tree rings can go back to 1500 AD or 1000 AD depending on which of two methodologies are used. For the NH land and ocean target, it’s even possible to get a coherent non-tree ring reconstruction back to 700 AD!
Results of this study without tree-ring data show that for the Northern Hemisphere, the last 10 years are likely unusually warm for not just the past 1,000 as reported in the 1990s paper and others, but for at least another 300 years going back to about A.D. 700 without using tree-ring data.
Numerous other news outlets and blogs picked up the “discovery”.
Mann et al 2008 was covered in numerous contemporary Climate Audit posts. It was quickly discovered that the heavily publicized no-dendro reconstruction used Tiljander’s Lake Korttajarvi sediments despite warning from Tiljander that the sediments had been heavily contaminated by modern construction and farming, making them totally unsuitable for inclusion in the Mann 2008 algorithm. The contamination was so severe that it resulted in Mann et al using the data upside-down to the climatic interpretation adopted by the original authors for the pre-contamination portion of the series.
The SI showed that Mann et al were aware of Tiljander’s caveats but used the contaminated sediments (upside-down) anyway, thereby compromising the no-dendro reconstruction. They purported to justify the inclusion of Tiljander sediments on the grounds that their inclusion didn’t “matter” because they could “get” a somewhat similar looking stick without the Tiljander sediments. The obvious question was – if they didn’t “matter”, then why use them, given the explicit caveats of the originating author? A question that has never received an answer – only the excuse that the use of the compromised proxies didn’t “matter” – an excuse that is now known to be untrue given the “validation” failure of the no-dendro network without the Tljander sediments.
In January 2009, a short comment by Ross and I on Mann et al was published by PNAS, a comment which included criticism of Mann’s use of the Tiljander sediments despite the caveats. In their response, Mann et al merely stated that our criticism of their use of the Tiljander proxies was “bizarre” and that their SI:
showed that none of our central conclusions relied on their use.
Given the central role of the claim to have a “validated” no-dendro reconstruction, this claim in their Reply now appears to be untrue.
Over the next year and half, Mann’s use of the Tiljander sediments has been a recurring issue over many blogs. A newcomer to the climate blogosphere (AMac), who entered with no particular preconceptions, became extremely frustrated with the obtuseness of the climate community to what seemed to be a black-and-white simple issue and challenged Mann’s defenders. It seemed obvious to him (and to me and to others) that Mann et al had erred in their usage of their proxy and that it was their responsibility to acknowledge and correct the error. AMac was persistent. The more that the “community” arm-waved, the more frustrated he got.
The issue was re-raised most recently at Keith Kloor’s Collide-a-scape in a lengthy thread where Gavin Schmidt argued that critics of Mann et al 2008 were refusing to listen, while Gavin’s critics countered that Gavin’s arguments didn’t make any sense, a position summarized by Lucia on June 18, 2010 with her characteristic lucidity as follows:
So let’s go back to Gavin’s closing complaint:
Thus what we have is not scientists refusing to engage with serious questions, it is the critics refusing to accept the answer.
What seems to have happened in comments here is a scientists gave what appears to be an answer so flawed that people of good faith could easily consider it to be flat out wrong. Critics refuse to accept the answer given by Gavin– a scientist– because the asnwer appears flat out wrong. People who support Gavin are suggesting the critics refusal to accept the answer somehow reflects badly on the critics. We await to see if Gavin returns to explain why his critics should not consider his answer to willis point (b) either flat out wrong or at best, highly misleading. Because currently, Gavin’s claim appears to be contradicted by the evidence he gave to support it.
A month later (July 22), Tamino inadvertently revived the unresolved dispute, by citing the Mann 2008 no-dendro reconstruction in his attack on Andrew Montford’s Hockey Stick Illusion as follows:
As a great deal of other research has shown, you can even reconstruct past temperature without bristlecone pine tree rings, or without any tree ring data at all, [linking to the RC post on Mann et al 2008 here ] resulting in: a hockey stick.
This was not the first time that a realclimate post had invoked the Mann et al 2008 no-dendro stick. They had done so in the Yamal controversy as well:
Oh. The hockey stick you get when you don’t use tree-rings at all (blue curve)? [again Citing http://www.realclimate.org/index.php/archives/2008/09/progress-in-millennial-reconstructions/%5D
Tamino’s post sparked renewed discussion of the no-Tilj no-dendro case. In response to a lengthy comment at RC by Judy Curry on July 24, Gavin pointed to a figure in the SI to “Mann et al 2008” showing a no-Tilj no-dendro CPS reconstruction – not mentioning that the figure was not part of the original article, but a correction to the website posted only in Nov 2009 – after Montford’s book was finalized other than the Climategate portion. Gavin:
[Response: Absolutely untrue in all respects. No, really, have you even read these papers? There is no PCA data reduction step used in that paper at all. And this figure shows the difference between reconstructions without any tree ring data (dark and light blue) compared to the full reconstruction (black). (This is a modified figure from the SI in Mann et al (2008) to show the impact of removing 7 questionable proxies and tree ring data together)
This figure is at Mann’s website here, where a change notice dated to November 4, 2009 says:
In the newly corrected figure, we have added the result for NH CPS without both tree-rings *and* the 7 potential “problem series.” Each of the various alternative versions where these sub-networks of proxy data have been excluded fall almost entirely within the uncertainties of the full reconstruction for at least the past 1100 years, while larger discrepancies are observed further back for the reconstruction without either tree-ring data or the 7 series in question, owing to the extreme sparseness of the resulting sub-network.The new figure can be downloaded here (PDF)
Later in the day, Gavin reiterated at RC the claim that there was “no material difference” in results with or without the Tiljander sediments.
The Tiljander stuff is moot since the Mann et al (2008) paper showed both with and without and found no material difference [ in line to 171 on 24 July Comment by D. Robinson — 24 July 2010 @ 8:16 AM]
On July 25, I responded here to some of the issues in Tamino’s post, noting once again the circularity of the original no-dendro no-Tilj argument in an inline comment. The point continued to be contested through the thread, including the following inline comment to Phil Clarke on July 28 ( who was also commenting at realclimate):
In November 2009, just before Climategate, Mann placed a non-Tiljander non-dendro reconstruction on his website. He did not issue a Corrigendum at PNAS nor did he publish a notice of the new information at realclimate. That Mann did so in late 2009 long after the fact did not refute the claim in respect to Mann et al PNAS 2008. It’s very misleading for Gavin to pretend that a website addition in November 2009 was part of the corpus of Mann et al 2008, that should have been considered in CA commentary on Mann et al 2008 in late 2008 (which was what MOntford was reviewing).
This comment was posted at RC by Judy Curry on July 28 occasioning a retort from Gavin who falsely accused Montford of attributing motives to Mann’s failure to present a no-Tilj no-dendro combination (Montford had observed the circularity and had not speculated as to motive). In his inline comment, Schmidt made what, to my knowledge, is the first public notice that the Mann et al 2009 (Science) SI (not Mann et al 2008 PNAS) had reported that the no-dendro reconstruction without the contaminated sediments did not verify prior to 1500. In other words, announcing the demise of the much vaunted “validated” no-dendro reconstruction back for 1300 years. Gavin:
[Response: Pure spin. The additional graph was posted because of inaccurate claims that there was something wrong with the no-dendro reconstruction because of the inclusion of the already-acknowledged-to-be-problematic Tiljander proxies. The sensitivity studies in the original paper didn’t include that the no-dendro/no-Tiljander combination but that does not justify the claims made by Montford that such a combination was impossible or was not included because it undermined the results. Indeed, you can do a no-dendro and no-Tiljander reconstruction with the code that was posted with Mann et al (2008), and that was what was added to the figure I showed. Montford was apparently happy to make up results and conclusions in late 2008 that were just not justified, and for this you give him a pass? Curious. For further information, the no-dendro/no-Tiljander sensitivity test is also part of the SI in Mann et al (2009) (figure S8), where it is noted that it doesn’t validate prior to 1500 AD. Of course if you remove all data that is imperfect, you will end up with no results. But as Salzer et al point out, there is likely to be useful climate information in the tree rings so I wouldn’t throw them out unnecessarily. – gavin]
The bolded sentence should have caught everyone’s attention, but it was mixed in with a number of other contentious issues and passed without further comment.
[Update: On July 30, I responded to Gavin Schmidt’s reference to the CPS reconstruction at Mann’s website in a lengthy CA post here, presenting my own estimate of the no-Tilj no-bristle CPS reconstruction using Mann 2008 methods on the AD1000 proxy network:
This differed dramatically from the image at Mann’s website. I postulated that the difference arose from Luterbacher gridded data as follows:
Mann’s graphics all show the results of spliced reconstructions rather than what you get with proxies going back to AD1000. The provenance of the network used in Mann’s November 2009 revision of a figure in his SI isn’t described as clearly as it might be. My interpretation of the figure is that the network includes 71 Luterbacher gridded European series which use instrumental temperature data. It is my surmise that in its latter portion, the stick-ness of the “new” no_tilj no-dendro reconstruction derives from splicing the Luterbacher gridcell data (using instrumental data) onto the horrible no-dendro reconstruction.
The next day, Phil Clarke re-capped (rather disparagingly) my analysis of the “frozen” AD1000 network at RC as follows:
His contention being that in the various sensitivity tests the combinations were carefully chosen to leave a mix that gave a Hockey Stick. Remove Bristlecones but leave in the inverted Tiljander, exclude Tiljander but add the Bristlecones back in etc. I read this post as an attempt to demonstrate this by rolling his own No-dendro, no-Tilj version that does not resemble an item of sporting equipment. No modern instrumental temperatures are plotted so this is a comparison of the ‘shaft’ only.
His surmise about why his reconstruction differs from the Mann figure (some commenters have done an overlay and the differences are really not that remarkable, to my eyes) is beyond my ability to parse. To do so seems to rely on having read previous posts where terms are defined. Fair enough, but two points are clear – the two warm periods reconstructed – Medieval and late 1700s are >0.6C cooler than recent NH anomalies of around 1C CRU], which means that while the details differ, McIntyre’s plot is fully consistent with the conclusion of Mann et al 2008 that recent warmth is unprecedented for 1,000 years or more.
Gavin’s inline response to Phil Clarke was the second mention that the Mann et al 2009 SI had admitted the “validation” failure of the no-dendro reconstruction prior to 1500:
[Response: It’s also worth spelling out some of McIntyre’s thimble hiding here. First off, after a 7 years you’d think that he would be aware that the reconstructions are done in a step-wise fashion – i.e. you use as much information as is available as far back as you can. Back to 1500 you use everything that goes back that far, back to 1400 a little less etc. So a proper no-dendro/no-Tijl reconstruction will not just be made with what is available in 1000AD. Second, given all of the bluster about validation statistics, he never seems to compute any. Since the no-dendro CPS version only validates until 1500 AD (Mann et al (2008) ), it is hardly likely that the no-dendro/no-Tijl CPS version will validate any further back, so criticising how bad the 1000 AD network is using CPS is hardly germane. Note too that while the EIV no-dendro version does validate to 1000 AD, the no-dendro/no-Tijl only works going back to 1500 AD (Mann et al, 2009, SI). So again, McIntyre is setting up a strawman, not performing any ‘due diligence’ and simply making stuff up – all in order to demonstrate some statistical prestidigitation to the adoring commenters. – gavin]
This time, the point wasn’t missed. A few minutes later,(529 July 31) Nicolas Nierenberg asked Gavin to confirm the surprising information that the no-dendro reconstruction did not validate prior to AD1500:
Gavin, So just to be clear with regard to your response to 525. Under either method (CPS or EIV) it is not possible to get a validated reconstruction to before 1500 without the use of tree rings, or the Tijlander sediments. I understand, of course, that as you remove proxies that the ability to project backward will naturally diminish.
[Response: That appears to be the case with the Mann et al 2008 network. Whether you can say more general things about medieval times using these and other proxies (cf osborn and briffa 2006) is another question. -gavin]
Read that again slowly:
Under either method (CPS or EIV) it is not possible to get a validated reconstruction to before 1500 without the use of tree rings, or the Tijlander sediments.
How many times had Gavin and others said that Mann’s use of Tiljander sediments didn’t “matter”? And now we learn that, without the contaminated sediments, 800 years of “validation” are eliminated. Gavin petulantly tried to close off the issue by saying said that “the exact level of the medieval warmth is not a very interesting scientific question”, not the position that was taken in September 2008 when they were issuing press releases about the no-dendro reconstruction.
Mann et al 2009 (Science) was published in November 2009, just before Climategate, and hasn’t been discussed here. In a way, I’d sort of presumed (prematurely, it seems) that people had stopped taking this sort of article in Science (or Nature) seriously.
There was nothing in the text of Mann et al 2009 that stated or even hinted that claims in Mann et al 2008 on the validation of their non-dendro reconstruction were conceded to be no longer valid. Nor did they issue a Corrigendum for Mann et al 2008 at PNAS where the no-dendro claim had actually been made. Nor was the withdrawal of the claim to have a 1300-year validated no-dendro reconstruction reported at the Mann et al 2008 website. Nor were there any press releases withdrawing the claim of a “validated” no-dendro reconstruction with equal prominence to the original press release. However buried in the SI to Mann et al 2009 at Mann’s website was the following admission (with a similar caption to their SI Figure S8):
In addition to the tests described by ref. S1 which removed alternatively (a) all tree-ring data or (b) 7 additional long-term proxy records associated with greater uncertainties or potential documented biases (showing the temperature reconstruction was robust to removal of either of these datasets), we here removed both data sets simultaneously from the predictor network (Fig. S8). This additional test reveals that with the resulting extremely sparse proxy network in earlier centuries, a skillful reconstruction is no longer possible prior to AD 1500. Nonetheless, even in this case, the resulting (unskillful) early reconstruction remains almost entirely within the estimated error bounds of the original reconstruction.
Ironically, attention to the no-dendro reconstruction was revived because of attacks by Tamino and Gavin Schmidt on Andrew Montford’s summary of the original CA discussion of Mann et al 2008 and the Tiljander sediments, a summary that is worth re-reading in light of recent admissions:
It turned out that the twentieth century uptick in Tiljander’s proxies was caused by artificial disturbance of the sediment caused by ditch digging rather than anything climatic. Mann had acknowledged this fact, but then, extraordinarily, rather than reject the series, he had purported to demonstrate that the disturbance didn’t matter. The way he had done this was to perform a sensitivity analysis, showing that you still got a hockey stick without the Tiljander proxies.
Great care is needed when reading scientific papers, particularly in the field of paleoclimate, and this was one of the occasions when one could have come away with an entirely wrong impression if the closest attention had not been paid. The big selling point of Mann’s new paper was that you could get a hockey stick shape without tree rings. However, this claim turned out to rest on a circular argument. Mann had shown that the Tiljander proxies were valid by removing them from the database and showing that you still got a hockey stick. However, when he did this test, the hockey stick shape of the final reconstruction came from the bristlecones. Then he argued that he could remove the tree ring proxies (including the bristlecones) and still get a hockey stick – and of course he could, because in this case the hockey stick shape came from the Tiljander proxies. His arguments therefore rested on having two sets of flawed proxies in the database, but only removing one at a time. He could then argue that he still got a hockey stick either way.
As McIntyre said, you had to watch the pea under the thimble.