Michael Mann’s new book, The Hockey Stick and the Climate Wars, discusses my comment [SMc note - this post is by Hu McCulloch] , “Irreproducible Results in Thompson et al., ‘Abrupt Tropical Climate Change: Past and Present’ (PNAS 2006),” that was published in 2009 in Energy & Environment. My comment pertained to an article by Lonnie Thompson and eight coauthors that had appeared in the Proceedings of the National Academy of Sciences back in 2006. As soon as my comment was accepted for publication, I posted a less technical summary on Climate Audit, entitled “Irreproducible Results in PNAS” (4/24/09).
Dr. Mann’s discussion of my comment is the first published feedback to it by any of Thompson’s associates, and hence I am very grateful for the attention he has drawn to it. However, I beg to disagree with Mann’s appraisal of it.
In my comment, I show that the 2000-year tropical ice core d18O composite index shown in Thompson’s Fig. 6 and tabulated in his supplementary Data Set 3 cannot be constructed as a linear combination of the seven individual ice core series on which it was supposed to have been based, as shown and tabulated (back to A.D. 1600 only) in his Fig. 5 and Data Set 2.
I knew McCulloch’s claim that the tropical ice core composite was “irreproducible” was false, as I was able to reproduce Thompson et al.’s results easily from their raw data. I was also able to identify McCulloch’s error — an incorrect assumption that the amplitude of variation in a series of measurements must be constant in time — in about a half hour of work. [fn. 60] (pp. 205-6)
While, as I noted already in a 5/1/09 update to my CA post, I don’t doubt that Mann can construct the 400-year ice core index shown (as 5-year averages) in Thompson’s Fig. 5 and tabulated in his Data Set 2, the bottom-line decadal 2000-year composite in Thompson’s Figure 6 and Data set 3 does not correspond to decadal averages of the illustrative 400-year series shown in Figure 5 and Data Set 2. Mann may have private access to different data than was archived with Thompson’s article, and perhaps this private data will generate the 2000-year composite, but I demonstrate that this cannot be done with the data provided to the public in Data Set 2, even for its truncated time period. Until Thompson provides this data as requested in my comment, his results in PNAS are irreproducible, as claimed.
Mann’s footnote 60 (on pp. 333-4) continues,
 For those who are interested in the technical details, the matter involved the statistical concept of standardized Z scores. These are the series of numbers that result from taking an original dataset, subtracting off the average, and dividing by the standard deviation. This yields a convenient new version of the dataset whose average is zero and standard deviation is equal to one. That latter property only holds for the full dataset. If one takes some subset of the data, the average will not in general be zero and the standard deviation will not in general be one. McCulloch’s error essentially amounts to having assumed that the original Z scores defined by Thompson and colleagues, over the full data interval A.D. 0-2000, would still have average zero and standard deviation of one over a much shorter interval of A.D. 1610-1970. Because of this error, McCulloch ended up using a different weighted average of ice core data than the simple uniform weighted average used in the original Thompson et al. paper, which was the only reason he was unable to reproduce the Thompson et al. result. The error was all McCulloch’s.
In fact, in my comment, I did find with a least squares regression that the tropical composite Z-score index in Data Set 3 is simply the average of the two regional composite Z-scores (for the Andes and Himalayas) in the same Data Set, to within rounding error. This led me to suspect that the regional composites were similarly computed by averaging Z-scores for the individual sites in the two regions, as I noted on p. 371 of my comment. Other possibilities were that he more logically averaged the raw d18O series before taking regional Z-scores, or that he weighted them using multivariate Ordinary Least Squares, multivariate Classical Calibration Estimation, or even Partial Least Squares as in Thompson’s recent article with Kinnard et al. in Nature (24 Nov. 2011). But all these methods produce fixed weights that would have shown up as a virtually exact fit in my regressions.
Although I did not have Mann’s access to the full 2000-year source data, and therefore could not compute the full-sample means or standard deviations as he did, the net coefficients should have been readily recoverable with an almost exactly fitting least squares regression using the 37 decades for which all cores had data in Data Set 2. In fact, the regression standard deviations were 100 times the expected rounding error for the Himalayan region, and 30 times the expected rounding error for the Andean region, using the best fitting coefficients. If Dr. Mann can find any exact fit to within rounding error, I hope he will share it with us.
Most of the sites in question have multiple cores, so in all likelihood Thompson simply used a different combination of cores for the 400-year series in Figure 5 than for the 2000-year series in Figure 6. Undoubtedly these cores vary in quality, and hence Thompson may have had valid reasons not to include all of them. However, if the subset used in Figure 5 is correct, then the index in Figure 6 is wrong, and vice-versa. Either way, the bottom-line 2000-year index in Figure 6 cannot be replicated, even over its last 400 years, from the data represented as its source in Figure 5.
If all of Thompson’s cores had been fully archived at the NCDC’s website, as is customary for this sort of data, it would be feasible to back out which ones he used by a least squares regression on decadal averages of all of them, region by region. The excluded ones would then simply show a zero coefficient. However, with the notable exception of Quelccaya, Thompson has kept most of this primarily NSF-funded data to himself (and a few select friends, apparently), for one, two, or even three decades after it was collected. If he does not release definitive versions of this valuable data soon, it may be lost to science forever.
Mann states that
in 2009 … an economist and climate change contrarian named Hu McCulloch alleged that he could not reproduce a tropical ice core record produced by Lonnie Thompson and colleagues, implicitly claiming either ineptitude or, worse, malfeasance. (p. 205)
Please note that “ineptitude” and “malfeasance” are Mann’s words, not mine. I merely claimed that Thompson’s results were irreproducible. Thompson and his colleagues could have easily corrected this problem, simply by providing the data from which his results were actually calculated and explaining how the calculation was done.
Mann concludes his footnote 60 as follows:
How McCulloch and all reviewers of the paper could have missed something as basic as this is rather bewildering, even more so since McCulloch could have simply walked over to the other side of campus to ask Thompson; they’re both faculty members at Ohio State University.
I naturally first attempted to contact Thompson directly in order to clear up the inconsistency in his data. After two voicemail messages went unanswered, I e-mailed Thompson, together with most of his coauthors, on Jan. 23, Jan. 26, and Feb. 6, 2008, but received no reply from any of them, as Mann would be aware if he had read p. 368 of my comment. Thompson himself undoubtedly has a very busy schedule and might have been traveling. However, the odds that all five of the addressees were out of town simultaneously were very slim. Pending a reply, I proceeded to send a one-paragraph letter to PNAS, with a link to my full paper as Supplementary Information.
According to Mann,
… McCulloch was unable to marshal a credible eough argument to publish a comment in the original journal of record (in this case, Proceedings of the National Academy of Sciences [PNAS]. (p. 205)
In fact, as I noted on p. 368 of my published comment, my letter “was rejected on the grounds that PNAS does not publish corrections to articles more than three months old.” In other words, any errors published in PNAS that have gone unchallenged for more than three months become part of the scientific canon as far as PNAS is concerned! Two climate journals then did reject my comment on the reasonable grounds that they did not publish the original paper. In the end, Energy and Environment kindly agreed to publish it.
Mann curiously refers to me in his discussion of my E&E comment by the informal screen name I used in my popularized Climate Audit post, rather than by the more formal J. Huston McCulloch that appeared on my complete E&E paper. Even in citing my comment in his note 59, he gives only the initial H. rather than J.H. While it is commendable that Mann reads Climate Audit, this gives one the impression that perhaps he did not actually refer to the published version of my comment, but only to the CA post, which omitted the actual equations I estimated and which were the heart of my technical argument. This may be why he did not understand my procedure.
Mann remarks (p. 205),
[Thompson’s] tropical ice core data provided independent support for the conclusion that modern warming was unprecedented for the past two thousand years and were featured in An Inconvenient Truth, making them a particularly tasty target for deniers.
In fact, Mann should know that Thompson's ice core data did not actually appear in Gore's AIT: The series Gore identified as “Dr. Thompson’s Thermometer” was in fact Mann’s own Hockey Stick, spliced together with the recent instrumental record so as to make them appear to be a single series. See Al Gore and “Dr thompson’s Thermometer” #2 (CA 11/10/07). During the question period after an OSU seminar on Jan 11, 2008, Thompson, who had been an official Scientific Advisor on the AIT project, admitted this error. I then challenged Thompson to correct this error with a publicly accessible statement, but he still has not done so to the best of my knowledge. See “Gore Scientific “Adviser” says that he has “no responsibility” for AIT errors”.
It should be mentioned that even if AIT had presented actual Thompson ice core data, it would not truly have been independent of the Hockey Stick as claimed by Mann, since one of the 12 proxies used in the crucial AD1000-1400 segment of the Hockey Stick was Thompson’s Quelccaya d18O record. This record happens to be one of the strongest of these 12 contributors to the Hockey Stick shape, after the disputed treering-based PC1.
Furthermore, Thompson has never actually calibrated his ice core indices to temperature except by eyeball, so that there is no statistical basis for Mann’s claim that they provide evidence of warming. I have attempted to correct this deficiency in a working paper I discuss in my 12/10/09 CA post, Calibrating “Dr. Thompson’s Thermometer”. I conclude, however,
It may be seen from Fig. 4 that “Dr. Thompson’s Thermometer” is in fact completely uninformative about the existence or absence of a Medieval Warm Period (MWP), Al Gore to the contrary notwithstanding. Temperatures throughout the period 1000-1990 could have been as high as 1.2° C warmer than 1961-90 or as low as 1.8° C colder. The estimates for the 1990s are considerably tighter because of the highly significant slope coefficient for LCZ4, but even that decade has a 95% CI of (-0.32, 1.75) °C.
The Steig et al. Corrigendum Affair
Mann’s new book goes on to call renewed attention to another 2009 controversy: In January 2009, Eric Steig and 5 co-authors, including Dr. Mann himself, wrote an article in Nature on Antarctic warming that attracted considerable attention on Climate Audit. On Feb. 26, 2009, I wrote a post here, entitled “Steig 2009’s Non-correction for Serial Correlation”, which showed that Steig and coauthors had failed to correct the standard errors of their temperature trend lines for serial correlation, and that when this was done their results were greatly weakened, though not reversed.
Since Nature has a policy of publishing corrections and comments “if and only if the author provides compelling evidence that a major claim of the original paper was incorrect,” and this correction did not in itself overturn their key result, I did not submit my comment to Nature, and only published it informally on CA instead. On Feb. 28, I alerted Steig and all of his coauthors by email of my post, inviting them to participate in the discussion.
None of the authors ever replied or participated in the CA thread, but on Aug. 6, 2009, they published a “Corrigendum” in Nature making essentially the same point I had made several months before in my CA post. See The Steig Corrigendum for discussion. A graph there by Roman Mureika shows that the portion of the continent that shows significant warming is greatly reduced when the correction is made.
On Aug. 6, I e-mailed the editors of Nature, reminding them that according to their Editorial Policies, “Plagiarism is when an author attempts to pass off someone else’s work as his or her own,” and requesting that they retract the Steig et al. Corrigendum and replace it with the short e-mail I had sent them, which contained the URL of my CA post.
Steig then wrote to Nature editor Michael White that he was in the field in Antarctica and not receiving e-mail when I had written him, that he was unaware of my post, and that he had not read it. White accepted Steig’s explanation, so I withdrew my complaint. See “Steig Professes Ignorance”.
Mann now writes (n. 61, p. 334),
McCulloch complained that Steig had appropriated his own finding. Yet it is self-evident that Steig et al. were aware of the need for the autocorrelation correction, since the paper explicitly stated (albeit, it turns out, in error) that it had been made. Had McCulloch notified Steig of the error when he first discovered it, or had he submitted a formal comment to Nature identifying the error, he would have received credit and acknowledgement. He chose, however, to do neither of these things. [Emphasis added]
In fact, I had attempted to notify not only Steig, but all 5 of his co-authors, including Dr. Mann himself, of my CA post. Mann himself is off the hook, assuming he does not read CA, because I received an automated e-mail from his computer indicating he was out of town and unlikely to read any e-mails that were not resent after his return. Steig’s computer did send me an automatic e-mail indicating he was in Antarctica until mid-March with no regular e-mail access. However, the message gave no indication that he would not read messages on his return. I have no reason to believe that any of the other four authors — David Schneider, Scott Rutherford, Josefino Comiso, or Drew Shindell — did not receive my message. All were co-authors of the Corrigendum, and hence all were responsible for its content. Even if all overlooked my e-mail, it is difficult to believe that none of their colleagues mentioned the CA post to them.
Furthermore, I did not submit my correction to Nature because of their explicit policy that they would not publish a correction that did not materially alter the conclusions of the original article. Evidently Nature relaxed this rule for Dr. Mann and his colleagues, but I naturally would rather this publication in Nature, however brief, had appeared on my vita rather than theirs.
In the end, the autocorrelation issue turned out to be the least of the original paper’s problems: Ryan O’Donnell, Micholas Lewis, Steve McIntyre and Jeff Condon (J. Climate April 2011, 24:2099-2115) have shown that the main results of the paper are dependent on oversmoothing that results from retaining too few principal components of the satellite covariance matrix. They find Antarctic warming to be concentrated in the Peninsula rather than spread throught West Antarctica as in the Nature paper. Furthermore, average trends are less than half what Steig et al. found for the entire continent, East Antarctica, and West Antarctica, yet were much stronger than what they found in the Peninsula itself. See O’Donnell et al 2010 Refutes Steig et al 2009.
Although Mann has much to say in his new book, he neglects to make any mention at all, that I can find, of the O’Donnell et al. refutation of his well-publicized paper with Steig et al.
As noted in “Thompson gets new NSF grant”, Thompson has in the last 8 months archived decadal data for Guliya and two Puruogangri cores back to 0AD, along with some previously unarchived annual data for Dasuopu.
However, in order to replicate the 2000-year index in Figure 6 of the PNAS paper, it would still be necessary to have the Sajama and Husacaran decadal data that was used back to 0AD. In addition, Thompson’s decadal data on either Dasuopu or Dunde must go back at least to 450 AD, yet both have only been archived back to 1000 AD.
With the newly archived Puruogangri data, plus a privately circulated spreadsheet with decadal data for Sajama and Huascaran back to 1000AD, it may now be possible to replicate at least the last 1000 years of the PNAS 06 2000-year index, though I have not yet had time to attempt that.