Last week, Mann et al published a letter in the Nature Correspondence section saying that it was "hard to imagine how much more explicit" they could have been about the uncertainties and blaming "poor communication by others" for the "subsequent confusion", disucssed here. The Mann et al letter is absurd and Ross and I decided to submit a short reply to Nature Correspondence, shown below together with Nature’s rejection.
The Steve and Ross Letter
In their recent correspondence, (Nature, 442, 627, 2006) Mann et al. claim that "it is hard to imagine how much more explicit we could have been about the uncertainties in the reconstruction" (Nature, 392, 779-787, 1998). In fact, it is not hard at all. They could have disclosed and explicitly discussed the lack of statistical significance of the verification r2 statistic for reconstruction steps prior to 1750, values of which were approximately 0 (S. McIntyre and R.McKitrick, GRL, 32, doi:10.1029/2004GL0217502005, 2005; E. Wahl and C. Ammann, Clim. Chg, accepted, 2006). Such disclosure would have shown that the uncertainties of their reconstruction were substantially underestimated, as the National Academy of Sciences panel recently concluded (p. 107).
Mann et al blame "poor communication by others" for "subsequent confusion about uncertainties", but ignore the fact that Mann was a lead author of chapter 2 of the IPCC Third Assessment Report, which stated that the Mann et al. reconstruction had "significant skill in independent cross-validation tests," without mentioning the verification statistic failures. They likewise ignore their own press releases, issued by the University of Massachusetts, and contemporary press articles linked at Mann’s website, which set the overconfident tone they now apparently regret. There is no evidence that Mann et al made any effort to correct these "poor communications" either at the time or subsequently.
Nature itself must share blame for the length of time it took to identify these statistical failures. In 2003, after Mann et al had refused to provide to us either the test scores, residual series or even the results of the individual steps for independent statistical verification, we filed a Materials Complaint with Nature requesting this data. Nature refused to intervene, saying that disclosure was up to the original authors. Perhaps this experience will encourage Nature to re-consider such policies.
Stephen McIntyre and Ross McKitrick
Within a few days, we received the following rejection letter from Maxine Clark, Publishing Executive Editor:
Thank you for your Correspondence submission, which I regret we cannot offer to publish. The paragraph about "poor communication" does not add substantially to our news report, to which Mann et al. were responding in their letter. Nature has no connection to the press releases you describe. Perhaps you would prefer to communicate directly with Mann et al. on your opinion on these matters, as you letter has more of a tone of a complaint about the authors than of something that Nature readers would find of interest?
Your last paragraph is confusing to us because Nature has already published a correction to this paper containing Supplementary Data, as part of the complaint you initiated. This has closed the matter so far as we are concerned.
I suppose that I should have known that it was foolish to expect Nature to give a vestige of equal treatment when it came to climate science. But you keep hoping. I went the following reply:
Your publication of the letter by Mann et al and rejection of our letter leaves a very uneven record of this particular controversy, which has attracted considerable attention even since the National Academy of Sciences panel report, including a further report criticizing Mann et al by senior statisticans (Wegman et al.) and two hearings by the U.S. House Energy and Commerce Committee. The response by Mann et al to your news report contained an absurd characterization of the quality of their original disclosure. That you rejected a rebuttal to this absurd characterization does not add to Nature’s credibility in this matter.
Your characterization of the situation in respect to Supplementary Data is also completely inaccurate. The Supplementary Information to the Mann et al Corrigendum did not contain results for the individual steps, that I had originally requested in 2003. On Aug 10, 2004, I re-iterated the request for this information with Karl Ziemelis, who took it up with the Editor. In Sept 2004, Nature refused to provide the information in the following terms: "And with regard to the additional experimental results that you request, our view is that this too goes beyond an obligation on the part of the authors". While the publication of the July 2004 Corrigendum may have "closed the matter as far as you are concerned", please do not confuse this with actually providing this particular and essential data, which remains undisclosed to this day.
Update (later that day): I received the following reply from Maxine Clark of Nature:
Thanks for your message. These issues are not appropriate for publication in Correspondence. From what you say below, and from looking into this matter myself before writing to you earlier, your previous complaint has been dealt with, even though you were not satisfied by the outcome. If you feel there is more to be said about that, please take the matter up with the editors who were handing your complaint.
Our News story reported the criticisms of Mann et al. Nature readers are well aware, via this and previous articles, that this is a controversial topic. Your letter serves mainly to express your negative opinion of these authors and also concerns a technical interpretation of the data or availability of the data. There are channels for you to take up these matters, but Correspondence is not appropriate, nor is the type of language you use in your letter about these authors and about Nature in general appropriate for publication. Incidentally, I don’t appreciate the tone of the comments you make to me personally, either, which are uncalled-for.
Update – May 3, 2007
On August 23, 2007, we re-submitted our comment so that the language tracked the MBH language as closely as possible. The MBH letter accepted for publication read as follows:
Your News story “Academy affirms hockey-stick graph”‘? (Nature 441, 1032; 2006) states that the US National Academy of Sciences (NAS) panel “concluded that systematic uncertainties in climate records from before 1600 were not communicated as clearly as they could have been”. This conclusion is not stated in the NAS report itself, but formed part of the remarks made by Gerald North, the NAS committee chair, at the press conference announcing the report.
The name of our paper is “Northern Hemisphere temperatures during the past millennium: inferences, uncertainties, and limitations” (Geophys. Res. Lett. 26, 759-762; 1999). In the abstract, we state: “We focus not just on the reconstructions, but on the uncertainties therein, and important caveats” and note that “expanded uncertainties prevent decisive conclusions for the period prior to AD 1400″. We conclude by stating: “more widespread high-resolution data are needed before more confident conclusions can be reached.” It is hard to imagine how much more explicit we could have been about the uncertainties in the reconstruction; indeed, that was the point of the article!
The subsequent confusion about uncertainties was the result of poor communication by others, who used our temperature reconstruction without the reservations that we had stated clearly.
Our revised letter read as follows:
Your News story “Academy affirms hockey-stick graph” (Nature 441, 1032; 2006) states that the US National Academy of Sciences (NAS) panel “more-or-less endorses the work behind the [Mann et al hockey stick] graph”. This conclusion was not stated in the NAS report itself nor by any of the panellists at the NAS panel press conference releasing the report.
Many specific findings of the NAS report contradict the claim in your story. For example, the NAS panel report stated that the Mann et al decentered principal components methodology should not be used; that temperature reconstructions should avoid the use of strip-bark bristlecones and foxtail proxies, that the Mann et al reconstruction was strongly dependent on these problematic proxies; that their reconstruction failed important verification tests; and that they had incorrectly estimated uncertainties in their reconstruction.
At the press conference, panel chairman North said that he agreed with the “substance” of the Mann et al reconstruction. However, this language is nowhere used in the report itself, where the panel expressly referred to the reconstruction merely as “plausible” and specifically withheld any attribution of confidence intervals for the period before 1600.
Jayne Hill wrote in early September declined publication stating:
Thank you for your Correspondence submission, which we regret we are unable to publish. Our news story was indeed citing North’s comments at the press conference, which as they say “substantially” support Mann et al., and which is clear from the text of the news story.
Thank you again for writing to us.
Notice the asymmetry in reasoning. MBH were allowed to comment on observations made by North on the basis that they were inconsistent with the NAS Report, but we weren’t.