Climate Audit

We (McIntyre and McKitrick) are profiled in the cover story of the Feb. 1, 2005 edition of Natuurwetenschap & Techniek (NWT), a prominent European science magazine (both Dutch and English versions at www.natutech.nl ). The cover story is based on two new peer-reviewed papers being published in the well-known science journals Geophysical Research Letters and Environment and Energy.

Our article “Hockey Sticks, Principal Components and Spurious Significance” has been accepted for publication in Geophysical Research Letters, copyright 2005 American Geophysical Union (doi: 2004GL012750). A pre-publication version is at http://www.climate2003.com/pdfs/2004GL012750.pdf. Further reproduction or electronic distribution is not permitted.

Our article “The M&M Critique of the MBH98 Northern Hemisphere Climate index: Update and Implications” has been accepted for publication by Energy & Environment and is available at http://www.multi-science.co.uk/mcintyre-mckitrick.pdf.

Our research shows fundamental flaws in the “hockey stick graph” used by the Intergovernmental Panel on Climate Change (IPCC) to argue that the 1990s were the warmest decade of the millennium. The original hockey stick study was published by Michael Mann of the University of Virginia and his coauthors Raymond Bradley and Malcolm Hughes. The main error affects a step called principal component analysis (PCA). We showed that the PCA method as used by Mann et al. effectively mines a data set for hockey stick patterns. Even from meaningless random data (red noise), it nearly always produces a hockey stick.

This “backgrounder” provides a road map and summary of the 3 articles. While these papers have been under review, Mann et al. have opened up their own weblog realclimate.org and criticized some of our earlier work. We include some comments here on this commentary and some FAQ.

Natuurwetenschap & Techniek

The NWT article presents a history of our original interest in the problem, proceeding through our first analysis of MBH98 in 2003, the responses by Mann et al., our counter-responses and so on up to the present. As part of their due diligence for the article, they showed our research to referees of their choosing for detailed assessment.

One was Dr. Rob van Dorland, IPCC Lead Author and climate scientist at the Dutch National Meteorological Agency. In the NWT article he is quoted saying it will “seriously damage the image of the IPCC.” He added: “It is strange that the climate reconstruction of Mann has passed both peer review rounds of the IPCC without anyone ever really having checked it. I think this issue will be on the agenda of the next IPCC meeting in Peking this May.”

Another was Dr Mia Hubert, a statistician at the Katholieke Universiteit Leuven in Belgium. She also agreed with our results, observing: “Tree rings with a hockey stick shape dominate the PCA with this method.” Professor Hans von Storch, an IPCC Contributing Author and internationally-renowned expert in climate statistics at the Institute for Coastal Research in Geesthacht, Germany, concurred, calling McIntyre and McKitrick’s criticism on this point “entirely valid.”

Geophysical Research Letters

Our article, “Hockey Sticks, Principal Components and Spurious Significance” has been accepted for publication in Geophysical Research Letters, copyright 2005 American Geophysical Union (doi: 2004GL012750). Further reproduction or electronic distribution is not permitted.

This article identifies what is almost certainly a computer programming error in the principal components method used in MBH98. The error causes their PC method to nearly always identify hockey stick shaped series as the “dominant pattern” in a data set (the so-called “first Principal Component” or PC1), even when the data are just random numbers. We carried out 10,000 simulations in which we fed “red noise”, a form of trendless random numbers, into the MBH98 algorithm and, in over 99% of the cases, it produced hockey stick shaped PC1 series. The figure below shows 3 simulated PC1s and the MBH98 reconstruction: can you pick out the reconstruction?

Figure 1. Three simulated PC1s and the MBH98 reconstruction.

The GRL article also examines the statistical significance of the MBH98 reconstruction in the controversial 15th century step. Although the Reduction of Error (RE) statistic is sometimes said to be “preferred” by paleoclimatologists, we show that it is necessary to look at more than one statistic before coming to a conclusion of statistical significance, because these simulated hockey sticks tend to have spuriously high RE statistics when compared to Northern Hemisphere temperatures during the calibration and verification periods of MBH98. The original article reported only one of the usual verification statistics used in paleoclimate studies. Mann et al. have not archived supporting data for this step and have refused to provide supporting calculations for this step. However, we report on emulations of MBH98, which indicate that the R2 and other verification statistics for MBH98 lack statistical significance. Using more accurate benchmarks for the RE statistics under the actual conditions of MBH98, we show that the MBH98 reconstruction lacks statistical significance in its early portion.

Elsewhere Mann et al. have insisted that climate reconstructions must pass a number of statistical verification tests or else they should not be considered. Applying that criterion, the MBH98 hockey stick graph should henceforth be considered invalid.

The computer script used to generate the simulations, figures and statistics in the GRL article has been archived with GRL, together with examples of hockey stick shaped PC1s. A mirror is located here. 100 example simulated PC1s are here.

Energy & Environment

The Energy & Environment article, “the M&M Critique of the MBH98 Northern Hemisphere climate Index: Update and Implications” examines the effect of two methodological changes on the Northern Hemisphere climate index of MBH98: using the version of a Gaspé tree ring series as archived by the originating researchers rather than as emended by MBH; and using a conventional (centered) principal components methodology rather than the nonstandard methodology applied in MBH98. Our emulation methodology fully implements all the details reported in the July 2004 Corrigendum by Mann et al., including the actual steps for their stepwise principal components calculation and the gridcell standard deviations from the (now obsolete) HadCRU data previously unavailable. The result is, as we had found in our 2003 study, relatively high early 15th century values as shown in Figure 2 below.

Figure 2. Northern Hemisphere temperature reconstructions: MBH98 and MM05 emulation of MBH98 using (a) centered principal components; (b) archived version of Gaspé series. Both series smoothed.

The E&E article canvasses various permutations and combinations of methodology now advocated by MBH98 in an attempt to avoid our criticism, showing how slight variations in methodology sometimes lead to high early 15th century results and sometimes to low early 15th century results. For example, the presence or absence of the PC4 in centered calculations determines early 15th century results. We point out that this is inconsistent with MBH98 claims that their method is “robust” to the presence or absence of dendroclimatic indicators.

The common factor in the bifurcation of 15th century temperatures is traced to the effect of bristlecone pines. There is an undoubted growth pulse for a small network of bristlecone pines in the Western USA in the late 19th and 20th centuries, and their ring width “chronologies” thus have a hockey stick shape. While these series represent only a small fraction of the MBH98 data base the resulting hockey stick shape is essentially an imprint of this bristlecone pine growth pulse. If world climate history is held to be uniquely characterized by the growth of bristlecone pines, we think that there should be a clear exposition of all the issues involved. We survey specialist literature on bristlecone pines and show that the original authors of the sites that dominate the MBH98 PC1 stated clearly that the 20th century growth pulse was not due to temperature. Even Mann’s co-author, Hughes, has stated in print that the bristlecone pine growth pulse is a “mystery”.

MBH98 data mining methods resulted in the selection of these hockey stick shaped series into the PC1, thereby ranking them as the “dominant pattern of variability”, whereas they appear in neither the PC1 nor the PC2 under a centered calculation. Regardless of the role of the data mining method, it is important to recognize and fully assess the role of these series in imprinting a hockey stick shape and to completely assess whether they are the correct way of measuring world climate history.

The computer script used to generate the figures and statistics in the E&E here will be located here [in a couple of days].

Frequently Asked Questions (FAQ)

Responses to comments made by Mann, Bradley and Hughes