Climate Audit

Continued from Part 1, Part 2

One more bit of review before we get to Ammann’s answer. As an excuse for not answering the request of the House Energy and Commerce Committee about the R2 statistic, Mann told them that his “colleagues and [himself] did not rely on this statistic” in the following terms:

The Committee inquires about the calculation of the R2 statistic for temperature reconstruction, especially for the 15th Century proxy calculations. In order to answer this question it is important to clarify that I assume that what is meant by the “R2” statistic is the squared Pearson dot-moment correlation, or r2 (i.e., the square of the simple linear correlation coefficient between two time series) over the 1856-1901 “verification” interval for our reconstruction. My colleagues and I did not rely on this statistic in our assessments of “skill” (i.e., the reliability of a statistical model, based on the ability of a statistical model to match data not used in constructing the model) because, in our view, and in the view of other reputable scientists in the field, it is not an adequate measure of “skill.” The statistic used by Mann et al. 1998, the reduction of error, or “RE” statistic, is generally favored by scientists in the field.

This sounds plausible, but it’s usually worthwhile checking what the Hockey Team says. MBH98 itself describes how they went about “not relying on” the r2 statistic as follows:

ß [RE] is a quite rigorous measure of the similarity between two variables, measuring their correspondence not only in terms of the relative departures from mean values (as does the correlation coefficient r) but also in terms of the means and absolute variance of the two series. For comparison, correlation (r) and squared-correlation (r2) statistics are also determined. Significance levels were determined for r2 from standard one-sided tables, accounting for decreased degrees of freedom owing to serial correlation

They made their lack of reliance on the verification r2 statistic very clear through the following illustration, showing the verification r2 statistics for each gridcell in the AD1820 step (which had 112 “proxies” including 12 actual temperature series as so-called “proxies” for temperature:

Original Caption: Figure 3 Spatial patterns of reconstruction statistics. … bottom, verification r2 (also based on 1854–1901 data). … For the r2 statistic, statistically insignificant values (or any gridpoints with unphysical values of correlation r < 0) are indicated in grey. The colour scale indicates values significant at the 90% (yellow), 99% (light red) and 99.9% (dark red) levels (these significance levels are slightly higher for the calibration statistics which are based on a longer period of time). A description of significance level estimation is provided in the Methods section. Continue reading →