Climate Audit

Jones et al. [1998] makes the reasonable policy that proxies should be validated against gridcell temperatures as evidence that they are temperature proxies, noting that this is not always done. This policy is endorsed in Jones and Mann [2004], who note that not all multiproxy studies had observed this policy, presumably including MBH98, which included precipitation measurements in the dataset.

As an exercise, I attempted to replicate their results, as reported in their Table 4, against the present HadCRU2 gridcell temperatures. A few points are illustrated in the replication tables – one for their annual series and one for the decadal averages. First, many of their reported results cannot be replicated, including the correlations for some well-known proxies. Second, I’ve done calculations to yield a t-statistic, which is easier to determine significance than a correlation statistic. For independent residuals, a t-statistic of about 2 is significant. For autocorrelated series and for cherrypicked series, it’s higher, but I’ve not done those calculations here, contenting myself with illustrating versus OLS significance. I’ve marked in bold red examples where the t-statistic is insignificant or where there are big declines in replication versus actual. Also I’ve calculated the Durbin-Watson statistic, which is one measure of autocorrelation in residuals, used to test for spurious regression. Granger and Newbold [1974] discussed here took the position that any model with a DW<1.5 is misspecified. These are also marked in bold red. Not much survives.

This is not the exact same dataset and the verification exercise is, to that extent, not a direct audit of their calculation against the temperature data set which they used – which is not necessarily available anyway. (I’ve experimented with the vintage temperature dataset archived with the MBH98 Corrigendum and can’t replicate results with that dataset, but that’s a different topic.) Scripte here.

Jones et al [1998] show correlations for the 1881-1980 period for both annual and decadally averaged data (his Table 4). Here I’ve extended the comparison to 1856-1994 for annual data and 1860-1989 for decadally averaged data, using all available data. It is trite that the correlation coefficient is equal to the square root of the R2 statistic of the corresponding linear regression model. (I’ve attached a script for all calculations and benchmark this for the specific functions that I used.) The advantage of using the linear regression approach is that you obtain standard errors and a t-statistic for which you can determine statistical significance. This is particularly relevant for the decadal correlations, where you only have about 10 measurements. I’ve not elaborately considered the issue of autocorrelation although this is always an issue. Statistics shown here are OLS statistics. I have shown the Durbin-Watson statistic for the fit.

Table 1 compares the reported results with my verifications. In 7 cases, the verification correlation was substantially lower than reported: two Briffa series – Urals, Western US density; Galapagos coral, two Villalba South American tree chronologies (Lenca, Alerce). Seven series had insignificant t-statistics (5 of 7 SH series) and 2 NH series: Kameda Greenland melt % and Jacoby treeline. Eight series had Durbin-Watson statistics less than 1.5, indicating autocorrelation in the residuals (per Granger and Newbold [1974], an unusable model : Tornetrask, New Caledonia, Greenland O18, Svalbard melt %, Galapagos, Lenca, Kameda melt and Jacoby treeline.
The only proxies which had a significant t-statistic and passed the DW test were: the C Europe and C England "proxies" and the Jasper and Tasmania tree ring reconstructions.

Table 1. Correlation Statistics for Jones et al [1998] Proxies

Table 2 shows the same information for the decadally averaged versions. There is an ongoing effort by paleoclimatologists to try to extract "low-frequency" information from proxies. There’s nothing wrong with this, but the smoothing ends up reducing the degrees of freedom so much that it is important to use more sophisticated statistics than a correlation coefficient to see what’s going on. The methodology here is as in the first table.

A number of series had substantially lower decadal correlations than reported: the most notable was the Polar Urals dataset which went from a reported 0.92 to minus 0.2 (!); four series had negative correlations and 8 series had insignificant t-statistics. Here the impact of the number of measurements is felt: for example, the Greenland (Crete) O18 series has a correlation of 0.49, which looks good, but the t-statistics is only 1.58. Six series had DW statistics below 1.5; 4 had DW statistics greater than 2, showing an over-correction, although there are so few values that none of these numbers mean a while lot. Again only 4 series passed both t- and DW-tests.

Table 2. Decadal Correlation Statistics for Jones et al [1998] Proxies

It should also be remembered that these series have been picked from a much larger population of series. The very act of selection means that the true t-statistic for significance is much greater than the rule of thumb of 2 [Ferson et al, 2003].