Reconstructions that have poor validation statistics (i.e., low CE) will have correspondingly wide uncertainty bounds, and so can be seen to be unreliable in an objective way. Moreover, a CE statistic close to zero or negative suggests that the reconstruction is no better than the mean, and so its skill for time averages shorter than the validation period will be low. Some recent results reported in Table 1S of Wahl and Ammann (in press) indicate that their reconstruction, which uses the same procedure and full set of proxies used by Mann et al. (1999), gives CE values ranging from 0.103 to –0.215, depending on how far back in time the reconstruction is carried.

NAS Report (2006) page 91.

This is the NAS report, of course, that accepted our assertion that the underlying PCA method biases the shape of the results by loading too much weight on the bristlecone pine record:

As part of their statistical methods, Mann et al. used a type of principal component analysis that tends to bias the shape of the reconstructions. A description of this effect is given in Chapter 9. In practice, this method, though not recommended, does not appear to unduly influence reconstructions of hemispheric mean temperature…The more important aspect of this criticism is the issue of robustness with respect to the choice of proxies used in the reconstruction. For periods prior to the 16th century, the Mann et al. (1999) reconstruction that uses this particular principal component analysis technique is strongly dependent on data from the Great Basin region in the western United States. Such issues of robustness need to be taken into account in estimates of statistical uncertainties.

NAS report (2006) page 106-107

The data to which they are referring is the bristlecone pine record,which the panel recommended should not be used:

The possibility that increasing tree ring widths in modern times might be driven by increasing atmospheric carbon dioxide (CO2) concentrations, rather than increasing temperatures, was first proposed by LaMarche et al. (1984) for bristlecone pines (Pinus longaeva) in the White Mountains of California. In old age, these trees can assume a “stripbark” form, characterized by a band of trunk that remains alive and continues to grow after the rest of the stem has died. Such trees are sensitive to higher atmospheric CO2 concentrations (Graybill and Idso 1993), possibly because of greater water-use efficiency (Knapp et al. 2001, Bunn et al. 2003) or different carbon partitioning among tree parts (Tang et al. 1999)… ‘strip-bark’ samples should be avoided for temperature reconstructions

NAS Report page 50

Among our other findings as reported to the NSA was that the uncertainty levels of Mann’s reconstructions were underestimated. Did they find that an inaccurate claim?

Regarding metrics used in the validation step in the reconstruction exercise, two issues have been raised (McIntyre and McKitrick 2003, 2005a,b). One is that the choice of “significance level” for the reduction of error (RE) validation statistic is not appropriate. The other is that different statistics, specifically the coefficient of efficiency (CE) and the squared correlation (r2), should have been used (the various validation statistics are discussed in Chapter 9). Some of these criticisms are more relevant than others, but taken together, they are an important aspect of a more general finding of this committee, which is that uncertainties of the published reconstructions have been underestimated.

NAS Report p. 107.

And then there’s McShane and Wyner 2011; and if Mann thinks the Annals of Applied Statistics is not a peer-reviewed journal, let’s see him get published in it. Wahl and Ammann make an appearance there too:

That the null models may be too weak and the associated standard errors in papers such as Mann et al. (1998) are not wide enough has already been pointed out in the climate literature (von Storch et al., 2004). While there was some controversy surrounding the result of this paper (Wahl et al., 2006), its conclusions have been corroborated (von Storch and Zorita, 2005; von Storch et al., 2006; Lee et al., 2008; Christiansen
et al., 2009).

McShane and Wyber 2011 pp. 17-18

and

We are not the first to observe this effect. It was shown, in McIntyre and McKitrick (2005a,c), that random sequences with complex local dependence structures can predict temperatures. Their approach has been roundly dismissed in the climate science literature…Ammann andWahl (2007) claim that significance thresholds set by Monte Carlo simulations that use pseudo-proxies containing ”short term climate signal” (i.e., complex time dependence structures) are invalid:

Such thresholds thus enhance the danger of committing Type II errors (inappropriate failure to reject a null hypothesis of no climatic information for a reconstruction).

We agree that these thresholds decrease power. Still, these thresholds are the correct way to preserve the significance level. The proxy record has to be evaluated in terms of its innate ability to reconstruct historical temperatures (i.e., as opposed to its ability to ”mimic” the local time dependence structure of the temperature series). Ammann andWahl (2007) wrongly attribute reconstructive skill to the proxy record which is in fact attributable to the temperature record itself. Thus, climate scientists are overoptimistic:
the 149 year instrumental record has significant local time dependence and therefore far fewer independent degrees of freedom.

McShane and Wyner p. 19

M&S are the gentlemen who showed, using Mann’s own data, that one cannot draw strong conclusions about the relative warmth of the MWP compared to today, which was the same point we made back in 2003:

Figure 14 reveals an important concern: models that perform similarly at predicting the instrumental temperature series (as revealed by Figures 11, 12, and 13) tell very different stories about the past. Thus, insofar as one judges models by cross-validated predictive ability, one seems to have no reason to prefer the red backcast in Figure 14 to the green even though the former suggests that recent temperatures are much warmer than those observed over the past thousand years while the latter suggests they are not.

McShane and Wyner pp. 30-31.

[We] conclude unequivocally that the evidence for a ”long-handled” hockey stick (where the shaft of the hockey stick extends to the year 1000 AD) is lacking in the data. The fundamental problem is that there is a limited amount of proxy data which dates back to 1000 AD; what is available is weakly predictive of global annual temperature…Consequently, the long flat handle of the hockey stick is best understood to be a feature of regression and less a reflection of our knowledge of the truth….Climate scientists have greatly underestimated the uncertainty of proxy-based reconstructions and hence have been overconfident in their models. We have shown that time dependence in the temperature series is sufficiently strong to permit complex sequences of random numbers to forecast out-of-sample reasonably well fairly frequently (see, for example, Figure 9). Furthermore, even proxy based models with approximately the same amount of reconstructive skill (Figures 11,12, and 13), produce strikingly dissimilar historical backcasts: some of these look like hockey sticks but most do not (Figure 14).

McShane and Wyner pp. 41-42.

Any other questions?

The statement that Dr. Mann “has molested and tortured data in the service of politicized science that could have dire economic consequences for the nation and the planet” is plainly factual and verifiable.

It “is plainly factual and verifiable” that Mann “mas molested and tortured data.” How do you verify somebody tortured and molested data? Do you ask the data where it was touched?

Subsequently, every peer-reviewed study that has examined McIntyre and McKitrick’s claims has found them to be inaccurate. (note 26)

26
See, e.g., E.R. Wahl and C.M. Amman, “Robustness of the Mann, Bradley, Hughes Reconstruction of Surface Temperatures: Examinations of Criticisms Based on the Nature and Processing of Proxy Climate Evidence,” Climactic Change, 85 (2007); 33-69, available at: http://www.cgd.ucar.edu/ccr/ammann/millennium/refs/Wahl_ClimChange2007.pdf; E.R. Wahl and C.M. Amman,“The Importance of the Geophysical Context in Statistical Evaluations of Climate Reconstruction Procedure,” Climactic Change, 85 (2007); 71-88, available at: http://www.cgd.ucar.edu/ccr/ammann/millennium/refs/Ammann_ClimChange2007.pdf

“There has been the most extraordinary series of postings at Climate Audit over the last week. As is usual at CA, there is a heavy mathematics burden for the casual reader, which, with a bit of research I think I can now just about follow. The story is a remarkable indictment of the corruption and cynicism that is rife among climate scientists, and I’m going to try to tell it in layman’s language so that the average blog reader can understand it. As far as I know it’s the first time the whole story has been set out in a single posting. It’s a long tale – and the longest posting I think I’ve ever written and piecing it together from the individual CA postings has been a long, hard but fascinating struggle. You may want to get a long drink before starting, and those who suffer from heart disorders may wish to take their beta blockers first.”

http://bishophill.squarespace.com/blog/2008/8/11/caspar-and-the-jesus-paper.html

Regards,

Perry

You seem to be under a misapprehension that the situation is symmetric. Falsification is only for the “wrong”.

You’ve probably seen it already, but the Team has tackled the Koutsoyiannis paper. It’s dismissed out of hand. The basic jist is – We’re smarter than they are, they don’t know what they’re doing.

Here’s a snippet:

With that in mind, I now turn to the latest paper that is getting the inactivists excited by Demetris Koutsoyiannis and colleagues. There are very clearly two parts to this paper – the first is a poor summary of the practice of climate modelling – touching all the recent contrarian talking points (global cooling, Douglass et al, Karl Popper etc.) but is not worth dealing with in detail (the reviewers of the paper include Willie Soon, Pat Frank and Larry Gould (of Monckton/APS fame) – so no guessing needed for where they get their misconceptions). This is however just a distraction (though I’d recommend to the authors to leave out this kind of nonsense in future if they want to be taken seriously in the wider field). The second part is their actual analysis, the results of which lead them to conclude that “models perform poorly”, and is more interesting in conception, if not in execution.

so let’s calculate the maximum difference between the two series

max(tau-recon) # 1.165734e-15

Might it be more complete if you also look at the minimum (i.e. most negative) difference, or take an absolute value? Though I doubt it will show anything different.

What would happen if we did both calibration and verification on the same data set. On an earlier occasion, I extracted the target sparse data set. Instead of splicing two different series, let’s look at calibration and verification on the same time series. This time the calibration RE is only 0.177 (!), while the verification RE still rounds to 0.48

Can you explain why this happens, i.e what the difference is between the two definitions? The nearest I could find was something by Bürger 2007, On the verification of climate reconstructions which explains the distinction between CE and RE, but not so I can easily see between verification and calibration RE if the period means are the same. I may have missed something obvious.

Steve: You’re looking for something that makes sense. This is nothing to do with definitions. They spliced two different series and calculated the calibration RE on one series and verification RE on another and then compared ratios from two different series !?!