I’ve made histograms of reported proxy correlations for 1850-1995, as reported in r1209.xls (which contains results for all proxies, unlike SI SD1.xls which withholds results below a benchmark.) The breaks are in 0.1 intervals. On the left is the histogram before screening; on the right, a histogram of the 484 proxies after screening.
Clearly a great deal of analysis could be done on this topic. I’ll just scratch the surface on this, as I’m going to be away for a couple of days, but felt that the issue warranted being on the table right away.
The first couple of things that struck me about the pre-screening distribution were -
1. there was odd tri-modality to the distribution, with a bulge off to the right with very high correlations;
2. the distribution was surprisingly symmetric other than the right-hand bulge, but was “spread” out more than i.i.d. normal distributions;
3. Mannian screening was, for the most part, one-sided, although high negative values were retained.
A little inspection showed that the right-hand bulge of very high correlations arose entirely from the Luterbacher gridded series, which, as I understand it (and I haven’t reviewed the Luterbacher data), contains instrumental information in the calibration period and is not a “proxy” in the sense of tree rings or ice cores. So when Mann says that 484 series passing a benchmark is evidence of “significance”, this inflates the perceived merit of tree ring and ice core data since the 71 Luterbacher series make a non-negligible contribution. Removing the Luterbacher series, one gets the more symmetric distribution shown below:
Next, the bimodality of this distribution calls for a little explanation. The vast majority of the proxies in this figure are tree rings, so we’re back to tree rings. It’s possible that this bimodality is a real effect, i.e. that some chronologies respond negatively to temperature and others positively. But it’s equally possible that a form of pre-screening has already taken place in collating the network with very “noisy” chronologies being excluded from even the pre-screened network. It would take some careful analysis of the tree ring networks to pin this down, but selection bias seems more likely to me than actual bimodality, but that’s just a guess right now.
Next, the correlations are more spread out than one would expect from i.i.d normal distributions, where Mann’s SI states that 90% of the proxies would be within -0.1 to 0.1 correlations. Given the fact that there are almost as many negative as positive correlations, this suggests to me that the effect of autocorrelation is substantially under-estimated in choosing 0.1 as a 90% standard. Given the relatively symmetric distribution, it looks far more likely to me that autocorrelation effects are wildly under-estimated in his benchmark and that the 90% benchmark is much higher. It’s not nearly as clear as Mann makes out that the yield of 484 proxies (less 71 Luterbacher) is as significant as all that.
This particular operation looks more and more like ex-post cherry picking from red noise (along the lines of discussions long ago by both David Stockwell and myself.) This is a low-tech way of generating hockey sticks, not quite as glamorous as Mannian principal components, but it “works” almost as well.
It’s pretty discouraging that Gerry North and Gaby Hegerl were unequal to the very slight challenge of identifying this problem in review.