Jean S, UC and I spent a considerable amount of time a couple of years ago, trying to figure out how MBH99 confidence intervals were calculated – see here. I asked the NAS panel to investigate the matter, but they failed to do so. After their report, I asked NAS president Cicerone to merely write to Mann asking him to provide an explanation – Cicerone refused. I asked Gerry North to do so; North agreed to do so, but I never heard anything more about it. So either he agreed to ask for an explanation and then didn;t follow through or Mann refused even a request from Gerry North.
North was supposedly one of the reviewers for Mann et al 2008, which also refers to confidence intervals. One would have hoped that North, already on notice about this issue, would have ensured that Mann et al 2008 clearly explained their calculation of confidence intervals. No such luck.
Figures 2, 3 and S5 all illustrate “95% confidence intervals” by “lightly shaded regions of similar color)”. Mann et al observe: “For the CPS (EIV) reconstructions, the instrumental warmth breaches the upper 95% confidence limits of the reconstructions beginning with the decade centered at 1997 (2001)” and in the caption to Figure 3 state “Confidence intervals have been reduced to account for smoothing.”
The “Methods” state:
Uncertainties were estimated from the residual decadal variance during the validation period based (32, 42) on the average validation period r2 (which in this context has the useful property that, unlike RE and CE, it is bounded by 0 and 1 and can therefore be used to define a ‘‘fraction’ of unresolved variance).
That’s it. There is no further explanation. The SI contains a column entitled “uncertainty” which changes in century-steps. I’ve examined the source code and, once again, it appears to be incomplete. I’ve been unable to locate any code evidencing the calculation of the confidence intervals.
Another Caramilk secret. I’ll experiment a bit with some of the endless Butterworth smooths and see if anything turns up.
Climate Audit 
12 Comments
Note to readers in the USA. Caramilk is a candy bar marketed in Canada. The “Caramilk secret” is a phrase from advertising for that candy bar.
Re: Hank (#1),
Thanks for the reference, It is useful for the ones who are not in the USA too.
That made me laugh. For the historically challenged, if you look up the meaning of Cicerone at wikipedia, you will find this:
I use the term Cicerone to describe the local friend who shows you his hometown. It is quite ironic that Cicerone doesn’t help Steve in his quest to find the way to Mann’s confidence intervals.
The confidence intervals are as or more important to the conclusion that current warming is unprecedented than the central values of the reconstruction. How any reviewer can approve this for publication without knowing, even in outline, how the confidence intervals are calculated is beyond me. We cannot ever know, absent a time machine, if the reconstruction is correct. We can only know how confident we are of the bounds of the reconstruction. With Mann, we have only his and his coauthor’s word.
Since when does smoothing reduce confidence intervals?
Re: DeWitt Payne (#2),
Smoothing has to use up degrees of freedom so I would have to ask the same question.
The trend over 130 years isn’t even over the resolution of the recorded tmin and tmax. Confident? Come on now. It “doesn’t matter”. It’s “time to move on”.
I just reviewed a paper with a novel analytical method. There was no discussion of the null model (what would happen by chance) or the ability to do a test of significance (which includes info on variability). I recommended rejection. Sadly, it was not a climate science paper.
Is a Caramilk Secret anything like a Ketchup Secret?
If so, I’d really, really like to know what it is.
Re: Andrew (#6),
Caramilk is a cream filled chocolate bar. The secret is how they put the cream in he chocolate
Unlike the mysteries of Climate Science, the Caramilk Secret has been unraveled.
Unlike Caramilk and Pepsi, there are not supposed to be secret ingredients in a science paper.
The Caramilk Secret was revealed on CA here
This thread is frustrating! I’ve developed a powerful craving for a candy I’ve never even tasted. How can a person in the lower 48 find a sample? Is there some FOI equivalent that would provide a sample to try?
One Trackback
[…] A New Caramilk Secret « Climate AuditThere was no discussion of the null model (what would happen by chance) or the ability to do a test of significance (which includes info on variability). I recommended rejection. Sadly, it was not a climate science paper. … […]