A New Caramilk Secret

Jean S, UC and I spent a considerable amount of time a couple of years ago, trying to figure out how MBH99 confidence intervals were calculated – see here. I asked the NAS panel to investigate the matter, but they failed to do so. After their report, I asked NAS president Cicerone to merely write to Mann asking him to provide an explanation – Cicerone refused. I asked Gerry North to do so; North agreed to do so, but I never heard anything more about it. So either he agreed to ask for an explanation and then didn;t follow through or Mann refused even a request from Gerry North.

North was supposedly one of the reviewers for Mann et al 2008, which also refers to confidence intervals. One would have hoped that North, already on notice about this issue, would have ensured that Mann et al 2008 clearly explained their calculation of confidence intervals. No such luck.

Figures 2, 3 and S5 all illustrate “95% confidence intervals” by “lightly shaded regions of similar color)”. Mann et al observe: “For the CPS (EIV) reconstructions, the instrumental warmth breaches the upper 95% confidence limits of the reconstructions beginning with the decade centered at 1997 (2001)” and in the caption to Figure 3 state “Confidence intervals have been reduced to account for smoothing.”

The “Methods” state:

Uncertainties were estimated from the residual decadal variance during the validation period based (32, 42) on the average validation period r2 (which in this context has the useful property that, unlike RE and CE, it is bounded by 0 and 1 and can therefore be used to define a ‘‘fraction’ of unresolved variance).

That’s it. There is no further explanation. The SI contains a column entitled “uncertainty” which changes in century-steps. I’ve examined the source code and, once again, it appears to be incomplete. I’ve been unable to locate any code evidencing the calculation of the confidence intervals.

Another Caramilk secret. I’ll experiment a bit with some of the endless Butterworth smooths and see if anything turns up.


12 Comments

  1. Hank
    Posted Nov 10, 2008 at 11:43 AM | Permalink

    Note to readers in the USA. Caramilk is a candy bar marketed in Canada. The “Caramilk secret” is a phrase from advertising for that candy bar.

    • Urederra
      Posted Nov 10, 2008 at 2:01 PM | Permalink

      Re: Hank (#1),
      Thanks for the reference, It is useful for the ones who are not in the USA too.

      After their report, I asked NAS president Cicerone to merely write to Mann asking him to provide an explanation – Cicerone refused.

      That made me laugh. For the historically challenged, if you look up the meaning of Cicerone at wikipedia, you will find this:

      Cicerone is an old term for a guide, one who conducts visitors and sightseers to museums, galleries, etc., and explains matters of archaeological, antiquarian, historic or artistic interest. The word is presumably taken from Marcus Tullius Cicero, as a type of learning and eloquence.

      I use the term Cicerone to describe the local friend who shows you his hometown. It is quite ironic that Cicerone doesn’t help Steve in his quest to find the way to Mann’s confidence intervals.

  2. DeWitt Payne
    Posted Nov 10, 2008 at 11:51 AM | Permalink

    The confidence intervals are as or more important to the conclusion that current warming is unprecedented than the central values of the reconstruction. How any reviewer can approve this for publication without knowing, even in outline, how the confidence intervals are calculated is beyond me. We cannot ever know, absent a time machine, if the reconstruction is correct. We can only know how confident we are of the bounds of the reconstruction. With Mann, we have only his and his coauthor’s word.

    Since when does smoothing reduce confidence intervals?

    • Kenneth Fritsch
      Posted Nov 10, 2008 at 5:18 PM | Permalink

      Re: DeWitt Payne (#2),

      Since when does smoothing reduce confidence intervals?

      Smoothing has to use up degrees of freedom so I would have to ask the same question.

  3. Sam Urbinto
    Posted Nov 10, 2008 at 1:17 PM | Permalink

    The trend over 130 years isn’t even over the resolution of the recorded tmin and tmax. Confident? Come on now. It “doesn’t matter”. It’s “time to move on”.

  4. Craig Loehle
    Posted Nov 10, 2008 at 1:32 PM | Permalink

    I just reviewed a paper with a novel analytical method. There was no discussion of the null model (what would happen by chance) or the ability to do a test of significance (which includes info on variability). I recommended rejection. Sadly, it was not a climate science paper.

  5. Andrew
    Posted Nov 10, 2008 at 2:45 PM | Permalink

    Another Caramilk secret.

    Is a Caramilk Secret anything like a Ketchup Secret?

    If so, I’d really, really like to know what it is.

    • Stan Palmer
      Posted Nov 10, 2008 at 2:56 PM | Permalink

      Re: Andrew (#6),

      Is a Caramilk Secret anything like a Ketchup Secret?

      If so, I’d really, really like to know what it is.

      Caramilk is a cream filled chocolate bar. The secret is how they put the cream in he chocolate

  6. jeez
    Posted Nov 10, 2008 at 3:16 PM | Permalink

    Unlike the mysteries of Climate Science, the Caramilk Secret has been unraveled.

  7. Craig Loehle
    Posted Nov 10, 2008 at 3:40 PM | Permalink

    Unlike Caramilk and Pepsi, there are not supposed to be secret ingredients in a science paper.

  8. John A
    Posted Nov 10, 2008 at 7:03 PM | Permalink

    The Caramilk Secret was revealed on CA here

  9. D Johnson
    Posted Nov 10, 2008 at 11:12 PM | Permalink

    This thread is frustrating! I’ve developed a powerful craving for a candy I’ve never even tasted. How can a person in the lower 48 find a sample? Is there some FOI equivalent that would provide a sample to try?

One Trackback

  1. By Models » tech enthusiasm in vietnam on Nov 25, 2008 at 12:24 AM

    [...] A New Caramilk Secret « Climate AuditThere was no discussion of the null model (what would happen by chance) or the ability to do a test of significance (which includes info on variability). I recommended rejection. Sadly, it was not a climate science paper. … [...]

Follow

Get every new post delivered to your Inbox.

Join 3,245 other followers

%d bloggers like this: