Replication #11: the 159 Series

Does anyone remember the 159 series said to have been used in MBH98?

Mann et al. [2003] stated:

MBH98 calculated PCs of proxy sub-networks separately for each interval in their stepwise reconstruction. This is the only sensible approach, as it allows all data available over each sub-interval to be used. This requires 159 independent time series to represent all indicators required for reconstructions of all possible sub-intervals, even though the maximum number ever used for a particular sub-interval is 112.

Here’s the rest of the story…

Presentation – Tues. March 1 at 4 pm

I will be making my first presentation on our 2005 papers to the Toronto Geological Discussion Group on Tues., March 1, 2005 4 p.m. at the Ontario Club, Commerce Court South, Toronto. Any readers in the area would be welcome to attend. 

Mann et al., submitted to Climatic Change, 2003

Jones and Mann [2004] states:

However, a careful analysis (M. E. Mann et al., Critical flaws in a recent criticism of the Mann et al. [1998] study, submitted to Climate Change [sic], 2003) of the McIntyre and McKittrick [2003] result reveals that their anomalous 15th century warmth results from their elimination of over 70% of the 15th century proxy data used by Mann et al. [1998a]. (page 21 of 42)

This analysis is incorrect and the reasons for the differences are discussed at length in MM05(EE). But here I wish to observe that, as of February 25, 2005, this submission to Climatic Change has not been published. Has anyone ever wondered to it? How does this non-publication reconcile with comment 2 at realclimate here)

Reference:
P.D. Jones and M.E. Mann, 2004. Climate Over Past Millennia, Reviews of Geophysics, 42(2), RG2002, doi:10.1029/2003RG000143..

Mann on Source Code

Mann told Antonio Regalado of the Wall Street Journal that he would not be “intimidated” into releasing source code for MBH98.

Here is an account of correspondence with Mann and with the U.S. National Science Foundation:

Even before publication of MM03, we politely requested clarification on issues in MBH98. This was a source of controversy in late 2003. Here is a record of correspondence with Mann which we made available some time ago.

After publication of MM03, Mann argued that MM03 contained an incorrect implementation of a stepwise principal components procedure (which was not documented in MBH98) . Details of this procedure have continued to drift in, with the first listing of the number of PC series retained in each calculation step/tree ring network combination provided in the July 2004 Corrigendum SI. This listing was inconsistent with prior information.

In August 2004, through Nature, we became aware privately of claims that a variation of Preisendorfer’s Rule N had been used to determine the number of retained PC series. This claim was published in November 2004. (We have not been able to verify actual application of this criterion, as actual numbers are impossible to replicate. See Was Preisendorfer’s Rule N Used?

In any event, immediately after we learned of the previously undocumented stepwise procedure, we asked to inspect MBH98 source code so that we can completely reconcile results and avoid this type of dispute. Attached is our correspondence after MM03, which obviously cannot be construed as a form of “intimidation”, but as an entirely proper request.

Subsequently, we located some Fortran code at Mann’s FTP site for the calculation of tree ring principal components. Although this code is only a very small fraction of the total code, it contains a procedure which was materially misrepresented in MBH98 and which additionally is not statistically valid. We reported on this in MM05 (GRL) and MM05(EE).

As I’ve pointed out in various postings on Replication, it is impossible on the present record to replicate important steps in other parts of MBH98.

After our unsuccessful attempts at obtaining source code, we asked the U.S. National Science Foundation for assistance. This was also unsuccessful. The correspondence is here.

We also made attempts with Nature and I’ll get to describing this on another day.

Data Policy #1: U.S. Global Change Research Program

I have sometimes been asked why I don’t start a FOI action with respect to source code and source data from the various multiproxy authors. I don’t preclude the possibility totally. However, my first inclination has been to attempt first to obtain the data and code from the authors through a direct and polite request, and, secondly, if that is unsuccessful, through a request to the funding agency and/or journals. I think that the stated U.S. policies make it very clear that source data and code pertaining to climate change policy should be publicly archived. To the extent that this is not being done, the acquiescence by the agencies, as well as the failures of the authors, are pertinent to policy-makers. The alphabet soup of U.S. agencies and U.S. policies is pretty confusing. I’ll try to set out some notes here on a road map to the confusing world of U.S. policy. Continue reading

"Detection and Attribution": Hegerl et al. [2003]

In some recent commentary trying to backpedal from the hockey stick, "detection and attribution" studies have been cited as alternative validations and Hegerl et al [2003] is cited as a key example. I have not looked in detail at these studies, but some features of Hegerl at al. [2003] struck me as so obvious that they are worth pointing out. Continue reading

Replication #10: NH Temperature from Archived RPCs

This next replication exercises shows the results of attempting to reconstruct the NH temperature index from the archived RPCs, with the usual puzzling discrepancies. Continue reading

Replication #9: MBH98 Instrumental Versions

Mann et al. have archived 3 slightly varying versions of the their "dense" subset and one version of their "sparse" subset. The dense series extend later than the temperature dataset archived at the Corrigendum SI and said to the the source for MBH98 instrumental data. The sparse subset can be reconstructed resaonably closely (but not exactly) from the archived temperature dataset, while there are the usual puzzling discrepancies when one tries to replicate the dense subset. Link

A Red Noise Spaghetti Diagram

I notice that the phrase "spaghetti diagram" is catching on a little. In connection with MM05, we archived 100 simulated PC1s, using the weird method of MB98. Just for fun, here’s a spaghetti diagram from 6 of these, chosen from the 100 at random, with 25-year smooths. You’ll see why I’m pretty unimpressed with spaghetti diagrams as a statistical technology. Given the proliferation of spaghetti diagrams, this obviously needs to be written up. I can splice the instrumental record and it will probably look better than the current round of spaghetti diagrams. The underlying issue is that the Mann’s PC methodology is only one form of cherry-picking, although it is a mechanized and very efficient form of cherry-picking. Spaghetti

Original SI for MBH98

If you go to the original Nature Supplementary Information for MBH98 here, and try to inspect the list of data and list of verification stats (such as they are), you will find that they are no longer there.

Update:
here