The MBH AD1450 Network

Most of my previous discussion of MBH pertained to the AD1400 network. In recent discussion over at Tamino, some of the posters have stated that BCPs only matter for the AD1400 network and that everything is fine for the AD1450 and later networks, relying here on statements in Wahl and Ammann 2007. (I don’t suppose any reviewers at Climatic Change actually checked the Wahl and Ammann calculations. ) Here is a sampling of the sort of statement about 1450 that’s becoming a mantra:

W&A tested for removal of bristlecones from the 1450 network and found that it passed significance testing. BTW, the 1450 network could actually be used back to 1428 if any real climate scientist (or anyone else for that matter) thought it was worth bothering with. …

Saying that “bristlecones improve the data quality of that stage (1400-1499)” is misleading because it omits the important fact that the bristlecones don’t enhance the data quality when PC summaries are used (as in MBH98) over 1450-1499….

I just want everyone to be clear that the bristlecones don’t become essential in MBH9x until before 1450, and if any scientist thought it was worth bothering with recalculation, they could actually be left out back to 1428….

There is a perfectly good hockeystick without bcps after 1450 (probably also after 1428 if anyone could be bothered checking). The only thing the bcps add is a longer shaft….

Chop off those 50 years, and you have a hockey stick from 1450-present that’s robust in the absence of the BCPs.

Now in one sense, it seems to me that salvaging the post-1450 network would be rather hollow. MBH99 splices onto MBH98 and it’s not just the period 1400-1450 that would be discarded, but the period from 1000-1450 or nearly half the results, including the MWP results. So it would be impossible to make any claims one way or the other about the “warmest year in the millennium”, which was where we started.

One preamble point of definition about “BCP removal”. TCO observed in the Tamino discussion that “BCP removal” for the purposes of the understanding that he is seeking is Mann’s “Censored” network plus Gaspe. TCO:

Pedant caveat: when I say bcp removal that is shorthand for “bcp cum Gaspe” removal (the trial data manipulation of Mann’s “Censored” directory.

Note: Whether combination of Gaspe with bcp is reasonable or too much as a robustness test is also a debatable position. I don’t have a firm position on that (although it’s interesting that Mike though to perform it). But my main concern with this clarification is to preempt any Groundhog Day like repetition of the snarky comments that it’s not just bcps.

This is the same definition that I use. MM2005b referred to the Censored directory as studying “a small group of 20 primarily bristlecone pine sites, all but one of which were collected by Donald Graybill and which exhibit an unexplained 20th century growth spurt”. While the Censored directory is “primarily” Graybill bristlecone sites, it also includes strip bark foxtail and limber pine sites. In the analysis below, “No Bristle” means that none of the 20 Censored sites (or cana036 – which is a duplicate use of Gaspé) are are used.

Another of Tamino’s posters purported to explain the supposed difference between the 1450 network and the 1400 network as follows:

The reconstruction for 1450-1980 contained many more proxies of different types and was therefore robust. As were the later ones.

Many commenters are at a disadvantage in this debate, because they haven’t actually gone to the trouble of learning what’s in the various networks. In fact, the AD1450 network only has 3 more proxies entering into the regression calculation than the AD1400 network: Jacoby’s Coppermine series (cana153), the Vaganov PC1 and a tree ring site in Mexico (mexi001) that is similar in character to series from the Stahle SWM network. None of these 3 series have a relevant HS shape (even though the Vaganov PC1 was calculated using Mannian pseudo-PCA.) The AD1450 network does not contain “many more proxies” of “different” types. It contains a very few additional tree ring series. In the NOAMER network, the number of sites increases from 70 to 86, but similarly none of the additional sites have relevant HS-ness.

So why does the AD1450 network supposedly yield “different” results than the AD1400 network.

No Gaspé, No Censored Sites

I’m not convinced that it does. I re-ran the AD1450 network emulating relevant cases from Wahl and Ammann 2007 – removing the 20 sites in the Censored directory and Gaspe; then applying Mannian pseudo-PC, correlation PC and covariance PC. In none of these cases did I get a HS result. Results are shown below:
ad145039.gif
ad145040.gif
ad145042.gif
Figure 1. AD1450 Network excluding Gaspe and the Censored Sites

In this particular example, the effect of Mannian PCs relative to centered PCs is not large, but even here is there is slight bias. Use of correlation versus covariance PCs has no discernible effect.

ad145041.gif
ad145043.gif
Figure 2. Differences between: top – Mannian and correlation PC results; bottom – correlation PC and covariance PC results.

Re-inserting Gaspé

As an exercise, I re-inserted the Gaspé site into both its duplicate uses in MBH98 – an exercise which helps show the remarkable impact that a single series can have in MBH-style methodology. First here is the AD1450 reconstruction using Mannian pseudo-PCs (top) with the next panel showing the difference relative to Case 6A. The introduction of one site changes the trend from the 19th to 20th century by about 0.5 deg C! Statisticians would not use the term “robust” to a describe a methodology, whose results change so markedly merely from one site. In this particular case, a substantial difference arises from using Mannian pseudo-PCs versus correlation PCs, retaining 2 PCs (as in the corresponding WA analysis). Retention of PCs does not appear to be a material issue in this case.

ad145044.gif
ad145045.gif
ad145046.gif

Figure 3. Top – AD1450 reconstruction without Censored sites, but with Gaspé in both duplicate MBH uses. middle -difference to Case 6A. bottom – two correlation PCs.


Going back to Wahl and Ammann

Based on these results, how could Wahl and Ammann have asserted or implied that BCP-Gaspé was confined to the 1400-1450 step? I searched all references to Gaspé in Wahl and Ammann 2007. The first mention stated:

The version in MM05b is based on the substitution of a new version of a single tree ring series from northeastern Canada’s St. Anne Peninsula (the “Gaspe” series) and on newly-computed PC summary series for North American tree ring data in MBH that are derived from the International Tree Ring Data Base (ITRDB), discussed below [cf. Section 2.4(5)]

This is the sort of sly Team description that is very annoying. We did not substitute a “new” version of the Gaspé series. We used the version archived at ITRDB. It was Mann who changed the archived version, by making an undisclosed and unjustified extension of the Gaspé series, with no apparent purpose other than to get it into the AD1400 network calculations.

RE statistics are reported in WA Table 2 for the three cases discussed above. I was unable to get verification RE stats for the AD1450 step anything like the WA results. For “No Gaspé, no Bristlecones/Foxtails”, using 2 Mannian pseudo-PCs, they reported an RE of 0.43 (in my Case 6A, I got -0.63); for 2 correlation PCs, they reported RE of 0.42 (I got -0.35 in Case 6B) and for 2 covariance PCs, they reported RE of 0.14 (I got -0.36).

ad145037.gif
ad145038.gif

Their Figure 4 shows their reconstruction in the case under discussion, together with their original caption. Notice anything about this graphic?

ad145052.jpg
Original Caption WA 2007 Fig. 4 Gaspé proxy restrictions as in Figure 3, with additional exclusion of 15 bristlecone/foxtail pine records from data set used to calculate principal components (PCs) of North American proxies from the International Tree Ring Data Base (ITRDB) (scenario 6, described in text; cf. Table 2). Thick pink (1400–1449) and blue (1450–1499) line includes range associated with following scenarios: reconstructions using standardized anomalies for ITRDB proxies (for input into PC extraction) referenced to 1902–1980 mean values (scenario 6a) and reconstructions using standardized anomalies for ITRDB proxies referenced to mean values over 1400–1980 and 1450–1980, for 1400–1449 and 1450–1499 reconstructions, respectively (scenario 6b). Purple (1400–1449) and green (1450–1499) reconstruction uses non-standardized anomalies for ITRDB proxies referenced to mean values over 1400–1980 and 1450–1980, for 1400–1449 and 1450–1499 reconstructions, respectively (with fitted instrumental PCs not rescaled by factor which equates variances of fitted and instrumental PCs over calibration period) (scenario 6c). WA (red line), zero reference level, and instrumental data same as in Figure 3. Pink- and purple-coded portions of scenarios 6a–c show validation failures according to criteria described in Section 2.3

Questions

On the graphic just shown, what you should have noticed is that they didn’t show the no Gaspé no bristle reconstruction after AD1500. It’s not just that they’ve deleted an “uninteresting” middle portion to highlight the two ends. They don’t show the calibration or verification period for this network in this diagram at all. Instead, they splice in the instrumental record. (Having said that the relatively high RE values are evidence that the (not shown ) reconstruction will have a noticeable trend.

So what’s going on with the apparent inconsistency of No Gaspé No Bristle results? I can match Wahl and Ammann apples-to-apples to five 9s accuracy. So the problem isn’t the usual perplexity of merely trying to replicate Team results.

I think that I know what causes the differences and am going to do a few calculations to illustrate it. In the mean time, can anyone else spot where the pea moved under the thimble in the WA analysis? (And there was a reason for quoting TCO here.)

11 Comments

  1. John A
    Posted Apr 8, 2008 at 4:39 AM | Permalink

    There are no error bars shown in the WA diagram, so no way of assessing how significant these wiggles are.

  2. Mike B
    Posted Apr 8, 2008 at 9:35 AM | Permalink

    In the mean time, can anyone else spot where the pea moved under the thimble in the WA analysis?

    Well, the only thing that immediately comes to mind that might explain the large discrepancy in the RE statistic is that WA used the MBH Gaspe series in their “No Bristlecone, No Gaspe” (i.e. MM) reconstruction. I know it sounds crazy, but this is The Team we’re talking about here.

  3. Phil Johnson
    Posted Apr 8, 2008 at 1:19 PM | Permalink

    For some reason, from Firefox 2.0.0.13, I cannot see the images in this post. I can see them when I open it in Explorer. I can see the images from Firefox the images in Steve’s post titled “Like a Dog on a Bone”.

  4. steven mosher
    Posted Apr 8, 2008 at 7:34 PM | Permalink

    re 3. ya. my FF was all messed up on CA today, so i held my nose and used IE

  5. Evan Jones
    Posted Apr 8, 2008 at 9:46 PM | Permalink

    I have tried to keep up with the lot of these posts. The long and the short of it seems to be that the BCP and Gaspe series comprise the backbone of the hockey stick–IF they are grossly overweighted. And if they are excluded (as in Loehle, etc.)the MWP pops up and the LIA drops right in.

    I won’t comment on your detractors other than to say that they do not treat people decently as befits scientific inquiry and I heartily distrust them.

    And now you report that the archived data has been screwed with. I find this outrageous. Rather than rolling logs under your feet, why do they not act like real-life scientists and try to get to the truth of the matter?

    It seem to me that the defenders of the hockey stick are what we refer to as “Bad Tribunes” in the history biz.

    A lot of the stats are over my head, but I will continue to track this debate. I am sure you know that the evidence from my side of the street supports the MWP bothe East and West (there’s not much archaeological evidence available from the SH one way or the other, unfortunately).

    What is your opinion of the Harvard-Smithsonian (2003) study on the matter?

  6. Steve McIntyre
    Posted Apr 8, 2008 at 10:01 PM | Permalink

    Well, the only thing that immediately comes to mind that might explain the large discrepancy in the RE statistic is that WA used the MBH Gaspe series in their “No Bristlecone, No Gaspe” (i.e. MM) reconstruction. I know it sounds crazy, but this is The Team we’re talking about here.

    That seems to be the answer. When you read the running text, scenarios 5 and 6, labelled “No Gaspé” are not actually “No Gaspé” cases. The pea moved under the thimble. What “No Gaspe” means is this: the problematic Cook Gaspe series is not used as an individual proxy in the pre-1450 segment (when there are only 1-2 cores), but is used for the periods after 1450.

    You have to drill through several sly descriptions. For example, Scenario 6 is described as follows in the running text:

    Scenario 6 illustrates this observation by direct exclusion of the bristlecone/foxtail pine records from the data set used to calculate North American ITRDB proxy PCs (note, the Gaspe data are also removed as in scenario 5). …

    Reconstruction was done over 1400–1499 using the same structure as scenario 5, with the additional elimination of the 15 bristlecone/foxtail pine records from the North American ITRDB data prior to PC calculations. (The 15 records eliminated are the same as those eliminated in scenario 3.

    Now merely reading the phrase “the Gaspe data are removed as in Scenario 5″ together with the caption “No Gaspe” might well lead an unwary reader to conclude that the Gaspe data were removed as a sensitivity test in both scenarios. This proves not to be the case. Elsewhere:

    Reconstruction was done over 1400–1499 by including PC summaries of the North American proxy data from the ITRDB: replacing the original MBH-calculated PCs with newly calculated PCs based on excluding the Gaspe series (i.e., removing it as an individual proxy over 1400–1449 and as part of the data set used to calculate the North American ITRDB PCs).

    Here’s what they did. MBH used Gaspe twice in the original study – once as treeline11.dat as an individual proxy and once as cana036 in the NOAMER principal components. WA said that they eliminated the duplicate use in the NOAMER network. The statement here also says that they removed it as an individual proxy in the 1400 network, but not the 1450 network.

    So the answer to the question about what’s different in the Wahl-Ammann 1450 network than the 1400 network: it’s the Gaspe series!!!

    But there’s also something else going on that I can’t figure out. I should be able to replicate their RE stats, but so far I can’t. (Actually I’ve got something else to test: I’ve been using Mann proxy weights in my calcs, but Wahl-Ammann “simplified” the algorithm by using uniform weights. This could have a noticeable impact on the Gaspe weight and RE statistics.

  7. DaleC
    Posted Apr 8, 2008 at 10:39 PM | Permalink

    Steve,

    Do you have R code scripts for your analysis above?
    Also, where are the scripts and data files which used to be at Climate2003 now located?

    Thanks,

    DaleC.

    Steve: I’ve put the data and script directories at http://www.climateaudit.org/data/climate2003/data and /scripts. I got tired of paying for climate2003 when I’ve got capacity here and have tried to figure out how to transfer the domain name to this server, but haven’t had any luck so far. The old site was at geocities, so if anyone knows how to wade through domain transfer, let me know.

    I’ll post up these scripts – I need to make some data input references to external files to make it transportable, but this shouldn’t take very long.

  8. Glacierman
    Posted Apr 9, 2008 at 8:45 AM | Permalink

    This should outrage people, but we are all completely desensitized to it. Will there be any young scientists coming up who will put science first, or have they all fallen in line to advance their careers?

    RE: Evan Jones:

    And now you report that the archived data has been screwed with. I find this outrageous. Rather than rolling logs under your feet, why do they not act like real-life scientists and try to get to the truth of the matter?

    Because the truth is inconvenient.

  9. Posted Apr 9, 2008 at 3:47 PM | Permalink

    Re: climate2003.com

    geocities was acquired by Yahoo. Based on the whois information

    Tech Name………… YahooDomains TechContact
    Tech Email……….. domain.tech -at- YAHOO-INC.COM
    Tech Phone……….. +1.6198813096

    Have you tried contacting them at that email address?

    Sinan

  10. chopbox
    Posted Apr 9, 2008 at 10:56 PM | Permalink

    I’m thinking of Steve giving his talk at the Georgia seminar and now I imagine some grad student at some seminar somewhere asking MBH(99) or WA why they do it that way (dropping the early bristlecone data and keeping the later stuff). I suspect they would answer that they are dropping the early stuff because there just weren’t enough data points (1 or 2 trees I think), but that the later stuff should stay because there are lots of data there. If the real reason is that the bristlecones are poor proxies for temperature, can somebody tell me how this message can get out there?

  11. Robinedwards
    Posted Apr 16, 2008 at 1:39 PM | Permalink

    Is there any available complete mapping of the data referred to in this (and other) threads to Mann’s 112 columns of data that apparently led to the hockey stick plot?

    I have the numerical data, with column names, courtesy of Steve McIntyre, but am now quite lost when I try to reconcile the names used in this thread with those that I am familiar with, or, equivalently the column numbers.

    I would be most grateful for some enlightenment in this regard.

    Robin

Follow

Get every new post delivered to your Inbox.

Join 3,306 other followers

%d bloggers like this: