Climate Audit

Most of my previous discussion of MBH pertained to the AD1400 network. In recent discussion over at Tamino, some of the posters have stated that BCPs only matter for the AD1400 network and that everything is fine for the AD1450 and later networks, relying here on statements in Wahl and Ammann 2007. (I don’t suppose any reviewers at Climatic Change actually checked the Wahl and Ammann calculations. ) Here is a sampling of the sort of statement about 1450 that’s becoming a mantra:

W&A tested for removal of bristlecones from the 1450 network and found that it passed significance testing. BTW, the 1450 network could actually be used back to 1428 if any real climate scientist (or anyone else for that matter) thought it was worth bothering with. …

Saying that “bristlecones improve the data quality of that stage (1400-1499)” is misleading because it omits the important fact that the bristlecones don’t enhance the data quality when PC summaries are used (as in MBH98) over 1450-1499….

I just want everyone to be clear that the bristlecones don’t become essential in MBH9x until before 1450, and if any scientist thought it was worth bothering with recalculation, they could actually be left out back to 1428….

There is a perfectly good hockeystick without bcps after 1450 (probably also after 1428 if anyone could be bothered checking). The only thing the bcps add is a longer shaft….

Chop off those 50 years, and you have a hockey stick from 1450-present that’s robust in the absence of the BCPs.

Now in one sense, it seems to me that salvaging the post-1450 network would be rather hollow. MBH99 splices onto MBH98 and it’s not just the period 1400-1450 that would be discarded, but the period from 1000-1450 or nearly half the results, including the MWP results. So it would be impossible to make any claims one way or the other about the “warmest year in the millennium”, which was where we started.

One preamble point of definition about “BCP removal”. TCO observed in the Tamino discussion that “BCP removal” for the purposes of the understanding that he is seeking is Mann’s “Censored” network plus Gaspe. TCO:

Pedant caveat: when I say bcp removal that is shorthand for “bcp cum Gaspe” removal (the trial data manipulation of Mann’s “Censored” directory.

Note: Whether combination of Gaspe with bcp is reasonable or too much as a robustness test is also a debatable position. I don’t have a firm position on that (although it’s interesting that Mike though to perform it). But my main concern with this clarification is to preempt any Groundhog Day like repetition of the snarky comments that it’s not just bcps.

This is the same definition that I use. MM2005b referred to the Censored directory as studying “a small group of 20 primarily bristlecone pine sites, all but one of which were collected by Donald Graybill and which exhibit an unexplained 20th century growth spurt”. While the Censored directory is “primarily” Graybill bristlecone sites, it also includes strip bark foxtail and limber pine sites. In the analysis below, “No Bristle” means that none of the 20 Censored sites (or cana036 – which is a duplicate use of Gaspé) are are used.

Another of Tamino’s posters purported to explain the supposed difference between the 1450 network and the 1400 network as follows:

The reconstruction for 1450-1980 contained many more proxies of different types and was therefore robust. As were the later ones.

Many commenters are at a disadvantage in this debate, because they haven’t actually gone to the trouble of learning what’s in the various networks. In fact, the AD1450 network only has 3 more proxies entering into the regression calculation than the AD1400 network: Jacoby’s Coppermine series (cana153), the Vaganov PC1 and a tree ring site in Mexico (mexi001) that is similar in character to series from the Stahle SWM network. None of these 3 series have a relevant HS shape (even though the Vaganov PC1 was calculated using Mannian pseudo-PCA.) The AD1450 network does not contain “many more proxies” of “different” types. It contains a very few additional tree ring series. In the NOAMER network, the number of sites increases from 70 to 86, but similarly none of the additional sites have relevant HS-ness.

So why does the AD1450 network supposedly yield “different” results than the AD1400 network.

No Gaspé, No Censored Sites
I’m not convinced that it does. I re-ran the AD1450 network emulating relevant cases from Wahl and Ammann 2007 – removing the 20 sites in the Censored directory and Gaspe; then applying Mannian pseudo-PC, correlation PC and covariance PC. In none of these cases did I get a HS result. Results are shown below:



Figure 1. AD1450 Network excluding Gaspe and the Censored Sites

In this particular example, the effect of Mannian PCs relative to centered PCs is not large, but even here is there is slight bias. Use of correlation versus covariance PCs has no discernible effect.

Figure 2. Differences between: top – Mannian and correlation PC results; bottom – correlation PC and covariance PC results.

Re-inserting Gaspé

As an exercise, I re-inserted the Gaspé site into both its duplicate uses in MBH98 – an exercise which helps show the remarkable impact that a single series can have in MBH-style methodology. First here is the AD1450 reconstruction using Mannian pseudo-PCs (top) with the next panel showing the difference relative to Case 6A. The introduction of one site changes the trend from the 19th to 20th century by about 0.5 deg C! Statisticians would not use the term “robust” to a describe a methodology, whose results change so markedly merely from one site. In this particular case, a substantial difference arises from using Mannian pseudo-PCs versus correlation PCs, retaining 2 PCs (as in the corresponding WA analysis). Retention of PCs does not appear to be a material issue in this case.

Figure 3. Top – AD1450 reconstruction without Censored sites, but with Gaspé in both duplicate MBH uses. middle -difference to Case 6A. bottom – two correlation PCs.

Going back to Wahl and Ammann

Based on these results, how could Wahl and Ammann have asserted or implied that BCP-Gaspé was confined to the 1400-1450 step? I searched all references to Gaspé in Wahl and Ammann 2007. The first mention stated:

The version in MM05b is based on the substitution of a new version of a single tree ring series from northeastern Canada’s St. Anne Peninsula (the “Gaspe” series) and on newly-computed PC summary series for North American tree ring data in MBH that are derived from the International Tree Ring Data Base (ITRDB), discussed below [cf. Section 2.4(5)]

This is the sort of sly Team description that is very annoying. We did not substitute a “new” version of the Gaspé series. We used the version archived at ITRDB. It was Mann who changed the archived version, by making an undisclosed and unjustified extension of the Gaspé series, with no apparent purpose other than to get it into the AD1400 network calculations.

RE statistics are reported in WA Table 2 for the three cases discussed above. I was unable to get verification RE stats for the AD1450 step anything like the WA results. For “No Gaspé, no Bristlecones/Foxtails”, using 2 Mannian pseudo-PCs, they reported an RE of 0.43 (in my Case 6A, I got -0.63); for 2 correlation PCs, they reported RE of 0.42 (I got -0.35 in Case 6B) and for 2 covariance PCs, they reported RE of 0.14 (I got -0.36).

Their Figure 4 shows their reconstruction in the case under discussion, together with their original caption. Notice anything about this graphic?

Original Caption WA 2007 Fig. 4 Gaspé proxy restrictions as in Figure 3, with additional exclusion of 15 bristlecone/foxtail pine records from data set used to calculate principal components (PCs) of North American proxies from the International Tree Ring Data Base (ITRDB) (scenario 6, described in text; cf. Table 2). Thick pink (1400–1449) and blue (1450–1499) line includes range associated with following scenarios: reconstructions using standardized anomalies for ITRDB proxies (for input into PC extraction) referenced to 1902–1980 mean values (scenario 6a) and reconstructions using standardized anomalies for ITRDB proxies referenced to mean values over 1400–1980 and 1450–1980, for 1400–1449 and 1450–1499 reconstructions, respectively (scenario 6b). Purple (1400–1449) and green (1450–1499) reconstruction uses non-standardized anomalies for ITRDB proxies referenced to mean values over 1400–1980 and 1450–1980, for 1400–1449 and 1450–1499 reconstructions, respectively (with fitted instrumental PCs not rescaled by factor which equates variances of fitted and instrumental PCs over calibration period) (scenario 6c). WA (red line), zero reference level, and instrumental data same as in Figure 3. Pink- and purple-coded portions of scenarios 6a–c show validation failures according to criteria described in Section 2.3

Questions

On the graphic just shown, what you should have noticed is that they didn’t show the no Gaspé no bristle reconstruction after AD1500. It’s not just that they’ve deleted an “uninteresting” middle portion to highlight the two ends. They don’t show the calibration or verification period for this network in this diagram at all. Instead, they splice in the instrumental record. (Having said that the relatively high RE values are evidence that the (not shown ) reconstruction will have a noticeable trend.

So what’s going on with the apparent inconsistency of No Gaspé No Bristle results? I can match Wahl and Ammann apples-to-apples to five 9s accuracy. So the problem isn’t the usual perplexity of merely trying to replicate Team results.

I think that I know what causes the differences and am going to do a few calculations to illustrate it. In the mean time, can anyone else spot where the pea moved under the thimble in the WA analysis? (And there was a reason for quoting TCO here.)