Mann et al. [2000] listed 5 quality control criteria, which were said to have been applied in MBH98 for the inclusion of tree ring sites. These criteria were again referred to in the MBH Corrigendum in July 2004. In total, the information in the SI to MBH98 showed the use of 394 sites, either as individual proxies or as a component of a principal component network. I have been able to identify original data at WDCP for 331 out of 333 non-Vaganov sites used in MBH98. 196 out of these 331 series failed one or more of the stated quality control tests. One Jacoby series failed the mean correlation test so spectacularly (mean correlation of only 0.045 between segments and archived chronology) that we contacted one of the originating authors (D’Arrigo) who requested WDCP to withdraw the data.
Summary: We examined 331 series used directly either as individual proxies or as a component of a principal component network. 196 of these series failed one or more of the stated quality control tests, most frequently the mean correlation, but occasionally each of the other tests (except for the end date of the series). Results of these calculations are shown in Appendices.
In our review of MBH98 sites, we noticed a material discrepancy between the series listed in the original SI and the series actually used. This is summarized in Table 1 below. In MM03, we had pointed out the discrepancy for the “miscellaneous” 5 series; after we obtained access to a previously unavailable directory at Mann’s FTP site in November 2003, we determined that 30 other series listed in the original SI were not used in calculations at Mann’s FTP site. We notified Nature of this and, in July 2004, these discrepancies, together with some, but not all of the problems affecting MBH98, were acknowledged in the Corrigendum, which provided a listing of the 35 series.
The Corrigendum purported to provide an explanation for the discrepancy as follows:
These series, all of which come from the International Tree Ring Data Bank (ITRDB), met all the tests used for screening of the ITRDB data used in Mann et al. [1998] (see Mann et al. (2000)), except one—namely, that in 1997, either it could not be ascertained by the authors how these series had been standardized by the original contributors, or it was known that the series had been aggressively standardized, removing multidecadal to century-scale fluctuations.
The tests listed in Mann et al. (2000), and presumably the ones referred to in the Corrigendum, were as follows:
- Reliable information on the methods used to remove biological trend was available;
- The median length of the individual segments used to build the chronology was greater than 150 years;
- The mean correlation of these individual segments with the site chronology was greater than 0.5;
- The first year of the chronology was before AD 1626, and it contained at least 8 segments by 1680;
- The last year was after 1970, and there were still 8 segments after 1960.
In this note, I report on verification attempts on the actual application of these criteria; I will examine the supposed explanation for the discrepancy in a forthcoming note.
Mann et al. use tree ring chronologies as part of 6 principal component networks, as individual proxies and indirectly in temperature and precipitation reconstructions from tree rings. Here I only consider quality control in the tree ring sites used in principal component networks and directly as proxies and, in another note, I will look at quality control for most of the tree ring sites used indirectly through precipitation and temperature reconstructions.
The original SI provided locations or ITRDB codes for 333 tree ring sites and reported the use of 61 Vaganov sites without providing locations. As shown in Table 2 below, out of the 333 sites with locations or codes, I was able to locate 331 series at WDCP/ITRDB, with one NOAMER series (ar045) and one Stahle.SWM series not being archived at ITRDB. I have not tried to determine the reasons for these discrepancies. Mann’s FTP site showed that 22 sites were used in the Stahle/SWM network rather than the 20 listed in the original SI. The Corrigendum attributed the 2 additional sites to Stahle, pers. comm., but unfortunately did not provide a location or code. Since the first 120-125 years of each series is identical to other series in the 20 identified sites, it appears that there has been a duplication or splicing. This remains unexplained. This affects retention policy in the 15th century. MBH98 stated that PC networks required a minimum of 7 series for calculation, except for the Stahle/SWM series,which, for some reason, only required 6 series. The unexplained duplications affect the Stahle/SWM network in the 15th century as, without the duplication, there are only 4 series. For final calculations, this probably does not “matter”, since nothing much “matters” in MBH98 except bristlecone pines, but it would be nice to reconcile the duplications.
At Mann’s FTP site, there are latitudes and longitudes and three-letter codes for the Vaganov sites. I spot-checked some Vaganov sites against ITRDB sites in similar locations and didn’t have much success in matching and accordingly excluded the Vaganov network from this quality control verification.
The quality control checks, said to have applied to the sites listed in the original SI, were checked for each of the 331 sites, which could be identified at WDCP/ITRDB. I found that 196 of the 331 sites failed one or more of the quality control criteria, said to have been applied, as summarized in Table 3 below. In some cases, there were differences between the WDCP Version and MBH version
Some Follow-Up on Quality Checks
One QC failure was so spectacular (a mean correlation between chronology and individual segments of only 0.045) that we followed up with the original author, Rosanne D’Arrigo, as follows:
The *.crn series at WDCP for the site cana158 has extremely and atypically low correlations to the individual trees used in the calculation. The *.crn series also extends to 1982, while the *rwl data goes only to 1978. By any chance, is there an error in the labelling in the series submitted to WDCP?
Also I note that several treeline *.crn series have been updated to 1990-92 (ak031, ak032 and cana177), but the *.rwl series have not been updated commensurately. Perhaps this is an oversight and, if so, perhaps you could remedy the matter.
This prompted the originating author to request the withdrawal of the chronology from WDCP.
Other QC Issues
During the course of this QC exercise, the following issues with certain WDCP/ITRDB datasets were noticed.
- 46 tree series in 7 South American sites had missing periods in the individual tree series, with the largest gap being 89 years for tree PP1993 at site arge015. All of these series were collected several decades ago and WDCP had no information on the reasons for the gaps (Bruce Bauer, pers. comm.). MBH provided no information on how they handled these gaps in their own quality control. All 7 sites affected by gaps in individual tree series were used in MBH98.
- The Polar Urals site and Tornetrask site also have many gaps. In these cases, the ring widths were apparently not measured prior to densitometric calculations, and gaps were lost re-orienting the core for densitrometric measurements (P.D. Jones, pers. comm.) but the number of rings were counted.
- arge015 has a misprint in the rwl file in which tree PP1982 is dated one millennium too early. The above was corrected in these calculations
Appendices
Quality Control Testing QC.XLSX
Climate Audit 
Sci-AM: Confirming the importance of MBH 99
A new profile of Michael Mann appears in the March edition of Scientific American (on the grounds that you can never have too many profiles) Update: the author of the profile is David Appell. Isn’t it a small world? You’ll have to pay real money to see all of it, but here’s a rather revealing extract (my emphasis added):
Continue reading →