One bit of housekeeping that I want to tidy up before more NAS postings: a couple of months ago, Eduardo Zorita kindly sent me comprehensive data from ECHO-G, on which, unfortunately, I’ve so far not been able to spend as much time on so far as I would have liked. So much to do, so little time. Included in the package were supporting calculations for their Comment on MM05 in which they stated:
Our results, derived in the artificial world of an extended historical climate simulation, indicate therefore that the AHS [Artificial Hockey Stick] does not have a significant impact but leads only to very minor deviations. We suggest, however, that this biased centering should be in future avoided as it may unnecessarily compromise the final result.
Obviously this is a different conclusion than we reached and I’ve been anxious to reconcile the different findings. Eduardo and I already exchanged code on our replication of Mannian PCs and I’ve confirmed that the key aspects coincide (although Eduardo did not use detrended standard devations in Mannian PC calculations).
As shown below, I’ve determined that the the AHS has no effect on Eduardo’s results, not because of the climate model, but because the PC series are essentially identical when pseudoproxies are constructed with white noise under both Mannian and conventional PCs (either correlation or covariance). I would submit that the implication of this is: if a network consists of a signal plus white noise, then the PC1 is similar under different methods. This defines a test for whether the network consists of a signal plus white noise, and does not prove that the differing methods have no impact on a Mannian network.
Von Storch and Zorita 2005 reported that they constructed pseudoproxies based on mixing white noise with gridcell temperatures generated by a climate model. In the run erik167 sent to me, the proportion of white noise appears to be 50%, as the correlation of pseudoproxies to gridcell temperature is around 0.7, at the upper end of the range in Jones and Mann 2004 cited by VZ (0.3-0.7). While the erik167 run was at the higher correlation range, VZ report that results are similar within the range and I expect that to be so. As we pointed out in our Reply to VZ, this is not an accurate value range for actual MBH98 tree ring proxies, where the correlations to gridcell temperature are around 0, although many proxies have correlations to precipitation around 0.4 and some have correlations to CO2 levels as high as 0.7.
VZ carried out PC analysis on three different "regions", two of which are illustrated below. The figures below show the PC1 using Eduardo’s implementation of the Mannian method (AHS) and the correlation PC1 (the covariance PC1 is nearly identical), as well as the difference. Note that Region 1 had more proxies (55 to 9) and the difference between results is less, although it is tiny in both cases.For pseudoproxies constructed from gridcell temperatures with 50% white noise, the PC1 from the AHS calculation is virtually identical to the PC1 from the normal calculation.
PC1 from VZ Region 3 – AHS and correlation PC, with difference. Legend of bottom panel is incorrect.
PC1 from VZ Region 1 – AHS and correlation PC with difference (legend of bottom panel is incorrect)
Now compare this to the North American tree ring network, illustrated below, showing the Mannian, covariance and correlation PC1. Regardless of the position of what one takes on which, if any, of these series is "right", the PC1s are obviously different and substantially so.
PC1 from MBH98 North American Network – Mannian, covariance and correlation.
Using two PCs from this network as in MBH98, these different results have a substantial impact on an MBH98-type reconstruction (although again their regression methodology is itself not neccessarily "right" a priori.) This is simply an empirical result. In our 2005 E&E article, we summarized a variety of results, including that the reconstruction using correlation PCs was intermediate between reconstructions using covariance PCs and Mannian PCs.
Notice the constrasting situation with the VZ network. In the VZ network, the Mannian PC1 and the correlation PC1 are essentially identical. Thus, there can only be negligible difference in the final reconstruction obtained by carrying these series into the regression module. The reason for the seeming lack of impact of the AHS effect is not that the climate model washes out differences in the PC series; it’s that there was simply no difference in the PC series.
Now let’s re-examine the conclusion of von Storch and Zorita 2005, cited by the NAS Panel:
Our results, derived in the artificial world of an extended historical climate simulation, indicate therefore that the AHS does not have a significant impact but leads only to very minor deviations. We suggest, however, that this biased centering should be in future avoided as it may unnecessarily compromise the final result.
Let’s be perfectly clear on what VZ have and haven’t shown: all they’ve proved here is that in a sufficiently "tame" network, and a network constructed from a signal plus white noise is about as "tame" as you get, Mannian PC methodology and conventional PCs give almost exactly the same answer. We agree with this and have always agreed with this. This has nothing to do with climate models; it’s simply to do with "tame" networks. This was the position that we proposed in our Reply to VZ and, in my opinion, the correctness of the position in our Reply is proven by the near identity of the PC series in the two VZ versions.
However, there’s something very interesting in this that we did not raise in our Reply and, oddly enough, it relates more closely to their dispute with Wahl, Ammann and Ritson than to their exchange with us. Ritson at realclimate has argued that North American tree ring proxies can be construed as having white noise.
However, if the MBH North American tree ring network consisted of signal plus white noise (or low order red noise), then the above results show that Mannian PC methods and conventional PC methods applied to the North American network should produce almost indistinguishable results. But they don’t. Ergo, the MBH North American tree ring proxies do NOT consist of a signal plus white noise in the 50-75% range. This doesn’t say what they are – however, it says something about what they aren’t. I would surmise that low-order red noise (say AR1 of 0.5 and lower) would yield identical results.
There is a further point. It almost seems to be a characteristic of "tame" networks that you "get" fairly similar results regardless of what you do. You see this point made from time to time in multivariate statistical literature (especially chemometrics) – that one gets fairly similar results using various methods.
On the other hand, if the network doesn’t contain a common signal, then different methods can produce quite different results and I’m not convinced that much purpose is served by trying to decide which one is "right". I’m inclined to think that the lack of consistency in the results is the take-home message and, if you can’t get consistent results using somewhat similar methods, then you probably have to abandom the attempt to extract a "signal" from that data set and go back to improving the data. I’m going to do a post in the next few days on what happens with very noisy data sets, drawn from what will seem like a quite unexpected source.