I’m progressing nicely with the process of parsing Ammann. The correlation between our AD1400 emulations of the MBH98 reconstructed PC1 is 0.9993201!
The emulations are virtually identical up to scaling/centering. W-A do not use archived MBH temperature PCs, but re-calculate them; this appears to account for scaling differences identified so far (and they should get washed out). I’m in the process of working through the downstream scaling/centering in W-A, which has some puzzling features in MBH98 methodology. For the AD1400 step (which is the one in controversy), there is only one PC in the reconstruction so the final NH reconstruction is going to be a linear transformation of the RPC1. So it looks almost certain that our emulation has been right on the money and that Wahl and Amman is an almost perfect replication of MM methods – and much closer to MM technically than to MBH. It would have been nice of them to acknowledge this. All of the outstanding questions about MBH98 methods which I’ve pointed out on this weblog and elsewhere will naturally remain outstanding even though the Hockey Team has ventured out of the foxhole with this code. I’m sure that you will all understand the many temptations to editorialize more but I’ll wait till I’ve done a little more on the code.
Figure 1. Comparisons of AD1400 Step RPC1 MM05 versus WA. Left – scatter plot; Right- RPC1s (not re-scaled). Here are some other odds and ends as progress to date.
Proxy Collation The order of the proxies in Ammann is a little different from MBH and Ammann provides no index for the proxies. In the AD1400 roster, the Stahle/SWM PC1 usually in the 16th spot occurs in the 22nd spot. I don’t plan to examine other steps. A first small difference – rounding before normalization of proxies. As far as I can tell, Mann’s practice is to normalize without rounding. Ammann sometimes rounds to the first decimal place before rounding. The differences can be as much as 0.15 deg C (e.g. below for the Tasmania temperature reconstruction. I don’t suppose that it matters much, but, if he’s trying to replicate, it would be easier to do little things the same. Otherwise, our proxy data set in our MBH98 replication is virtually identical to Ammann.
Figure 2. Difference between MM05 and WA Version of Tasmania Proxy (both normalized), due to varying rounding procedures
Temperature Principal Component Collation
Ammann’s approach seems pretty weird here. In his covering notes, they point out that an erroneous argument in MBH98 (also pointed out previously by others including us) that you needed to have more months than gridcells to do principal component calculations. You don’t. Ammann calculates principal components using annual series. The dataset used is an annualized version of the underlying temperature dataset (92 years x 1082 gridcells). Their version is available for download. Their dataset has been regularized to deal with missing data. MBH98 Corrigendum said that they interpolated missing values in the monthly series, but didn’t explain how they dealt with missing opening and closing values. The exact results are affected by the order of annualizing and regularizing, which is not provided. The selection of 1082 gridcells cannot be replicated, but this is not considered by Ammann. I haven’t checked the selections yet. The correlations between Ammann’s temperature PCs and Mann’s temperature PCs declines in lower order PCs: 0.997 0.965 0.922 -0.846 0.878 -0.059 -0.543 -0.284 -0.240 -0.772 -0.085 0.261 -0.342 0.079 -0.214 -0.189. Here are plots of temperature PCs 6:9. PCs can be calculated from any data set. It’s hard to believe that these PCs have any sort of physical permanence and can be replicated from different periods. Ammann certainly doesn’t examine this loose end.
Figure 3. Differences between Temperature PCs 6-9 between WA and MBH Archived
For what it’s worth, I was able to replicate MBH98 temperature PCs more closely than Ammann [link to Replication #x]. The correlation between the PC1s is very high, so this won’t impact the AD1400 step, but the discrepancies will presumably affect the regional reconstructions, which look pretty dubious to me. (For example, I’m pretty sure that, in the gridcell reconstruction for Vienna where there is a long history, the North American PC1 has more impact than the observed temperature – which seems like the ultimate in lousy RE. The actual temperature dataset is included as a “proxy” and is lost in the reconstruction. At least no one can accuse them of peeking at actual temperature records.) Let me re-iterate that the differences here are between WA and MBH; I’m not here comparing to our own emulations, as we used archived MBH values here.
The Mann-Ammann Cosine Error
There is much to be said for the Biblical advice to remove the “beam in your own eye before trying to remove the mote in your brother’s”. It’s easier said than done and I’m sure that inconsistencies on my part can be pointed out. But it’s still worth doing. Many of you will remember the huffing and puffing about a cosine error in McKitrick and Michaels – which was promptly identified because they made their script public. They promptly acknowledged the error and published a Corrigendum, including an impact assessment. (Despite this, in an effort to block publication of our materials, Mann made false allegations to Natuurwetenschap & Techiek, stating that McKitrick and Michaels had failed to issue a correction notice, but that’s another story.) There has been a rather delicious latent cosine error in MBH98. Von Storch et al.  pointed out that, for the purposes of their EOF calculations, they should have normalized by the square root of the cosine rather than the cosine. I’ve seen a reference to the need to use a square root of the cosine in an article by Wallace of University of Washington [link to come]. If Ammann was not trying to exactly replicate MBH98 temperature PCs (since he has not done so), you would think they would have tidied up this error. The data set used for temperature PC calculation has already been standardized and regularized and there is no code provided for these processes. I’m not sure when I’ll get to trying to reconcile these steps. They don’t appear critical, but, given that they are on much better behaviour about code, it would have been nice to see.
Calibration Test The correlation between our coefficents and theirs is 0.9992717. There’s a difference in scaling – I’m not sure why right now, but I can’t see why it would matter (since it merely affects downstream coefficients).
igure 4. Scatterplot of MM and WA Calibration Coefficients in AD1400 Step
All in all, this is looking pretty good so far. One could scarcely have contemplated that WA should replicate our results so accurately. The irony is delicious.