Bring the discussion of a couple of days ago further forward, here is a very interesting graphic from Ravelo and Ward [2004] showing the transition from Pliocene to the Pleistocene (read R to L), which gives a somewhat different perspective on the Pleistocene than just looking at the Vostok core back to 800K (although that’s interesting as well).
Figure 1. Ravelo and Ward Figure 1 with original caption. Continue reading
I gave a speech last week at my tennis/squash club on climate change (which presents speeches from time to time). I included some geological concepts that I don’t usually have an opportunity to talk about. The class of scientist who tend to be most unimpressed with IPCC-type climate science are geologists – which is where I got started in this. If you took an Oreskes-type survey among geologists, I don’t believe for a minute that you would get anything like IPCC solidarity. Unlike most scientists, geologists also happen to know a lot about climate history. Here’s an interesting graphic that I used: 
Continue reading
I notice that there has been interest recently in the question of the difference between (shortwave) solar radiation and (longwave) infrared radiation as it affects ocean heating. Water is essentially opaque to infrared radiation, while shortwave radiation (especially in the blue wavelengths) can penetrate to substantial depths. realclimate takes the position that this doesn’t matter. I get the impression that GCMs, used in IPCC climate modeling, treat both shortwave and longwave radiation identically. I don’t propose to comment on these matters, but I happen to have read some interesting specialist articles on the topic, which are web available. It doesn’t look to me like the realclimate position on this matter is a gimme. There are some interesting connections between biological (phytoplankton) activity and penetration. Anyway here are a few articles that I’ve seen – no representations as to completeness – and which are available on the Internet. Continue reading
I’m now completely file compatible with Wahl and Ammann. In terms of my algorithm, I had to tweak the procedure for scaling RPCs a little, but the RPCs themselves were identical. It took some patience to reconcile differing data setups. I archived results from a complete run-through from their base case a couple of days ago. They have not archived results yet, but the graphs look very similar.
Our standing prediction for MBH98 type reconstructions is that that verification statistics other than the RE statistic will be insignificant. I suppose that another prediction is that the Hockey Team will not report these other verification statistics. Both predictions are correct here. Needless to say, only the RE statistic is at Ammann’s website. The archived program does not even calculate other statistics, although it’s hard to imagine that they have not peeked to see what the other statistics are.
As someone with prospectus experience, I think that these statistics are highly relevant to readers and that they should not be withheld by the Hockey Team. If they then want to argue that verification statistics other than the RE statistic are incorrect, then let them. (They will then have to defend this position against their own use of these other statistics in their other writings where it is to their advantage.) Anyway, here are my calculations for standard verification statistics for the AD1400 step from the Wahl-Amman run-through. The cross-validation R2 is virtually 0, the sign is correct only 54% of the time just better than random, the product mean test has a t-distribution and is insignificant, the CE statistic is negative. No wonder they don’t want anyone to look at these statistics.
| RE (cal) | RE(ver) | R2 (ver) | CE | Sign Test | Product Test | |
| Run-Through | 0.16 | 0.47 | 0.02 | -0.24 | 0.54 | 0.91 |
| Reported | 0.39 | 0.48 | NA | NA | NA | NA |
I’ve done a complete run using the WA basecase using the Windows-modified script that I archived a couple of days ago. From the output of this run, I extracted the results of the AD1400 step and made a stepwise splice, which I’ve saved here together with the MBH98 stepwise reconstruction. This may be of interest to some people. Here is a smoothed version of the MBH98 stepwise reconstruction and the WA stepwise reconstruction. While it substantially emulates MBH98 results (as did our emulation), you’ll notice the divergence in the problematic early 15th century, which I also had a problem with. There’s no reason why exact replication should not be possible. Obviously this replication does not deal with issues like bristlecone pines; this is just benchmarking. Continue reading
As I discussed previously here, I had been able to quite accurately replicate Mann’s temperature principal components (the first 16 of which have been archived for a long time). Notwithstanding that these calculations can actually be replicated quite accurately, Wahl and Ammann calculated new reference temperature PCs using annual data on the basis that they get similar results. Perhaps. But if the purpose is replication, it makes a lot more sense to use the MBH98 temperature PCs (or at least replications of them) . Also pardon me for being a bit skeptical about Hockey Team assurances that something “doesn’t matter”. I’m getting to find my way around WA code a bit better now and I substituted the Mann PCs for the WA PCs and carried on. The normalized proxies used by WA were very slightly different than mine, owing to slight differences in the order of rounding and normalization: in some cases, they rounded before normalizing, while I didn’t (I think that my interpretation of MBH methods is the more probable).
Then I re-did the calibration-estimation calculation up to Reconstructed Principal Components. In Early Returns, based on Wahl-Amman PCs, I reported that there was a correlation >0.999 between the two calculations, but scaling differences. Well, using MBH98 PCs and non-rounded proxies, guess what: the replication of the RPC1 was exact. I mean to 10 decimal places for all 581 values!
You’d think that this would have been worth a mention in their press release and article. I’m still wading through the downstream scaling to NH temperature. Because the scales were all different using the WA PCs, it complicated this process considerably and I didn’t finish that yesterday. However, now that I’ve got everything on the same scales, de-coding the NH scaling should come a little faster today. For reference, the incisions were surgical on the following: svdproduct$TmatU, svdproduct$TmatV, svdproduct$TmatD (temperature PCs) and datamatrices$Ymat.anomaly (proxies).
I’m working on the 15th century step only for now. I’ve unpacked the functions into a running script.
Afternoon Update: I think that I’ve pretty much reconciled the scaling to NH temperature.
This has always been a bit mysterious – Ammann’s code shows a scaling step added in April 2005, so he seems to have had some trouble here as well. Von Storch et al. seemed unaware that there was a scaling procedure in MBH98 – I mentioned this in an earlier post [here]. In our previous calculations, because I couldn’t get the scaling to reconcile, I ended up simply re-scaling the emulation to NH temperature. I should be able to eliminate this plug in my emulation, which will be nice. I anticipate being pretty much compatible with WA by tomorrow. I’ve done some preliminary runs and obtained R2 and RE results pretty much identical to previously reported results. Once I’ve got complete replication, I’ll re-execute all reported results and report on them. That may be a few days as I’ve got a few other things to do simultaneously. It’s interesting to see some programming approaches in WA. I like some ways that they handle output logfiles.
On the other hand, their handling of matrices is often very awkward. For example, there is a profusion of differently named matrices such as “Tmat.anomaly.cal.fitted.NH.mean.corrected” and “Tmat.anomaly.precal.fitted.NH.mean.corrected”. The phrase *.cal.* and *.precal.* distinguish matrices in the calibration period 1901-1980 and the “pre-calibration” period 1400-1901. These could have been combined into one matrix using an index for the calibration period (or other period) e.g. Tmat.anomaly..fitted.NH.mean.corrected [cal,]. This would eliminate needless duplication. I also don’t like their handling of situations where each column of a matrix is multiplied by an element of vector. For example, if A is a matrix and v is a vector, the simplest way of doing this is A %*% diag(v), (%*% is matrix multiplication). WA typically will do t( t(A)*v). It works but you lose sight of what’s going on, especially when they create new and unnecessary matrix entities with *.transpose suffix. These are not big criticisms, but they are matters that could have been done a little simpler. (I’m sure that there are lots of criticisms of my coding style, but these are just things that I noticed and had to think about to figure out what they were doing.)
I’m progressing nicely with the process of parsing Ammann. The correlation between our AD1400 emulations of the MBH98 reconstructed PC1 is 0.9993201!
So far there have been 4 Comments submitted to GRL on our paper, including Wahl and Amman which is 4th in the queue, plus the WA submission to Climatic Change. Se we’re being noticed. The GRL format for a Comment is 50% of the length of the original article, with an equal length allocated to the Reply. In each case, we have been invited to reply. So there’s a lot of aftermarket work dealing with this deluge of comment. It’s curious that the climate science community has suddenly discovered due diligence. Too bad that similar energy didn’t go into verifying MBH98 in the first place. Nobody’s submitted anything that IMHO lays a glove on our findings or that hasn’t been anticipated in some form. Michael Jankowski in a comment here pointed out one such example where we had anticipated and reported on statistical insignificance, a matter which Ammann and Wahl go on and on about without acknowledging our prior findings.
Wahl and Amman code is designed for Unix. I’ve done the tweaks for Windows, doing tweaks for both the masterfile (for their base case) and for the file of directories. There might be some simpler way of doing the tweaks, but this version doesn’t crash on my machine. This should transport OK to other machines. I presume that many users (like me) use Windows. So here you go. Now I’m seeing what it does. Windows Wahl and Amman Master Windows Input Parameters
Climate Audit 
A small observation…
Realclimate have removed their links to Roger Pielke’s Prometheus blog, as a demonstration of their commitment to rational, scientific dialogue.
Update: They’ve put the link back now. It looks like the tantrum is over. Just another ripple in the ocean of climate science.