Thinking about what you say, it seems to me that there should be no need to calibrate in this way. Surely if a proxy is proven to be good in relation to temperature, it should be included, and if not, then it should be discarded.

It seems that the PC1 Bristlecones, which are known not to be a good proxy for temperature in general, just happen to fit for the 20th century.

The other PC’s which by your given definition (because they do not match the calibration), are not such good proxies for 20th century temperatures should also be discarded for prior periods.

Then we are left with no reliable tree based proxy.

this is related to the trending/detrending problem. If I test my statistical method on the interannual variations, where I have much more degrees of freedom, I can in theory test the skill more robustly than just focusing on the 20th century trend, which is essentially just 1 degree of freedom. The problem even with detrended calibration, as Steve has explained elsewhere, is that the predictor network has 112 predictors, diminishing in time. I think that this guarantees overfitting.

I think we’ve talked about the danger from the calibration period before. The thing that I don’t understand is how to do it right or at least to assess what one is doing. To exlain:

I want to pick proxies that work as thermometers. Temp has gone up last century. So I pick proxies that have gone up last century. Then I look at what happens in the out years and see that tha result is much less varaible (shaft of the hockey stick). But how do I know that this is from a relevant thermometer and not just because proxies in general average out to zero and I just picked the few that had nice blades (to match recent temp increase) but that in past they average out to no impact. (Since they are not really that accurate a thermometer). How can I tell mathematically my likelihood of doing one or the other?

Eduardo, I take the fundamental point of von Storch et al 2004 as being that it’s essential to benchmark these multivariate methods to see how they work. It’s amazing that such an observation would cause controversy. And even if one is concerned with a network being too “tame”, if one doesn’t understand what happens in a tame network, how can one possibly understand what happens in a wild network?

I’ve been trying to spend time understanding exactly what happens in a "tame" network by experimenting with the erik167 network using different multivariate methods. The more time that I spend at it, the more foolish the hyper-ventilating of Wahl et al and Rahmsdorff seems. HAving said that, I think that there is some useful additional perspective from examining the coefficients resulting from the different methods (obtained in the MBH case by unpacking the algebra) . I’ll probably have a post up in a day or two.

I essentially agree, as you state in your posting, that a white noise error model for the real proxies is too tame, and methods should be shown to be robust by testing them with more complex error models. The MBH methodology has a number of “dangerous” aspects and the decentered pc-calculation is just one them. My hunch is, however, that the main aspect contributing to the hocke-stick shape is the overfitting in a calibration period with a strong trend. This would also mine data for strong trends in the calibration period.

if all the permutations have been carried out, why is it that random numbers put into the Mannomatic consistently apparently produce a hockey stick?

The thing to understand is that the calibration interval is fixed to the 20th century instrumental temperature which is essentially a HS blade. So the Mannomatic will think that only those “proxies” containing a temperature rise in that period are good proxies and will give them a high weighing. Since this rise is primarily the recovery from the LIA you’d expect that going backward we’d have higher temperatures. (Regression to the mean and all that.) So our fixed points are a highish temperature at the beginning, a low temperature in the 1800s and a highish temperature today. Take a bunch of Proxies, some of which have those three points and random noise in between and what do you get when you combine them; weighing those with the 3 magic points highly?– A hockey stick. That’s MBH98 in a nutshell. The off-center PC thingee primarily results in the shaft being smoother (no prominent MWP or LIA) whereas a regular PC allows more of the other features in the proxies to show up.