Re #30 re #28. I am the same way. I have generally found z tests to underestimate errors, and like to construct simulations to understand what is going on. As to means, I have a theory that the temperature of the MWP from the reconstructions is related to the normalizing temperature for the series. If I get a chance I will collate and plot the zero anomoly temperature against the MWP temperature for a number of series and see if there is a positive slope! Once again, not proof, but suggestive of these proxies are simply expressing hidden ‘maintained hyportheses’.

Re #28, because I am a simple minded old guy (or maybe because I live most of my working life in a business world), I like to see the numbers and the math. So I try to build my simulations in spreadsheets first with a much simplified version. The “Can one reconstruct from principal components with short sample weighting” issue — that’s a mouthful — becomes very clear in the spreadsheet: no REs, no rho-squareds, not even Theil’s Us, just some simple graphs and the interocular test. The results hit you between the eyes: with weightings (the post-multiplying matrix) from the full sample (which can’t be done in reality), one can reconstruct; with weightings from the short sample, one can’t.

Note, this does not rule out reconstruction from one or a few proxies to one temperature series. But it cast considerable doubt on the reconstruction from many proxies to a many temperature or to a temperature aggregate series. Making this claim truly applicable resolves on how well one can model the range of reasonable specifications of temperature. If temperature were strictly periodic with, say, white noise and with constant spatial correlation, then short sampling would do pretty well. It is easy to go to a fully randomly periodic variation, but how does one move on the continuum between the two? Or how much should the spatial correlations be allowed to change over time? All these things make a big difference in the results. The MBH98 type of reconstruction is a complicated way of saying that the relevant temperature patterns of the past are sufficiently strong in the current instrument record that even if we had the full record, we would change how much to weight PC1, PC2, etc. to reconstruct temperature series A and so forth for series B, C … That condition is a “maintained hypothesis” (untested and assumed a priori to be true, and I think unstated) of the MBH98 model. They would say my type simulations only apply if the simulations could produce short sample weightings that reconstructed temperature approximately correctly. They would probably phrase it in the form that the work is irrelevant unless it simulated strong forcings that affected all series approximately the same way. {Ever think about why someone would have to run a climate model to test if the MBH98 transformation mined for hockey stick? Running a tree growth model (e.g. TreeGrOSS, PTAEDA, STIM, TREGRO — go take a look at it to see what the denrochronology guys are missing by not teaming up with the commercial forestry guys) or maybe something that David figured out in his kind of modeling might make sense as the underlying DGP, but not a climate model. It merely serves to restrict the DGP of the proxies, not estimate it from empirical data.}

Finally note, while we can never test the maintained hypothesis on temperature, we can do a “proxy” test, in both senses of the word. Can the proxy variable series be reconstructed from the short sample? If the proxy variable sets can’t be reconstructed by principal component analysis, then there is more than reasonable doubt as to efficacy of doing so on the temperatures. Any interest?

I’m still thinking about ways to extract what we want, btw. What I do for radar (ECM as noted) correlates noisy elements against a known signal or primary element, which is something not necessarily present in climate science (if we had a 1000 year true temperature record we could, but we don’t). perhaps we could “test” the known flawed proxies against the noisy ones to show the correlation between them?

Mark

John responds: Quite right! Equation updated. At least someone is awake on this blog….

re your interjection in the original article by Mark – I think you missed out a square on the second Y hat.

That there are lobes in the directional portion of the desired signal is irrelevant to this discussion. Where I noted similarity is the concept of finding the correlation across an array of proxies. In this sort of system, the correlated bits are actually the desired signals (as opposed to the noise we want to reject in a radar application, for example), sort of a reverse of the canceller problem. I.e. we are in a sense pulling the correlated signals out of the noise.

I typically used a form of a mofified Gram-Schmidt orthogonalization to implement the GSC and generate the weights – one per element or proxy in this case (do a search at the USPTO for “Gerlach, Karl”
and you’ll see the actual algorithm I’m running). The cascaded structure I use has computational benefits that aren’t important when the process is not real-time. However, I’m wondering if a similar approach can be used, without a main-lobe analogy, to find a correlation across a proxy set and “pull out” the signals from the background noise.

Hopefully we haven’t derailed to far off topic, Steve.

Mark

Even if the flawed proxies do not show up as noise at least they will be given reasonable weightings

The dangers of using voice recognition system.