a. clearly label the features that distinguish real proxies from non-real with descriptors (e.g. upper treeline, or deciduous) into category.
b. cleary communicate the failure of the original hypothesis and preserve and communicate all the details of the overall study (the rejected i1 series).
c. validate the selection variables by new tests (in time or place).

http://www.null-hypothesis.co.uk/icecream.html

"Using Historical Climate Data to Evaluate Climate Trends: Issues of Statistical Inference "

It can be found here: http://www.ingentaconnect.com/content/mscp/ene/2004/00000015/00000001/art00002

"Abstract:

A strong case for global warming has been made based on reconstructed global climate histories. However, certain unique features of paleoclimate data make statistical inference problematic. Historical climate data have dating error of such a magnitude that combined series will really represent very long-term averages, which will flatten peaks in the reconstructed series. Similarly, dating error will prevent peaks (e.g., of the Medieval Warm Period) from multiple series from lining up precisely. Meta-analysis is proposed as a tool for dealing with dating uncertainty. While it is generally assumed that a proper null model for twentiethcentury climate is no trend, it is shown that the proper prior expectation based on past climate is that climate trends over a century period are likely. Climate data must be detrended before analysis to take this prior expectation into account."

It seems like a good way of avoiding a statistically meaningless analysis.

Steve: Thanks for drawing this to my attention. A point which definitely bothers me about Jacoby’s northern hemisphere temperature reconstruction, which I’ve mentioned here, is that he picked the 10 “most temperature-sensitive” of 36 sites studied and averaged them with an 11th selected site. If you have red noise series, this procedure will generate hockey stick shaped series. The multiproxy authors do not report how many series were canvassed before they selected their series, but if they examine 3 series for every 1 selected, I suspect that this will impart a selection bias sufficient to nearly always yield a hockey stick shaped series from red noise of proxy-type persistence.

Identification of Closed Loop Systems – Identifiability, Recursive Algoritms and Application to a Power Plant, Henk Aling, 1990, Dissertation Delft University.

This highly mathematical study tries to find constraints when an event in a power plant, say a pressure wave, can be traced to a source fluctuation (fuel or oxygen).

One of his conclusions:

“In practise the estimated covariance function of the joint output/input signal obtained by a closed loop experiment will [b]never[/b] have the structural properties associated with the feedback system. This is due to the finiteness of the dataset, model structure mismatch and other circumstances by which the ideal assumptions, used in the derivation of the identifiablity results are violated.”
(emphasis mine)

In other words, every feedback system has signals that cannot be attributed to a given forcing. The whole effort to match the mid-20th century cooling on aerosols is an example.

For more theoretical background – if you like heavy mathematics – the work of Kitoguro Akaike is a good start.
http://www.ism.ac.jp/~kitagawa/akaike-epaper.html