Most CA readers are aware that proxy reconstructions use linear methods and that, accordingly, all the huffing and puffing of complicated multivariate methodologies simply end up assigning a vector of weights. Surprisingly this obvious point was not understood by paleos when I started in this field.
Because one can assign a vector of weights, it’s possible to make highly informative maps showing the weights of proxies by the size of the disk at the proxy location, designating the sign by a different color. Unfortunately, this sensible practice of examining proxy weights has not been adopted by paleos. Their failure to show proxy weights inevitably leads to quite a bit of (in my opinion) aimless thrashing, with the Smith paper being a recent example.
Smith was perplexed by the difference between McShane-Wyner reconstruction. The figure below shows what bothered him. In this case, retaining one PC led to an MBH-style Stick, while retaining 10 PCs had a pronounced MWP.
Figure 1. Mc-W Figure 14. Red – with one PC; green – with 10 PCs.
Last fall, I tweaked the McShane Wyner code so that weights of the various proxies was extracted in the process. I prepared the graphics below last October but didn’t post them up at the time – I guess that I must have gotten distracted by something else.
The figure below shows the weights for the “red” reconstruction (made from one retained PC.) The overwhelming weighting of US southwestern tree ring chronologies is evident. The largest weights are in gridcells with bristlecones. Nothing else really contributes. This is a classic distribution of weights in a Mann network. MBH reconstructions in its various guises also weight the bristlecones. The reason why supposedly “independent” reconstructions look so similar is that the proxies are commonly not independent. The Mann et al 2008 network of 1209 proxies contains the same Graybill bristlecone as MBH98.
Notice that the Central American lake sediment series (which have a prominent MWP) are flipped over. This is a result of the PC algorithm. Flipping them over makes them line up better with the bristlecones.
Figure 2. Weights for Red Reconstruction. Orange dot is only because red dot was too big. Red – positive weight; blue negative weight.
Update – For amusement, here are the weights for Gavin Schmidt’s selection of 55 of 93 proxies using the McShane Wyner Figure 14 method. Needless to say, Gavin’s reconstruction is fully addicted to bristlecones. For greater certainty, Gavin flipped over the Central American lake sediment series that offended Smith.
The green reconstruction is not dominated by bristlecones. More prominent are sediment series in Central America, a speleothem in Yemen and some Chinese speleothem series. This time, there are a number of negatively oriented series – some tree ring series in the southeast US that were originally reported as precipitation proxies, a speleothem in Scotland. The Tiljander series are flipped over to cohere better with lake sediments in Central America.
Figure 3. Weights for Green Reconstruction.
Obviously lots of readers will “like” the green series, but it’s not clear to me that it makes any more sense than red reconstruction.
In my opinion, the problem is that you can’t simply throw a bunch of inconsistent time series into a multivariate mannomatic and expect to get a statistically significant response. If a scientist cannot specify the sign of a proxy in advance, then the proxy shouldn’t be used.
Climate Audit 
45 Comments
Could the use of a Mannomatic algorithm be viewed as being something equivalent to the parliamentary process of voting by proxy?
Moreover, could one vote successfully by proxy if one couldn’t identify in advance the respective definitions of the words “yea” and “nay”?
What a beautiful picture!
Entirely agree. But one has to admit that (in the UK anyway) Red means stop your car and Green means you can drive just as fast as were already. Not for the first time I understand those who feel that you’re reinforcing our fossil-fuel addiction. But it can’t be helped, if the ‘science’ telling us to give it all up is so pathetic.
This might be dumb, but should not the grey shaded area be extended to completely enclose the red and the green? If there is no described cause/effect relationship between the mechanism and the math, apart from the assumption of stationarity, the error envelope should surely enclose most of the credible math.
I entirely agree with your last paragraph Steve. We are not going to make much, if any progress, in determining palaeotemperatures until we have good constitutive models for the response of proxies to temperature. Such models will allow us predict a-priori the sign of a response with ,for example, an increase in temperature. What we have at the moment are crude phenomenological descriptions in which the phenomenology is allowed to change over time.
ONe amusing example of opportunistic sign-changing in Mann et al 2008 – I forgot whether I posted on this or not – is that one speleothem has opposite orientations depending on whether it is “late calibration” or “early calibration”. This has a noticeable impact on statistical performance, but is blithely ignored by the Team.
A first hit:
https://climateaudit.org/2009/04/15/more-upside-down-mann/
A second hit:
> [B]ack in 2008, I’d noted that the M08 algorithm permitted the same proxy to have opposite orientation depending on calibration period and that at least one proxy did this. Note the Socotra (Yemen) speleothem in the weight map. This has opposite orientations in the two reconstructions – something that seems hard to justify on a priori reasoning and which appears to have a noticeable impact on the differing appearance of the two reconstructions.
https://climateaudit.org/2010/08/07/mann-and-his-bristlecones/
There might be another hit in 2008, but we’ll stick to two links.
Another thing I’ve had a lot of doubt about is the accuracy of the scaling of the proxy values into temperature values. An overlap with instrumental data of only a century doesn’t seem like a very accurate method due to the large amount of noise in both proxy and T data.
In factor analysis it doesn’t matter what the sign and magnitude of any time series weighting factors are. Factor analysis estimates how much of the variability is due to common ‘factors’ (unobserved variables) that usually number less than the observed number of variables and can have arbitrary magnitude and sign.
As factor analysis and PCA are closely linked I’m not sure that what appears to be a gross adjustment of the input variables – sign change – is actually bad. Note I’m not fully certan certain and it’s been decades since I did factor analysis. YMMV.
Oops Variance, not variability.
I should also mention that the factors are orthogonal.
I don’t think that the behavior of the novel and untested mannian short-centered PCA can be compared to PCA. It should be a relatively easy experiment to demonstrate that sign inversion matters or not in the parallel universe of mannian PCA.
Steve,
I think the Caption for Fig 3 should read “Weights for GREEN reconstruction”.
It still amazes me after all this time that anyone can take tree rings of any kind of tree from any location seriously as a true temperature proxy.
And I completely agree with your last paragraph – unless you know the sign of a proxy from first principles, then all that you’re doing is a sophisticated form of divination from entrails.
I would go even further; unless you can pre-assign both sign and magnitude of a proxy response to temperature change, based on physical understanding of the processes involved for each proxy, then the entire exercise is little better than divination from entrails (which is not at all a sophisticated methodology). It is all absurd rubbish.
Re: stevefitzpatrick (Jun 10 07:51),
Mann’s statement in his Reply to our PNAS comment was amusing. This was in the context of our observation of upside-down Tiljander.
Mann et al 2008 CPS only permitted sign-flipping for speleothems, sediments and documentary. They specified that tree ring correlations had to be positive – an opposite approach to MBH. This had an odd impact on some tree ring species where there was a known negative relationship to temperature e.g. Woodhouse and Brown’s post oaks in the US plains. All but one of 16 were screened out. They kept one quirky series that had a fluky positive correlation. Needless to say, the specialists maintained the silence of the lambs.
1. Demonstrate the proxies ACTUALLY represent temperature over the FULL time period
2. Collect as many proven reliable proxies as you can – dont ignore ones that “dont look right”
3. Don’t flip/crop/reverse them
4. Create an area weighted average of them or just simply average them if there’s even coverage
5. Present results without hiding anything and let the results speak for themselves
6. Compare with other known geological / historical data and report the results
Is it that hard????
Man, the reply thing isn’t working at all correctly for me, always drops me down to the bottom.
https://climateaudit.org/2011/06/09/mcshane-and-wyner-weights/?replytocom=285707#comment-285707
I think it is hard because they don’t know the temperatures for the full lengths of the series’, which is why they have to come up with proxies. It’s always going to be an assumption. But as Steve and others point out, if you have no established mechanism as to why tree rings, lake sediments, or anything else accurately (or even adequately) measure only or even mostly local temperature, then entrail divination might actually be MORE accurate.
“If a scientist cannot specify the sign of a proxy in advance, then the proxy shouldn’t be used.” It is curious how that one sentence, and how it applies to climate reconstructions, can be so devastating. Yet it is.
Are there any proxies where it’s plausible to allow the sign of the proxy to be flipped automatically to improve the fit?
If I stretch my imagination, I can conjure up some sort of symbiotic / parasitic relationship that would fit the bill; a plant that’s free-standing shows growth spurts when the temperature increases; the same plant is constrained in it’s growth if it’s adjacent to a larger plant / tree that itself has a marked response to temperature increases and so robs it of nutrients.
Even then I presume you’d have to state your expectations / classify differently according to proximity (and of course it’s a tortuously made up “example”)
So are there known proxies of this sort?
There are the three M&W10 reconstructions of Fig. 1 (M&W10 Fig. 14) — 1 proxy PC (red), 10 proxy PCs (green), and a two-stage model (1 local temp PC and 10 proxy PCs; blue).
Then there are the proxy-weight maps (Figs. 2 & 3), with red and blue dots.
For these figures, I believe that a red-colored dot signifies that the proxy is oriented in the same manner as was proposed in M&W10, who simply took this information from Mann et al (2008). For the 19 longest-running proxies, Mann08 specifies these orientations in Figure S9 of the Supplemental Information; the specs for the entire set are in an Excel file, IIRC.
I believe that a blue-colored dot in Fig. 2 signifies that for the 1 PC reconstruction, the orientation of the proxy was flipped (i.e. inverted), with respect to how it was proposed to be oriented in Mann08.
Likewise for Fig. 3 dots and the 10 PC reconstruction.
In other words, consider a proxy X, where the dot for that proxy is red in Fig. 2 and blue in Fig. 3. To construct the red curve of Fig. 1/14, M&W10 used X such that a higher (e.g.) signal in a given year was interpreted to indicate a higher temperature for that year. To construct the blue curve of Fig. 1/14, X was used in the flipped orientation, such that a lower signal indicated a higher temperature.
Is this correct?
If so, it’s noteworthy that M&W10 don’t seem to have discussed this seemingly bizarre feature of this approach to multiproxy reconstructions. Searches of the PDF for strings beginning with backward, orient, upside, flip, andrevers come up empty.
Update – For amusement, here are the weights for Gavin Schmidt’s selection of 55 of 93 proxies using the McShane Wyner Figure 14 method. Needless to say, Gavin’s reconstruction is fully addicted to bristlecones. For greater certainty, Gavin flipped over the Central American lake sediment series that offended Smith.
Re: AMac (Jun 10 07:27),
AMac,
it’s a bit unfair to complain about McShane and Wyner not calculating the final weights of proxies through the many multivariate operations as no one in the field does so. Indeed, the very possibility of doing so was contested early on. I’m not sure that this affects all of their analysis either. As I recall, they kind of shift gears prior to Figure 14 – this should be parsed.
In a proxy situation with “sensibly spaced” proxies, the coefficients of the PC1 (the eigenvector) should all be positive with an ex ante orientation. Any lower PCs will then be contrasts i.e. half the coefficients will be positive and half will be negative. I’ve mulled over this phenomenon for a long time and it seems to me that in a network of actual proxies, any introduction of lower order PCs – even a PC2 – will degrade the reconstruction. See my 2006 post on this in the zorita tag.
If you don’t have a “sensible” network i.e. one region is over-represented, then that distorts things. If the over-represented region is made up of known problematic proxies (bristlecones), then the distortion is even greater.
These sorts of maps and analysis, though simple and obvious, is thus far unique to Climate Audit. None of the specialists use them. So McShane and Wyner are hardly alone. In my implementations of the code, I’ve learned to take care with the linear algebra. Reconstructions have a concatenation of right matrices. The typical approach in a proxy reconstruction is to successively multiply the matrix of proxy one by one by the right matrices. I collect the right matrices and get weights.
Could you provide an explanation of the meaning of Dot Color in Figs. 2 and 3, supra?
Specifically, does Red in Fig. 3 (green 10-PC reconstruction) signify “agreement” with the orientation claimed by Mann08, with respect to PC1?
This would be consistent with your remark, “In a proxy situation with ‘sensibly spaced’ proxies, the coefficients of the PC1 (the eigenvector) should all be positive with an ex ante orientation.” And also consistent with common sense.
However, the meaning of red and blue Dot Color would get complicated, if the orientations of certain proxies weren’t consistent even within Mann08, from figure to figure. (Your remark, “Mann et al 2008 CPS only permitted sign-flipping for speleothems, sediments and documentary.”)
Also, Mike Edwards (Jun 10, 2011 at 2:15 AM) appears to have spotted a typo. Caption for Fig 3 likely should read “Weights for
redgreen reconstruction”.red is a positive coefficient to the series as archived; blue is negative.
Mann 2008 presumes a positive correlation with tree rings, ice cores in screening, but chooses sign for speleothems by correlation. I need to refresh on what they did with sediments.
The issues in these various methods are all a little different. For example, ex post screening of seemingly like series is an issue with Mann et al 2008. Opportunistic flipping is less of an issue. However, it shouldn’t be an issue at all. It’s a particular issue with speleothems in Mann et al 2008.
Or to make Steve’s explanation simpler, a red dot means read the proxy as a high value means high temp, low value means low temp. A blue dot means the proxy is opposite – a high value means low temp and a low value means high temp. The size of the dot represents how much weight the values from the the proxy are given in the analysis part of the system. Strip bark pines are more important than anything else in Mann’s PC1 analysis, as well as Gavin’s. In the McS & W PC10 analysis, weights are varied but there is no one truly dominant series. Strip bark actually becomes a minor contributor.
“If a scientist cannot specify the sign of a proxy in advance, then the proxy shouldn’t be used.”
This principle seems so self-evident that I am indeed optimistic that Mann and his colleagues might be willing to concede the point sometime in the next several decades 🙂
Perhaps then we could discuss a second principle that seems relevant: “Scientists should have objective criteria to guide (a priori) their decisions to include or exclude plausible, readily available proxies.”
If one wished to predict the stock market based on various measures such as camping gear sales, home prices, divorce rates, etc. it might not matter what their sign was (though retrodiction would be wildly iffy). But if we are using biological and physical responses, negative sign means that sometimes trees grow better when it is warmer but some trees grow worse when it is warmer, and there is no way to divine which trees are which a priori. Also, that sometimes mineral ratios in sediments mean warm and sometimes cold. It isn’t just “data” it has meaning. Finally, the maps Steve has made show that the global temperature ends up being simply predicted by bristlecone growth, with data from the rest of the world ignored. That is piling absurd on top of absurd.
“If one wished to predict the stock market based on various measures such as camping gear sales, home prices, divorce rates, etc. it might not matter what their sign was (though retrodiction would be wildly iffy).”
If someone tried to predict the stock market based on such data I would dismiss the effort as an example of “data mining”. Obviously, from the universe of time-series data one can find some that are spuriously correlated with the stock market. Accordingly, I would insist that the inclusion of such predictors be supported by a plausible economic theory, which by necessity would include an a priori prediction of the indicator’s sign.
I mention this not to be picky but simply because among economists, the problem of data mining is well understood and explained in introductory courses in econometrics. Since my academic background is in economics, I have been bewildered why this simple concept is not better understood by scientists.
Re: Craig Loehle (Jun 10 10:12),
We cited some economics literature on data mining in our early articles – Ferson et al springs to mind. Also Phillips on spurious correlation.
COming from a stock market background, I’m very aware of the risks of data snooping and data mining. I find the re-use of proxies with known properties and representing the results as “independent” as very objectionable, but it’s deeply ingrained in the field.
Here’s a useful graphic that I did back in 2006 showing the contribution of different proxy “classes” to MBH98,yet another way of showing that the “other” proxies are irrelevant to the reconstructions.
Figure: Spaghetti graph showing top- absolute contribution to MBH98 reconstruction (1400-1980 for AD1400 step proxies) by the following groups: Asian tree rings; Australia tree rings; European ice core; Bristlecones (and Gaspé); Greenland ice core; non-bristlecone North American tree rings; South American ice core; South American tree rings. Bottom – all 9 contributors standardized.
Shockingly obvious. In my field, should a graph would be interpreted to suggest the red data has problems or is measuring something different than the other sources.
It looks like the variability of the non-bristlecone proxies increases when merged with the bristle cone data. When the proxies are run separately, the other proxies are a relatively tight bunch, with the variability being roughly +/- 1 from the mean. The bristlecone proxies runs across the map, roughly +/- 3 or 4 from the mean. When the two are combined, the variance increases for the entire set to +/- 4. What this suggests is that the amplitudes of the non-bristlecone proxies were increased to match the variance in the bristlecone data, to make it not look like such an outlier.
Nice, this is a very helpful graphic for those of us coming to these topics more recently. The 2006 CA article at your link is also valuable — those geographic plots of proxy weights really tell a tale.
The climate history (and current weather) of the US Southwest (bristlecone pine country) is almost always significantly at odds with the rest of the continent. If I understand the discussion, bristlecone pines have basically served as the ‘IPCC’ proxy of US climate (or global climate) from 600 years ago. If so, that is quite interesting to me, if only for the reason that this period 600 years ago reflected a relatively brief cooling period of the US Southwest. That was a mere hiccup within a long 15,000 year (give or take a few thousand) march from overall subglacial wet and cool predominant climate to the current state of dominant and extensive aridity in that region. (see W. Dick-Peddie, 1993 “New Mexico Climate. Past Present and Future” UNM Press, Albuquerque NM.
It seems almost impossible that a series can just be flipped to fit better when, as others have stated, these numbers are based on real physical or biological processes. Are you absolutely sure this is happening?
As I understand it individual series are produced by other scientists than those producing the multiproxy reconstructions. Don’t these scientists flip themselves when they see their hard work being (mis)used in this way?
Absolutely sure
Start here for the Tiljander lake sediment story:
https://climateaudit.org/2008/10/02/its-saturday-night-live/
If you track this through to the current state-of-play, you will see that the original authors of the Tiljander paper are outraged that their hard-won proxy series was flippantly flipped
It’s always refreshing to see someone else’s incredulous reaction to things that regular observers have long taken for granted.
How anyone can argue that this practice is acceptable defies belief.
HR (Jun 11, 2011 @ 6:13pm) —
If you take the advice Ianl8888 offered (Jun 13, 2011 @ 8:05pm) and look at the Tiljander saga, note also that there are only three independent Lake Korttajarvi data series, although Mann08 uses four. “Darksum” is a computed value, derived by Tiljander as “thickness” minus “lightsum” (all are varve thicknesses in millimeters). (XRD is the final data series).
It was almost certainly another simple and innocent error — the relationship was obscured by the common practice of log-transforming these series before adding them to the proxyhopper.
This mistake is easily understood and verified. However, to my knowledge, it has never been addressed or acknowledged by Prof. Mann, any of his co-authors, or any advocate of the pro-AGW Consensus position.
Most of the dendro guys (and gals) seem to have their ears full of their fingers. There is little evidence they are listening to the type of conversation going on here. Scientists like Craig Loehle need to work on a strategy to overcome this. Doesn’t look like an easy pine cone to crack.
I have pretty much washed my hands of tree ring data. Too many confounding factors. A summary of my work on this topic is now available from Amazon “Time Series Analysis and Climate Change”
http://www.amazon.com/Time-Analysis-Climate-Change-Detection/dp/3844392564/ref=sr_1_23?s=books&ie=UTF8&qid=1307917222&sr=1-23
Still waiting to hear from the experts on this thread.
(sarc).
Changing the weightings is exactly what is done when training a neural net (curve fitting).
You can achieve any result (curve) you want by selecting the appropriate weightings. If you don’t get the result you want, change the weightings and try again.
It is a simple matter to write a computer program to do this, to select the weighting through trial and error, to converge on the result you want.
Thus, if I was a climate science researcher that wanted to obtain a “hockey stick” shape to eliminate the MWP and LIA, I could do this simply by varying the weightings in my analysis.
However, this is not science. It is marketing (advocacy). You maximize the data that supports your point of view (by increasing the weightings), while minimizing the data that contradicts your point of view (by reducing the weightings).
If one looks at the example above labelled “Gavin Schmidt versions”, you can see that the Pacific South West sample is much larger than all the others.
It would be like giving Liechtenstein greater weighting than India, China and Brazil when assessing population rends.
This suggests that Gavin is engaged in marketing, not in science.
If the weighted mean of proxy series is a valid estimate of the global mean temperature, then it is surely a short and logical step to assume that the same weighted mean of actual observed temperatures is also a valid estimate of the global mean temperature. Indeed it may even be better than the more commonly used spatially weighted mean. I wonder if anyone has tried this? I mean take whatever temperature series are available for the grid squares with proxies (for the last 50-100 years maybe) and combine them with the PCA weights. This new “global mean” temperature could then be compared with the usual one. If the series are very similar then there is some possibility that the PCA method works. If not, we can draw another conclusion.
A useless exercise: http://www.uoguelph.ca/~rmckitri/research/globaltemp/GlobTemp.JNET.pdf
Well…I was trying to suggest a way of demonstrating the absurdity of believing that a weighted average of a few points gave a meaningful estimate of the global mean. But I guess the irony didn’t come across. However, the argument that the global mean temperature isn’t a measure of anything with a physical meaning applies equally well to the whole proxy reconstruction concept.
2 Trackbacks
[…] McShane and Wyner Weights on Mann 2008 Proxies Steve McIntyre, Climate Audit, 9 June 2011 Cancel reply […]
[…] Source: https://climateaudit.org/2011/06/09/mcshane-and-wyner-weights/ […]