Nature's Style: "Naturally Orthogonal"?

Wilson and Luckman 2003 observe:

The first PCs from the RW and MXD PCA are naturally orthogonal (r = —0.006) over the 1900—1991 period suggesting the in’?fluence of different forcing mechanisms upon these parameters.

They move on without pausing here, but this point should not be left without a commentary. The issue here is one that I’ve mentioned in passing before, but this is a very nice example. Both RW chronologies and MXD chronologies are supposed to be temperature proxies. The “noise” in dendro proxies is widely believed (e.g. realclimate here ) to be at worst low-order red noise – also see the controversy between Wahl et al and VZ on this. Think for a moment about whether these positions are compatible in this case.

The answer is that they are not compatible. If the two series y_{rw} and y_{mxd} are each correlated to temperature x , write the following for i in {rw,mxd}

(1) y_i = \alpha_i + \beta_i x + \epsilon_i

then the condition of y_{rw} being uncorrelated to y_{mxd} requires that there be a relationship between y_{rw} being uncorrelated to \epsilon_{mxd} and vice versa i.e. that the true relationship to temperature is between temperature and y_{rw}+y_{mxd} or some other linear combination. I’m not saying that there is any such relationship, only that once you’ve observed that the RW and MXD chronologies are “naturally orthogonal”, you can’t just drive on. You’ve got to explain how two naturally orthogonal series can both be temperature proxies. I showed the example of the sum of RW and MXD as a possibility, not because I have any reason to think that there’s any magic linear relationship between RW and MXD that resolves the problem, but an article on Varimax Rotation discussed recently raised an example where there was a correlation between two uncorrelated series and third series, which they posited as being a result of the third series being dependent on the difference between the two uncorrelated series. Houghton et al 1991 (discussed here recently) observed of Atlantic SSTs:

The Atlantic dipole index is the difference of two uncorrelated time series. … Our results are not incompatible with the correlation calculations of Moura and Shukla 1981 which show that high rainfall in NE Brazil is associated with warm (cold) SST south (north) of the ITCZ. Consider two uncorrelated time series N[t] and S[t] such that their time-average product =0. Now define a third time series D[t]=N[t]-S[t]. It is easily shown that = ||N||^2 and = -||S||^2. We have constructed an example where two uncorrelated time series are correlated with opposite sign to a third. If to some degree the NE Brazil rainfall were related to the N-S difference, it would explain the observed rainfall-SST correlation.

Again, I’m not saying that this is going on in this particular case. I don’t know what’s going on. But you can’t just observe that two “proxies” are uncorrelated and drive on as though nothing has happened. In passing, whatever the actual answer is, it is impossible that the “noise” is in each case uncorrelated low-order red noise. If so, the two “proxies” would be correlated. So this example would seem to be a rather neat disproof of the Rasmusian post at realclimate on tree ring red noise, a point which any visiting dendroclimatologists might make over there.

It’s worth observing that, at a site level, dendrochronologists test the presence of a signal through expedients like average mean correlation. I think that this precaution would be worthwhile when one is seeking to combine RW and MXD series as well. A while ago, I posted up some plots showing joint distributions of RW and MXD measurements for Jaemtland – no special significance to this example. Obviously neither the marginal nor the joint distributions are normal (BTW for interested people, the topic of a copula function linking marginal distributions needs to be studied in this context). These RW distributions are before age adjustment and so I also did a plot – see following figure – with log RW to allow for some sort of negative exponential effect in aging.


Figure 1. Jeamtland distrbutions from http://www.climateaudit.org/?p=411

The left hand panel shows the RW-MXD plot and the right panel the log(RW)- MXD distribution. Are the joint RW-MXD distributions normal? Obviously not.

Figure 2 – Jaemtland distributions. Left – MXD vs RW; right MXD vs log(RW,2)

The above plots are for individual measurements. What happens when you do the same thing for chronologies – here’s a similar plot for Tornetrask and Urals chronologies. My impression of the distributions is that any relationship between the Urals MXD chronology and RW chronology is heavily dependent on the tail for what are presumably very cold years – years without a summer, that type of thing. In “ordinary” years, even if you know the value of the MXD chronology, it doesn’t appear that you can conclude very much about the value of the RW chronology. If that’s the case, how does one know very much about the temperature? (With distributions like this, calibrations involving the cold 19th century probably over-emphasize the tail and exaggerate the goodness of fit.


Figure 3. Distribution of RW-MXD chronologies from http://www.climateaudit.org/?p=733

What does it all mean? I don’t know.

However, I will venture to say that “naturally orthogonal” are not a good thing for reconstructions. I first noticed “naturally orthogonal” proxies in connection with the AD1400 MBH98 proxy network, which remarkably, is also close to being “naturally orthogonal” – a point that I think that I’ve made previously on the blog. I’ve shown that you can easily get MBH type reconstructions with one spurious trending series and 21 white noise series in a network. If the proxy network is “naturally orthogonal”, then Mannian Partial Least Squares regression becomes identical to OLS regression and an MBH style reconstruction becomes nothing other than a regression in a period of length 79 against 22 more or less “naturally orthogonal” proxies and, of course, you get overfitting. The Wahl and Ammann effort to “fix” MBH is a particularly amusing example of overfitting as they do a PLS regression in a period of length 79 against 90 proxies.

With apologies to Cohn and Lins, is Nature’s style “naturally orthogonal”? If so, there are many traps still to be sprung on the unwary dendroclimatologist.

Wilson, R.J.S. and Luckman, B.H. 2003. Dendroclimatic Recons truction of Maximum Summer Temperatures from Upper Tree-Line Sites in Interior British Columbia. The Holocene. 13(6): 853-863. http://www.geos.ed.ac.uk/homes/rwilson6/Publications/WilsonandLuckman2003.pdf


26 Comments

  1. Posted Apr 4, 2007 at 7:29 PM | Permalink

    Steve, I really enjoyed this post.

    For added entertainment, I forced some of my less statistically inclined friends to first read out loud the meanings of the word “orthogonal” (found here) and then made them re-read

    However, I will venture to say that “naturally orthogonal” are not a good thing for reconstructions.

    ;-)

    Thank you for an insightful post.

    Sinan

  2. Steve McIntyre
    Posted Apr 4, 2007 at 9:49 PM | Permalink

    Sinan, you might be interested in re-reading this post on overfitting in Ammann and Wahl, where I make some very pretty reconstructions using Ammannian overfitting.

    Another post, which was one of my favorites, but you have to scroll down to the middle portion to get to the interesting part is this one http://www.climateaudit.org/?p=370 where I make Mannian reconstructions from stock prices plus white noise, stock prices plus non-bristlecone proxies. At a further remove, the theoretical point that’s underlying these posts – that would be nice to pull together some time – is what happens when you combine two forms of spurious regression: a classical univariate spurious regression plus a network of white noise or low-order red noise (“naturally orthogonal”).

  3. JMA
    Posted Apr 4, 2007 at 10:45 PM | Permalink

    The one thing which you do not metion, Steve, is that RW and MXD target different seasons. RW targets growing season (length?) and MXD targets late growing season temps. I know you are better at stats than I am (I intend to rectify that) but I do think that this may be an example of misinformation. I would also add that this paper, as any good paper should do (need to justify more funding, of course) explicitly calls for more research. Has there been any follow on research on this question? You don’t say that Rutherford and Mann seem to have done some work on the target season question, but you don’t want to talk about that, do you?

    So, are there any follow-on papers? Enquiring minds want to know,

  4. JMS
    Posted Apr 4, 2007 at 10:45 PM | Permalink

    That last post was from everyone’s favorite around here, JMS.

  5. Steve McIntyre
    Posted Apr 4, 2007 at 11:13 PM | Permalink

    #3. In this post, in the posts under discussion here, RW and MXD are not used to target different seasons.
    Did they follow up on the observation of naturally orthogonal, – jeez, they didn’t even discuss the implications, hence the post. There have been many posts on this blog about Mann and Rutherford, so why would you say that I “don’t want to talk” about their work. I am unaware of any consideration by Mann and/or Rutherford of the relationship between RW and MXD series – quite frankly, it’s not the sort of thing that either of them get into.

  6. Rob Wilson
    Posted Apr 5, 2007 at 3:01 AM | Permalink

    Hi Steve,
    JMS partly hits it on the head.
    For the current year, the RW and MXD respond to different monthly parameters. Look at Figure 2.
    RW – June, July and September
    MXD – March-June and August.
    June is the only month of response ‘overlap’

    The MXD response to August is clearly the strongest response and agrees basically with similar observation using MXD from all over the NH.

    The reconstructed season (May-August) is a compromise season reflecting the common response of both combined parameters. The correlations of the MXD and RW chronologies with May-August temperatures are 0.64 and 0.39 respectively.

    The non correlation between RW and MXD is/was, I admit, a surprise. However, from a multiple regression point of view, the non correlation reduces multicollinear problems which helps reduce variance inflation. If you look at Briffa’s NH work, the relationship between RW and MXD is quite variable between sites and presumably reflects the differing response windows between the different parameters – i.e. in some cases there will be a greater response window ‘overlap’.

    Back to the BC example – with a common response to at least one month, one would expect some coherence between the two time series. NB. however that there is actually little coherence between the summer months themselves – the strongest correlation is between July and August at 0.175.

    For the IBC reconstruction, over the 1600-1997 period, the correlation between the RW and MXD composite chronologies (not orthogonal PC scores) is 0.097 (p = 0.053 – not adjusted for autocorrelation). If I convert the series to 1st differences, the correlation improves to 0.20 (p 1.6), and resulted in vastly superior verification results. My aim was to develop the best possible reconstruction and therefore I included the RW data.

    At the moment, the definitive reconstruction for the region would be Luckman and Wilson (2005) although the reconstruction compares very well with Wilson and Luckman (2003). In the newer study, the RW data entered twice into the regression model at lags T and T+1 – the latter being inversely weighted to account for the inverse correlations with previous years August temperatures. The entering of RW twice, but with opposite signs effectively cancels out the lower frequency signal in the RW data, and therefore only strengthens the high frequency signal in the final reconstruction. This therefore ‘empirically’ gets around the issue of the non temperature controlled low frequency signal in the RW data.

    I have probably given you all lots of things to discuss and criticise. I am not too worried. If I say so myself, and of course I am biased, I think both Wilson and Luckman (2003) and Luckman and Wilson (2005) present fairly robust reconstructions. Certainly in the newer study, the longer term trends compare very well with the glacial record and multi-decadal and short term cool events coincide with known periods of solar minima and volcanic events. These help independently validate the proxy temperature series.

    To end – hopefully, I have addressed the issue of ‘natural orthogonality’ between the RW and MXD. It is partly related to different seasonal response, but also reflects the fact that the lower frequency signal in the RW data is not entirely controlled by temperature. This latter observation is not ideal – I admit – but ongoing work is trying to study this observation.

    I am away for the next few days, so will probably not be able to respond. Therefore, it might not be wise to pose direct questions to me as I will likely not be able to answer them.

    Happy Easter
    Rob

  7. DougM
    Posted Apr 5, 2007 at 5:22 AM | Permalink

    Weather is chaotic in a period of days. Therefore RW and MXD are orthoganal.

  8. John S
    Posted Apr 5, 2007 at 5:57 AM | Permalink

    Certainly there is something to think about here.

    Now my thinking is that, if there is a regular seasonal pattern, then there should still be some correlation between the two if they are useful for extracting a low frequency temperature trend. If the seasonal pattern were perfectly regular then the only difference between the two series should be a constant (or a log constant if its multiplicative). Now I’m sure there isn’t a perfectly regular seasonal pattern, but I think the implication of them being uncorrelated is still uncomfortable for the concept of some global temperature trend. (Particularly if that trend is non-stationary as instrumental observations are.)

    I’m just going on statistical intuition here as I haven’t the stamina to do matrix algebra at this time. But I’m sure there are others who have the stamina and could correct me if I’m wrong.

  9. Steve McIntyre
    Posted Apr 5, 2007 at 8:09 AM | Permalink

    the lower frequency signal in the RW data is not entirely controlled by temperature. This latter observation is not ideal – I admit – but ongoing work is trying to study this observation.

    It’s certainly not “ideal” for reconstructions relying on RW chronologies.

  10. bender
    Posted Apr 5, 2007 at 8:35 AM | Permalink

    Wilson’s argument is correct IF the assumed responses are real. However …

    1. the June (current year) ‘overlap’ in Fig 2 is not insignificant. Neither is the Aug (previous year) overlap. So there’s some non-orthogonality in the response, despite the orthogonality in the RW and MXD time-series.

    2. response function analysis is over-rated. It is exploratory & correlative, not definitive & causal. These monthly “responses” are often weak and have little or no biological basis. Interpretations are all post-hoc. For example, how does one interpret a positive December RW response? Tenuously, that’s how. Especially at r = 0.2. Conduct 20 correlation tests and 1 in 20 is bound to be significant at the 95% confidence level by random chance alone. When has any response function analysis ever corrected for this?

    Criticism #2 was in fact posited on another thread recently (was it James Lang or Paul Linsay?). I noticed no dendroclimatologists bit.

    Yeah, I know. Here comes the dendro backpedal:
    “We admit response function analysis is exploratory. What’s wrong with that? Science is fundamentally exploratory.”
    “We know the correlations aren’t proof of causalit. We’re not stupid.”
    “We realize all these tests have weaknesses. That’s why we use multiple tests.”
    “The science is evolving. We’re working on it.”
    “If we had more money for more samples the methods would eventually work themselves out.”

    Dendros: it’s time to grow up. Playtime is over. Business-as-usual is not an option. You need a serious “tree-ring statistics” working group. No “buts”. Just do it.

  11. UC
    Posted Apr 5, 2007 at 9:19 AM | Permalink

    pathological science is by no means a thing in the past
    (the science of things that aren’t so)

  12. bender
    Posted Apr 5, 2007 at 9:37 AM | Permalink

    Statistics (e.g. a correlation coefficient) can be used in two modes: descriptively (applied to samples) or inferentially (applied to sampled populations). The problem is that many dendros don’t realize this – that *constructing* a working hypothesis based on samples is not the same thing as *testing* a working hypothesis about populations. I think some dendros get comfortable using statistics as descriptive tools, and don’t realize the higher standard of evidence that is required for strong inference. Contrary to the earlier claim that dendroclimatology is one of the few fields where validation is done, I think it is one of the fields where real validation is most deperately needed.

  13. bender
    Posted Apr 5, 2007 at 9:42 AM | Permalink

    Axes in Fig 3 in opening post are not labelled. log(MXD) vs log(RW)?

  14. bender
    Posted Apr 5, 2007 at 9:49 AM | Permalink

    At the risk of sounding like the old TCO …
    1. Fig 2b in opening post is inadequate to judge joint normality. You need a 3d histogram, or an overlay of pinprick-sized dots, to see if the smudge of points is 3d bell-shaped.
    2. Non-normality is a minor sidebar issue compared to orthogonality. Non-normality is one of the least important assumptions in regression, and one of the easiest to solve by transformation.

  15. Steve McIntyre
    Posted Apr 5, 2007 at 9:57 AM | Permalink

    #14 bender, you’re probably right about the sidebar-ness. Sometimes I’m thinking out loud and want to see data from different perspectives. Here that’s probably a distraction – and maybe I’ll just link to the RW-MXD distributions and remove the graphics from the post so that orthogonality stays front and center.

    “from a multiple regression point of view, the non correlation reduces multicollinear problems which helps reduce variance inflation”

    I’d better go have a coffee or maybe a beer before I comment on this.

  16. Otto
    Posted Apr 5, 2007 at 10:04 AM | Permalink

    Steve:
    Is it your position that all temperature reconstructions from ring-widths are useless?
    Do you feel the same way about precipitation reconstructions from ring-widths as you do about temperature reconstructions?

    Bender:

    When has any response function analysis ever corrected for this?

    Not sure if this is what you mean, but Biondi and Waikul describe bootstrapped response functions for dendro(Computers and Geosciences 2004)

  17. Steve McIntyre
    Posted Apr 5, 2007 at 11:55 AM | Permalink

    Is it your position that all temperature reconstructions from ring-widths are useless?

    I try to avoid generalizing and “useless” is not a word that I usually use. I try to discuss particular studies as much as possible. The tree ring information is a vast and interesting data set. I think that one can be critical of existing statistical practices without saying that all the information is useless. I thought that Naurzbaev et al 2004 and Miller et al 2006 were interesting and novel applications of tree ring data to temperature reconstruction. We brought these studies to the attention of the NAS panel, which, in turn, cited them both favorably.

    I have a much better impression of the precipitation reconstructions than the temperature reconstructions, but I haven’t studied them in detail. Cook, Woodhosue et al, a precipitation reconstruction, had good verification r2 statistics as well as good RE statistics – so that gives me a much better impression.

  18. Steve Sadlov
    Posted Apr 5, 2007 at 11:57 AM | Permalink

    I recommend:
    1) Cutting out arid and semi arid sites.
    2) Emphasizing sites in Marine West Coast climates
    3) All the other caveats highlighted in IPCC 2006

  19. jae
    Posted Apr 5, 2007 at 12:53 PM | Permalink

    I can see how RW and MXD might be orthogonal. When times are good for the tree–adequate moisture and temperature–RW should be large and MXD should be moderate (large cells in both earlywood and latewood). When it is cold, RW should be small and MXD should be low (not enough photosynthate to make dense LW cells (?)). When it is hot (or drought), RW will be small and MXD should be high (fewer and smaller earlywood cells, fewer latewood cells but they will be flatter (narrower) and more dense). ref.

  20. Willis Eschenbach
    Posted Apr 5, 2007 at 6:09 PM | Permalink

    jae, I don’t understand your idea about orthogonality. The matrix you propose looks like this:

    Condition____RW_____MXD
    Hot_________small___high
    Cold________small___low
    Good________large___moderate

    Since the assumption is that these are treeline sites, temperature limited, the two most common conditions would be Cold and Goo. I can’t see how that is orthogonal … which may be my own lack of understanding.

    I also didn’t understand Rob’s post. He says it makes sense that they are orthogonal, since the times of significant correlation only overlap in June. He also says that he was surprised that they were orthogonal. But the correlation over the entire summer (May to August) is what they are looking at, and I assume that there is a positive correlation (although not significant) in the other months for both variables. I say this because Rob says:

    The correlations of the MXD and RW chronologies with May-August temperatures are 0.64 and 0.39 respectively.

    which is higher than the correlation with the individual months.

    w.

  21. Steve McIntyre
    Posted Apr 6, 2007 at 9:04 AM | Permalink

    I’d like to refresh an earlier comment by Rob Wilson on MXD and RW in this part of the world here.. Rob said:

    The data that Jan Esper used are ONLY ring-width. Again, as I have told you in earlier e-mails, maximum density and ring-width data are doing quite different things in this region of the world. However, they compliment each other very nicely when combined using multiple regression to develop a reconstruction. More discussion on this is detailed in an earlier paper:

    http://freespace.virgin.net/rob.dendro/pdf/Wilson%20and%20Luckman%202003.pdf

    I didn’t pick up on the fact that Rob’s citation here stated that the RW and MXD series were “naturally orthogonal” in this article, so it’s pretty obvious in retrospect that the fit is going to be “improved” merely because of the orthogonality of the RW and MXD series, regardless of anything else. In the post in question , I presented the following graphic showing differences between the Esper Alberta data, which I had only just obtained and the Luckman-Wilson reconstruction, which is shown again below (I observed at the time that the differences between the two versions at merely one site seemed inconsistent with Mann’s confident claim of knowing past temperature within 0.2 deg C):

    When we re-visit this particular graphic, here’s another take on this in light of subsequent experiments: the greater low-frequency content of the Esper reconstruction here is undoubtedly due in considerable measure to the fact that he “ONLY” used RW data. Rob’s OLS regression onto two orthogonal series (RW and MXD) surely has the impact of attenuating low-frequency content. It’s hard to say how much attenuation takes place. These issues are quite independent of whether RW and MXD are measuring different months and are mathematical in nature.

  22. jae
    Posted Apr 6, 2007 at 11:02 AM | Permalink

    20, Bender: My thought (really a question, I guess) is that most of the sites they use are not always temperature limited. They are temperature limited during some years/months, but not other years/months, so that the trees express multiple anatomical responses from year to year. You need to look at more than just ring width or latewood density. I would try to combine ring width, ring density, earlywood density and latewood density, if there were some way to do so.

  23. Mark T.
    Posted Apr 6, 2007 at 2:35 PM | Permalink

    This is the issue with mixing matrix/source stationarity, jae. Component analysis assumes that the mixing, or sources, are consistent over the period of estimation.

    Mark

  24. jae
    Posted Apr 6, 2007 at 5:59 PM | Permalink

    23: OK. As uttered by a famous person, “I am not a statistician” (but I also don’t pretend to be. But I can’t figure out what could be stationary about tree growth.

  25. Paul
    Posted Apr 7, 2007 at 11:16 AM | Permalink

    #21

    That graphic says it all for me. The variation between those time series is greater than variation within them. For me it just makes it all noise.

  26. Steve McIntyre
    Posted Apr 7, 2007 at 11:36 AM | Permalink

    #25. I think that you’re jumping to conclusions. What it shows is that two people who like trees but not math can produce different answers from noisy data. IT doesn’t prove that there is no usable information there.

Follow

Get every new post delivered to your Inbox.

Join 3,415 other followers

%d bloggers like this: