Anthony Watts draws attention to a new Nature article (LI et al 2011) purporting to reconstruct El Nino activity.
The Supplementary Information shows a very obvious Chladni pattern, that went unnoticed by the Nature reviewers.
The eigenvector shown here is what one would expect from principal components carried out on spatially autocorrelated data on a geometric region of the shape shown (which equate to Chladni patterns.)
The weights of the first eigenvector are somewhat over-concentrated in the center and under-weighted on the periphery. A simple average is more meaningful.
Some CA readers may recall that a featured Nature article in 2009 (Steig et al) also misinterpreted Chladni patterns. See here for a discussion. Also see https://climateaudit.org/tag/chladni/ for other discussions.
We reported this in the initial submission of O’Donnell et al 2010. Eric Steig as “Reviewer A” required that we remove the observation that Steig’s supposedly “significant” eigenvectors were merely Chladni patterns – a requirement that we reluctantly acquiesced in, as neither reviewer Steig nor Journal of Climate editor Broccoli disclosed Steig’s adverse interest in the review.
(As I’ve previously observed – and my view here is not necessarily the same as Ryan’s-, I think that responding to Steig’s reviews diminished the quality of the eventual publication, with the loss of the Chladni section being an important loss.)
As “Reviewer A”, Steig argued that Chladni patterns were already well known to practitioners in the field – obviously not well enough known to avoid practitioners and reviewers from falling into the same error in two recent Nature articles.
Climate Audit 
23 Comments
This sort of errors may be generalized by the statement that those authors and editors generally don’t want to quantify the error margins of their measurements.
One may get these patterns, with the maxima near the center etc., but if one carefully calculated whether this principal component etc. actually corresponds to something physically meaningful, he would find out that the errors are such that we can’t reliably say whether the component is actually localized at the center.
There’s just way too many emotions and visual impressions from randomly obtained patterns, instead of a rigorous calculation of the probability of well-defined statements.
Ah, Reasonableness Testing, where art thou?
Are “Nature”? Where art thou? Ya ain’t what ya used to be.
Nature’s impact factor seems to be just under 35. I guess they must be doing something right.
In operational flood-frequency analysis, the tendency to overfit spatially correlated data has been recognized since the early 1980s. At that time standard methodology switched from relying on “visual” fits using Ordinary Least Squares (OLS), which typically resulted in hard-to-explain patterns of regional variability, to Generalized Least Squares (GLS) and now Bayesian/GLS (See Reis et al., 2005, http://www.agu.org/pubs/crossref/2005/2004WR003445.shtml). Among other things, the new methods “reveal” that variation across regions can often be attributed entirely to noise.
“The Supplementary Information shows a very obvious Chladni pattern”
It’s not obvious to me. Chladni patterns are resonant modes with prescribed boundaries, which have physical significance. It’s not clear what the boundaries are here. Certainly the contours don’t follow the boundary of the marked region as a Chladni pattern for that region would.
You have it backwards Nick. In the case of a physically significant boundary, such as the edge of a plate, you know that the pattern is itself physically significant. However, when you have a physically insignificant boundary such as the artificial truncation of data, a lack of data and so forth if you do not account for that you create an effective boundary in which the PCA will create Chladni figures in the eigenvectors. The difference is these are not physically significant, they are only a by-product of artificial data boundaries.
Re: DocD (May 9 06:10),
Let me ask again – where are the boundaries? And how do these supposed Chladni patterns conform to them?
Looks to me like the boundaries have been imposed on a sort of triangle as shown in that figure as it appears the data is only for land. The artificiality of the pattern seems to be indicated by the maximum values hitting the boundary at roughly halfway along each side if the triangle the data forms over the map, with a curve inwards at each corner. But a definitive statement would require the actual data and inspection of how the edges are handled. If the data is truncated or reflected in some way then that will produce an artificial pattern. It looks very much like a zero-order mode one would expect for some arbitrary boundary condition that approximates a triangle.
In this paper:
Click to access triangle04.pdf
there are some pretty pictures of the resonant modes of an equilateral triangular plate. If you imagine the plate to be distorted as the image above you see how this Chladni pattern arises as a distortion of the zero-order mode.
let me understand this, So these Chladni pattern are analogues to the lines of high order harmonics that you see when you do a 2d feurian transform on an image. harmonics caused by the artificial edge of the image. there for artificial produces with no relevance. (but they have ‘meaning’ they mean u hit the edge of the data set!)
Nice work, Steve.
In chemistry we have these same Chladni patterns, only we call them “atomic orbitals”. Chladni patterns are 2D mappings of atomic orbitals. I don’t get where Nick Stokes is coming from with his “defined boundaries” here: these eigenvector mappings are probability mappings which finally reach zero at infinity–there are no “defined boundaries”, far from it. (Such is my understanding; perhaps I have ignorance of which I am not aware.)
I agree that the deletion of this argument from the O’Donnell et al. paper was a considerable and mournable loss. If SM could get this more into the print literature, climate science would be much better off for it.
I’ve been playing with Chladni patterns on a sphere, which are probably similar to the orbital patterns of chemistry. The sphere is interesting because it wraps back on itself with no boundaries. However, in Steig09 and Li11, they are looking at a “small” area that is practically flat, and so has relevant boundaries.
To picture the higher Chladni patterns on a sphere, imagine a round balloon full of water in zero-gravity, then tweak it various ways to make it vibrate in different modes. The first order Chladni pattern just corresponds to inflating or deflating the balloon, so that the whole surface rises or falls together.
On a circle with equally spaced reference points, the Chaldni patterns are just Fourier harmonics plus a zero frequency constant term. With unequal spacing, the first several patterns are basically harmonic, but then they get irregular, depending on the spacing.
But on a bounded violin back, Antarctica, or SW US, even the first order Chaldni pattern is constrained by the boundary. See cool pictures on Wikipedia, Chaldni Patterns.
I’ll try to do a post on the spherical patterns when I’m done with the 23 other things I’m in the middle of!
Hu,
On the sphere, the Chladni patterns generated by the wave equation are the Laplace spherical harmonics. Wiki has a lot of pictures (scroll down).
I think people are using the name “Chladni patterns” here to cover a range of closely related but not quite identical phenomena. Chladni himself was largely interested in the patterns of nodes in resonant plates, while DocD’s references are to simple geometric systems which are constrained to have nodes at the boundaries. I’m not quite sure whether Nick is referring to either of these, and in any event I don’t think the pattern displayed is strictly speaking a Chladni pattern of either kind.
That doesn’t, of course, alter the general comment that one must be extremely cautious in ascribing physical significance to the shapes of eigenvectors.
Chladni patterns are the resonant modes of a wave equation (or similar, as in plate vibrations). Since the wave energy can generally propagate, resonance requires that the energy be confined in some way. That is the role of the boundaries. There’s usually a condition there that the amplitude is zero, or it’s normal gradient is zero, etc.
That’s why this doesn’t look to me like a Chladni pattern. There is no boundary of that kind. There would be nothing to prevent energy freely propagating SW.
The role of the boundary can be clearly seen in these atomic orbital analogues. Note the fixed boundary (for the analogue only) enclosing the region.
Atomic orbitals are different, in that there is a potential well (not present in the normal wave equation) which confines the energy and plays the role of a boundary.
BTW, I can’t see why identifying Chladni patterns matters here anyway.
Nick, principal components decomposition of spatially autocorrelated data on squares or other geometric shapes gives rise to patterns that are strikingly similar to Chladni patterns. The point regarding climate applications and goes back to at least Buell who discussed “castles in the sky” – see CA post on this.
The boundary in these cases is the boundary of the region being studied.
In some cases, climate scientists misinterpret these patterns as having physical significance, as opposed to being purely mathematical – Steig et al 2009 is an example. MBH98 is another.
Of course, you, like the Team, admit nothing.
Re: Steve McIntyre (May 9 17:28),
Well, where’s the misinterpretation in Li et al 2011? As I said, I can’t see that there is a Chladni pattern here at all. But even if there were, how would that diminish their paper?
Steve: in the case of Steig et al, the authors incorrectly interpreted the Chladni patterns as having physical significance and used this incorrect interpretation to retain only a few PCs. It mattered in that case – not that you or anyone else admitted it.
I’m interested in Chladni patterns and have pointed them out from time to time and did so here.
Right now, I don’t know whether the effect on Li et al is as material as on Steig et al. There is no list of sites that they used or data archive. Without such information, it’s hard to analyse and I don’t plan to pursue this until they archive their data.
Taking the long view, this could be a fitting epitaph. But not for our host, for the people who didn’t listen and shut him, Ross and so many others out. It will all change, I have no doubt – one funeral at a time.
From reading the paper, it’s apparent that the PCA for the SW USA was performed as a secondary, redundant procedure, after first performing the PCA on data for North America. The leading PC for North America does not resemble a symmetric ‘Chladni pattern’, but shows strong unimodal loadings primarily over the SW USA. This was the PCA used for the rest of the paper. The regional domain for the PCA you criticize appears to have been purposely selected to further demonstrate unimodal regional behavior.
The leading empirical orthogonal function (EOF) of NADA, which accounts for 23.5% of the total variance, shows a distinct moisture pattern heavily loaded over southwest North America (Methods and Fig. 1a), a region strongly affected by ENSO variability15, 16. The explained variance of the first EOF for the southwest North America domain rises to 49.5% (Supplementary Fig. S3), and its principal component (PC) has a correlation of 0.99 with that for the whole of North America over the past 1,100 years. This confirms that NADA PC1 dominates variability in southwest North America.
The authors then list several procedures to establish the leading (North American) PC as ENSO-sensitive:
The fidelity of NADA PC1 in representing ENSO variability is demonstrated in five ways: (1) NADA PC1 is significantly correlated with equatorial Pacific sea surface temperatures17 (SSTs) during the instrumental period (Fig. 1c). For example, its correlation with the January–March (JFM) ENSO Niño3 index for 1870–2002 is 0.51 (P<0.001) (Supplementary Fig. S4). (2) NADA PC1 is significantly correlated with a modern coral record from Palmyra Island in the central tropical Pacific13, at r=−0.58 for the common period 1891–1994 (P<0.001) (Supplementary Fig. S5). After adjusting relict coral U/Th dates within the analytical error windows13, we found possible correlations between North American drought and tropical corals that persisted throughout the past millennium (Methods and Fig. 1d). Considering the independent nature of these proxy records, this agreement is remarkable, strongly indicative of a coherent relationship between ENSO and North American drought over the past millennium. (3) NADA PC1 is highly correlated with, but is significantly longer than, the existing ENSO reconstructions (Supplementary Table S1). Most of these reconstructions are largely, although not entirely, independent, as they only share a few common tree-ring data from southwest North America. This suggests that NADA PC1 represents reasonable estimates of ENSO variability over the past millennium. (4) Separate composite analysis of NADA for periods of large and small PC variability over the past millennium shows nearly identical patterns (Supplementary Fig. S6). This pattern stability suggests that the westerly waveguide is stable in tropical forced stationary waves, and that NADA PC1 represents the modulation of ENSO itself. (5) Separate EOF analysis shows that the first EOF of NADA is heavily loaded over southwest North America both before and after the 1976/1977 climate regime shift18, 19, and the PC is highly correlated with tropical eastern Pacific SSTs for both periods (Supplementary Fig. S7). Although there exist some differences in the EOF pattern and correlation with tropical SSTs between the two periods, these results indicate that moisture variability in southwest North America has been sensitive to ENSO variability both before and after the 1976/1977 climate regime shift. The relationship to ENSO is further confirmed by the SST correlation pattern for 1870–2002 (Fig. 1c). Therefore, teleconnections of canonical eastern Pacific ENSO (hereafter simply ENSO) are probably stable over southwest North America. On the basis of these validations, we consider that NADA PC1 represents a continuous record of interannual ENSO variability during the past millennium.
It doesn’t appear to me that the authors are recklessly interpreting some random or artificial mode as an ENSO indicator. They have been pretty thorough about establishing an ENSO connection.
The tree-ring data is archived here. It’s a gridded data set composed of many separate tree-ring records:
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/drought/NAmericanDroughtAtlas.v2/
Steve: before you uncritically assume that there is any physical meaning to eigenvectors from tree ring networks, I urge you to read prior CA posts on Chladni patterns, starting form the earliest https://climateaudit.org/tag/chladni/. In my opinion, the evidence is quite convincing in previous situations. If sites are not distributed in a spatially uniform or random pattern, the eigenvector resulting from PCA from spatially autocorrelated sites will have a different appearance than Chladni patterns on a simple geometric shape, but that doesn’t mean that the eigenvector has any physical meaning. I’ve discussed such cases in the past. Unfortunately, the issue doesn’t seem to be well understood by practitioners – otherwise, Steig et al wouldn’t have gotten confused on the matter.
I am aware of the link to which you refer and had examined it. My point about archiving is correct. The site to which you refer does not contain a list of the sites used in constructing the gridded series.
I don’t ‘uncritically assume’ anything. I quoted you text where the authors establish a physical meaning in 5 different ways. They take the steps of testing correlations of their PC with ENSO and ENSO proxies, and these turn out to be highly significant. They illustrate composite maps of their tree-ring PC anomalies, and they correspond to wet/dry conditions in the SW (SI Fig. 6).
I think this study makes appropriate and cautious use of PCA.
I didn’t provide the data link as a challenge to any ‘point about archiving’. You stated in an earlier comment that it would be difficult to perform your analysis without the data, so I pointed it out. You certainly don’t need a site list to replicate Li et al. and their PCA, which is the central topic of this post and my comment. I read their paper, and I read your post, and I disagree with your assertion that Li et al. are misinterpreting unphysical ‘Chladni patterns’. That’s all.
One Trackback
[…] Source: https://climateaudit.org/2011/05/08/nature-publishes-another-chladni-pattern/ […]