CA reader hfl, who cited Buell’s documentation of the dependence of principal component patterns on shapes, has sent me a scanned pdf version now available here. It concludes by observing that analyses that fail to consider this phenomenon (and there is ample evidence that Steig et al falls into this category) “may well be scientific level with the observations of children who see castles in the clouds”. [Update: Here is a rendering by a CA reader (now using Lucy Skywalker hockeystick version)]
Here is hfl’s summary:
Forgive me if this has been discussed in past threads on PCA (although I couldn’t find it in a quick search of the site), but it’s worth noting that the issue of principal component pattern dependence on domain shape is (or was) well known within the atmospheric sciences community. It was first documented by C. Eugene Buell in a 1975 paper published in the Proceedings of the Fourth Conference on Probability and Statistics in Atmospheric Sciences. Buell’s work focused on square/rectangular domains, particularly because the latter approximated the shape of the conterminous U.S. This was followed by a second paper at the Sixth P&S Conference in 1979 which states: “When a region with well defined boundaries is concerned, the EOF’s computed over this region are expected to be very strongly influenced by the geometrical shape of the region and to a large extent independent of where the region is located. As a consequence, the interpretation of the topography of the EOF’s in terms of geographical area and associated meteorological phenomena should be looked on with suspicion unless the influence of the effect of the shape of the region has been completely accounted for. Otherwise, such interpretations may well be on a scientific level with the observations of children who see castles in the clouds.”
Buell’s work generated considerable discussion within the atmospheric sciences literature because PCA (or, as they referred to it, EOF [empirical orthogonal function]) analysis was in widespread use. Mike Richman, in a paper published in the Journal of Climatology (1986) entitled “Rotation of Principal Components” made the case that component rotation appears to eliminate the problem of domain shape dependence. Richman’s paper is worth reading in that it reviews and considers nearly all of the important work that had been done using PCA on meteorological and climatological data up to that time, including work by Gerry North and others familiar to CA readers. So these problems have been well documented and understood for a long time. Like so many other elements of meteorology and climatology, though, modern climate science appears to have forgotten some important statistical insights produced by it’s own practitioners . . . indeed, some of the practitioners themselves appear to have forgotten what they wrote.
Climate Audit 
51 Comments
With all due respect, I find the “castles in the clouds” statement somewhat funnier than “done on the back of an envelope.”
It’s interesting to see older papers like this with hand-drawn graphs compared to modern equivalents with computer-generated plots which are simultaneously more accurate (not to imply scientific accuracy or fitness) and aesthetically pleasing. Science sure has changed.
The doesn’t contain anything as poetic as “castles in the clouds”, but here’s the full paper on the problem as recognized in population genetics. Instead it uses phrases like “mathematical artifacts” and “providing a’null’ expectation against which observed PCs may be compared and contrasted”.
Click to access NovembreStephens2008.pdf
Re: MichaelJ (#2),
nice reference. I think someone might have mentioned this paper in a prior thread last year. It looks a bit familiar.
Re: Steve McIntyre (#4), Steve, I posted this last year, as well. I figured it was worth are re-post if for no other reason than the irony of appearing in a Nature publication.
Sorry, a personal note/funny coincidence. My brother coauthored papers with the those population genetic researches.
and I definitely need to proof my posts.
Re: jeez (#5),
I wonder if someone could make a nice version of the CA masthead with a castle among the clouds? Perhaps with a signboard reading “Team RC” and a BBQ with a weather station nearby.
Re: Dave Dardinger (#6),
Be careful what you wish for, Dave 😉
Didn’t have a BBQ model, but I made a crude MMTS, and placed in the best possible location for optimal temperature measurements.
Re: Jeff Alberts (#10),
The RC flag is a nice touch. 😀
Re: Jeff Alberts (#10),
Thank you! I suppose you could have some smoke rising near the MMTS and leave the BBQ to our imagination. Unfortunately there don’t seem to be any windows nearby to have an air conditioner in. The arm on the flag does have a nice hockey stick shape, though, I notice.
If this shape dependency holds, how is it accounted for in the determination of the AAO which is part of Steig et al’s finding? see link and associated method. Surely this must be accounted for in the generation of the AAO?
MichaelJ, nice paper. I particularly enjoyed this part (emphasis mine):
Reference 17 is Richman, M.B. Rotation of principal components. J. Climatol. 6, 293–335 (1986).
I can’t find a copy online, but the abstract says:
So the problem has been recognized for over twenty years … but in the meantime, the Team has moved on. Kinda reminds me of the Steve Martin movie where he’s in a restaurant and says something like “Hey, you can’t fool me. This wine is twenty years old! Bring me some of the new stuff!”
w.
Re: Willis Eschenbach (#8),
Hey, that’s twenty year old research. That’s obsolete, nobody needs those numbers anymore.
Incidentally, this has interesting back-of-the-envelope hand drawn graphs: thisisindexed.com
Re: Willis Eschenbach (#8), Willis:
It is beyond me, but you may be able to tell whether the current method for measuring the AAO takes the issues associated with PCA into consideration.
Is it just me? There really is a castle in those clouds…
Gotta stop spending so much time with the kids. Dang this economy.
No slight to hfl in the OP, but Richman (1986) has been mentioned before – in the context of Wilson’s dendro work:
http://www.climateaudit.org/?p=1285
It is also cited in the paper mentioned by MichaelJ in #12 and it contains the start of the answer to bernie’s #12. By finding a “suitable” rotation you emphasize “simple structure” and your eigenvectors move away from being a parody parade of harmonics. That’s the theory. Read it.
Re: bender (#16), Nice tip. Personally, and rightly or wrongly, I have always used a rotation to make sense of a factor. With survey data it is very difficult to interpret the factors any other way because the definition of the factors has to somehow reflect the questions that were asked. (We are plagued however by our own forms of autocorrelation such as leniency or halo).
Wow, I made the OP, lol.
FYI, to see the pic in full size, in Firefox, right-click the image and select View Image, then you can zoom in and see more detail.
Re: Jeff Alberts (#17),
Without using the RegEM filter in Photoshop? 😉
Re: bernie (#33),
The world is a complex place and climate factors have been known to affect different geographical cofigurations (high altitude-low altitude, continental coast-interior) in different ways. Rotation of axes can, in some cases emphsize those relationships.
Look at a simple (non-climate) example. Suppose you graph a bunch of peoples’ heights versus their weights. Assume that you scaled them in such a way that most of the subjects fall close to a 45 degree line in the graph going through the origin. Now rotate the coordinates 45 degrees retaining orthogonality of the axes. It is not inconceivable that the new axes represent somewhat different factors than just height and weight. One could label one axis as “size” which differentiates larger and smaller persons, and the other might be viewed as “body shape” (e.g. taller thinner persons or heavier and shorter persons) in the population from which you took the subjects.
It is not inconceivable that similar considerations could apply here. However, without reasonable statistical considerations AND demonstrated physical (non-teleconnective) causes, it would be no more than arm-waving and pointing at the castles in the skies.
As a side note. when I was teaching, I used to call it “horsies in the clouds”. I suppose that my students weren’t as mature as as the crowd inhabiting CA…
So will Prof. Steig reveal his data and confirm or collapse your conjectures, or will he withdraw the paper and conceal his castle in clouds forever? Decisions. Decisions.
==============================================================
Castle in the Air with Hockey Stick
Re: Lucy Skywalker (#20), Lucy I see you forgot to cut off the divergence data at the end. 😉
Re: John F. Pittman (#21),
Worse, Castle RC is now converted into the MWP and is higher than the modern warming period.
Re: Lucy Skywalker (#20),
LOL! Nice warp! Let’s see your data and methods!!
Re: Jeff Alberts (#25), Female privilege to decline…
Re: Lucy Skywalker (#20),
So the bit with the flag is the MWP?
Jolliffe (1987):
Bold mine.
Re: bender (#22),
Buell (1975):
Bold mine.
Re: bender (#23),
Richman (1987):
Bold mine.
And this is for Dr. Steig:
Richman (1987):
This is one of Steve M’s main contentions.
Dr. Jolliffe,
May I invite Drs. Steig & Mann & Schmidt to spend a month in your lab this summer?
Thank you for entertaining this request.
bender
Re: bender (#27), Would that be an unpaid internship?
Reflecting a little on Buell and Richman, I think that there are some important aspects of the relationship that have been reported here than are not present in the earlier literature.
I haven’t parsed the articles in detail and do not exclude the possibility that I’ve missed something relevant, but in a first read, I don’t see that either article identified the form of the patterns as clearly as we’ve pinned down here. We’ve put a functional form on the eigenfunctions, relating them to Chladni patterns (and other homomorphic structures). I didn’t see that in the earlier literature.
Given the present (more informed??) interpretation of these patterns as Chladni-type patterns, I’m not at all sure that rotation of principal components is an omnibus cure, so the specific recipes in Richman would need to be parsed. I would place more weight on the recognition of the problem, than the prescription.
bender, as to the bolded item in Jolliffe’s 1987 comment, I doubt that Jolliffe would preclude the possibility of domain structure being dominant in some circumstances or that he would disassociate himself from the statistical requirement that the authors show that their eigenvectors are not mere castles in the clouds (as in the nice Richman quote that you made on this topic.)
Steve:
If the pattern of the non-rotated PCs are essentially associated with the spatial autocorrelation, in what ways would a rotation actually reveal some underlying structure reflective of physical processes as opposed to spatial relationship. Shouldn’t one or more of the rotated factors actually reflect the spatial relationships? For example, if all that was there was the spatial relationship then they should appear as the rotated factors? But I am treading at the edge of my current ability in this arena – which may be obvious by my questions.
RomanM:
That’s how I recall it. I guess the question I have is the extent to which the AAO is a statistical artifact. Given the horsepower available, it is hard for me to imagine that those responsible are not aware of these spatial correlation issues.
Re: bernie (#35),
Of course they are aware of it. The question is why, given the known issues, do they make claims that they don’t bother to prove. And I think the answer is simply Nature’s ultrastrict word limit. Nature has no business fast-tracking speculative climatology papers. Save that for the specialist climatology literature.
.
The story here is not so much what Steig et al did inside the paper. It’s what the Nature reviewers didn’t do, and, even more, what the Nature media people did to spin the story. Steig et al. went along for the ride. And why not?
Re: bender (#36), “And why not?”
Absent scientific integrity.
A CA reader writes offline:
Steve, sorry to post this here, but didn’t have an email address for you.
I rendered up a new header graphic for the site, if you care to use it.
Re: Jeff Alberts (#38), cute, but that’s making a little more of a statement than I want to make. But I like the idea – the present header’s been fine but, after a while, it’s a little nondescript.
Re: Steve McIntyre (#39), Well, I wasn’t really reading much into it, just making something visually appealing and at least partially relevant. 😉
Re: Jeff Alberts (#38), (and for anyone interested in header graphics) — it’s best when the header can seamlessly repeat (one way or another). People have different screen widths; it’s nice when the header “works” for everyone.
Re: MrPete (#45),
I agree, MrPete. However, I was using Steve’s current header graphic as a template for width and height (copied the image and used the raw size as opposed to the visible size in the browser). The width is greater than 1600 pixels, so while you may have a screen res wider than that, is it likely that you’d have your browser taking up the entire width? A lot of sites also have fixed width content areas with fixed width headers, so it’s not a given by any means. Anyway, it was a quick and dirty…
I did a couple of videos of the satellite temperature trend because I wanted to see how PC3 changed the temp anomaly. It is visible in the anomaly video, you can see the oscillations back and forth along PC2 and up and down from PC1 and a sudden left-right motion in PC3. I then did a simple enhancement to make it more visible, what I didn’t expect to find was that the actual satellite data from Steig’s conclusion has a temperature reversal from the eastern rim temps to the center of the antarctic.
Unless I’m missing something, you will probably be surprised.
http://noconsensus.wordpress.com/2009/02/28/a-little-bit-of-magic/
Re: Jeff Id (#41),
Nice job, Jeff! That’s a great way to show 3,305,400 numbers and see what they tell you.
You could become a statistician if you work at it. 😉
Re: RomanM (#42),
Quite a compliment, another method to bore my friends to death. 🙂
When I see a semicircle of temps different from the interior it bothers me a bit. The ocean certainly would dampen trends, I’ve lived on both sides of lake Michigan for years at a time. On the west, there’s not usually much effect but on the east wow! It mau be possible with circular weather patterns that the edge of the antarctic would all have similar temps, I’m not sure. I’m guessing the muted temparature half the way around the antarctic could actually exist in the satellite data overlaying the surface stations. This could cause regem to weight the interior with a heavier or even inverted trend from what the surface stations show (e.g. if the trend in the interior is stronger in the sat data, trends at the coast will be amplified in the interior. Since these surfacestations by my above assumptions are therefore in a high gradient area, small adjustments of even things like the 50Km gridsize or .5 degrees of latitude could make a real difference in result.
Either way, it’s interesting to see the PC’s at work. I think the reason Steig chose not to describe the physical meaning of the 3rd PC may be because it doesn’t exist pre-1982. Without the sat data it’s just speculation of course.
As requested, Steve, versions
Cloud Castle 01
Cloud Castle 02
Cloud Castle 03
Cloud Castle 04
Hope it’s what you want.
Re: Lucy Skywalker (#46),
Just a suggestion. Clouds that look like Lord Stanley’s Cup. People holding the cup in the air seem to be on cloud 9.
Re: Lucy Skywalker (#46),
Can you add Gavin pouring boiling oil over the parapet onto some peasants ?
Alan Stern, NASA’s “hard-charging” and “reform-minded” Associate Administrator for the Science Mission Directorate, resigned on March 25, 2008,[44] to be effective April 11, after he ordered funding cuts to the Mars rovers and Mars Odyssey that were overturned by NASA Administrator Michael D. Griffin. The cuts were intended to offset cost overruns for the Mars Science Laboratory. Stern, who served for nearly a year and has been credited with making “significant changes that have helped restore the importance of science in NASA’s mission.”, says he left to avoid cutting healthy programs and basic research in favor of politically sensitive projects. Griffin favors cutting “less popular parts” of the budget, including basic research, and Stern’s refusal to do so led to his resignation.
In response to the internal review, policies at NASA would be changed in a variety of ways: Flight surgeons would be present during the pre-mission suit-up activities, flight surgeons would receive additional training in psychiatric evaluation, and although there was an unofficial code of conduct in place, an official “Code of Conduct” would be written up for employees NASA