Calibration RE
The NAS Panel notes the following about several statistics used in proxy studies:
If 
are the predictions from a linear regression of 
on the proxies, and the period of interest is the calibration period, then RE, CE, and 
are all equal.
Here’s a result about MBH methods (and applicable to related methods with re-scaling) that has not been reported:
If are the predictions from an estimate of in which has been re-scaled so that its standard deviation matches the standard deviation of the target, and the period of interest is the calibration period, then
(1) CE=RE = 2*r-1.
I can prove this (the proof is fairly trivial but you have to think a little or else some one else would have observed it already) and have verified it against actual results. Notice that the relation is with 
directly and not . I thought of this when I saw a comment of Bürger and Cubasch about correlations, in which they mentioned that correlation and r2 were inappropriate when scale was involved. It’s not like I’m some advocate of one statistic over another, so much as someone who says that you should look at all aspects. In this case, there’s a reason why correlation links directly to explained variance. OK, Gerd, do you agree? This would be a good puzzle for Rob Wilson (who uses re-scaling procedures) to think about as well.
Climate Audit 
10 Comments
I have some fundamental confusion of the correlation coefficient itself and of the rsq. Need to go look in a basic book.
Ok, this is embarresing. I googled and Pearson’s r is just a value of -1 to 1 that gives some idea of how well the two variables vary together. It’s different from k, the slope of the line. And what do we call that? Hmmm…anyway. Here is a place that tells you what rsq is significant.
http://www.swrp.org/Student_Presentations/Preparation/Poster/Statistics/critical_values_table.htm
The most telling part of that equation for me, in the climate-politics sense, is that it shows if one is calculating ‘r’ one is also calculating ‘CE’ and ‘RE,’ because they all directly interconvert. That means Mann’s comment in favor of using RE that, ‘calculating r^2 would be silly and incorrect,’ is nonsense. Calculating one is calculating all the others. Maybe that’s why your advice about using the Durban-Watson statistic is so useful, Steve, and was recommended by the NAS panel. Perhaps it provides an truly independent measure of statistical skill.
Admit something: You’re having some of the most fun of your life, doing all this. 🙂
Pat, I’ve always said that I was doing this for fun. It’s nice that you understand this. If you look at the etymology of “amateur”, having fun is part of it. It’s funny that all the climate scientists that get paid to provide services assume that I’m doing this for some payoff. (Leaving out the obvious simile), it never occurs to them people would do it for fun.
The link between r and RE is exact only in the calibration period. As I mentioned before, there’s a very simple and pretty explanation for this phenomenon, which will show the limited mathematical understanding of the Team. The proof is the type of thing that I was good at when I was 17 or 18. I finally found my old Linear Algebra text: Greub, Linear Algebra (Greub taught our 3rd year course.) It’s a pretty tough-looking text for those who wonder what math courses were like at University of Toronto in the late 1960s.
When I see the fancy stuff that my classmates Scherk and Bierstone do, it’s awfully trivial, but it’s like I’m a one-eyed man on the hockey rink.
The relationship doesn’t hold necessarily out of the verification period, but it’s worth understanding why it changes, and I’m still mulling that over.
#4, “…but it’s like I’m a one-eyed man on the hockey rink.”
Rather, that’s the one-eyed man on the hockey rink of the blind.
Regarding fun, many years ago I wrote a letter to Chemical&Engineering News to flog a prominent chemist who had written that scientists were a “tribe” who were motivated by “Nobel lust” and other such egotistical stuff. After my reply appeared, I got a hand-written note from a retired and well-known chemist named Stan Tarbell. He agreed with me and added that scientists work for the pleasure of it. He was right, so far as I’ve experienced it. So, your reaction seemed entirely predictable.
I’d warrant you’ve made it much less fun for Mann and the HT, though. Something about tangled webs comes to mind here.
To join in the discussion of scientific motivation- I agree with Pat- I have so much fun as an industrial scientist that I honestly feel inclined to conceal the fact, knowing that some of the more dour and puritanical bean-counting types would not like it. I don’t have all the freedom an academic does, but I find that working under industrial constraints has a certain charm. It becomes another puzzle, how to find something both interesting and practical to work on. Going from laboratory to product is gratifying- I appreciate the pay, and couldn’t do this without it, but I try hard to retain that part of the amateur spirit that makes the work fun.
I make no pretense of understanding all the stats or climate science discussed here. It has motivated me to learn- I have dusted off my own linear algebra texts, and worked hard to figure out the principle components method under so much discussion here. I appreciate the guidance of people here when I ask a particularly stupid question.
Still, I am not more than rank amateur- the really technical discussions are like when I watch sitcoms on Univision to keep my Spanish current- I recongnize enough to get some of the jokes, and follow the plot, and I reach for my dictionary or verb conjugator book for the rest.
But the truth is that the disputation is what I really come for. It’s fascinating. It has been especially great to see things progress to the point that M and M weighed in at the NAS panel, far beyond the inter-blog pissing contest much of the ‘debate’ descends to. Great stuff. I am a little surprised that the results have not be more accurately reported in the press. I am more dismayed but less surprised at how some scientists have handled it. I’d wager that the effect of M and M’s work will continue to make an impact. And that the snark and fighting will continue, too. Cool.
Apologies if this has been discussed already (refer me to thread, if so), but how consistent are the findings above with this June’06 publication from the:
Committee on Surface Temperature Reconstructions for the Last 2,000 Years? Gerald R. North (Chair), Texas A&M University, College Station; Franco Biondi, University of Nevada, Reno; Peter Bloomfield, North Carolina State University, Raleigh; John R. Christy, University of Alabama, Huntsville; Kurt M. Cuffey, University of California, Berkeley; Robert E. Dickinson, Georgia Institute of Technology, Atlanta; Ellen R.M. Druffel, University of California, Irvine; Douglas Nychka, National Center for Atmospheric Research, Boulder, Colorado; Bette Otto-Bliesner, National Center for Atmospheric Research, Boulder, Colorado; Neil Roberts, University of Plymouth, United Kingdom; Karl K. Turekian, Yale University, New Haven, Connecticut; John M. Wallace, University of Washington, Seattle; Ian Kraucunas (Study Director), National Research Council’s Board on Atmospheric Sciences and Climate.
http://darwin.nap.edu/books/0309102251/html/79.html
9 Statistical Background
RE, CE, and R2 analysis value is layed out in OCRs for pages 88-92, but especially in “OCR for page 90” and “OCR for page 91” found at the bottom of the page. Tough to read, I had to copy & paste into MS-Word to begin to wade through it.
It appears they are proposing guidelines to limit RE, CE, and R2 impact in publishing?
Re #7, McCall
That publication is the report from the NAS Panel Steve refers to in his post. There are lots of threads here discussing it, and a category “NAS Panel” in the right-hand sidebar.
I agree with you about that scanned version of the report – you can also get a pdf of it, as described here.
Re #4 Steve for completeness in your relationship between RE and r, shouldn’t you include that y and the proxies need zero mean in the calibration period in order for CE=RE = 2*r-1 to hold. During the verification period this fact doesn’t necessarily hold and the relationship between CE and r won’t hold either.
re: 8 — Thanks. Should have caught that. With the format differences and no “OCR” hit* in the .pdf search, I thought it was yet another supporting document with conflicting positions.
* Missed the obvious meaning, duh — thought OCR was climate related.