Nature on NAS

Jul 5, 2006 – 2:17 PM

Nature has two online articles pertaining to the NAS Panel – one about the NAS Panel primarily relying on the press conference and another mentioning a prospective new NAS investigation into the data access problems left untouched by the past panel. The Nature reporter asked an interesting question at the press conference. I’ve provided a partial transcript of this exchange and compared it to the article. Continue reading →

By Stephen McIntyre | Posted in NAS Panel, News and Commentary | Comments (35)

Benchmarking from VZ Pseudoproxies

Jul 4, 2006 – 2:09 PM

Von Storch et al 2004 advocated using climate models to generate pseudoproxies to test the properties of proposed multivariate methods. Hardly unreasonable. I might argue that these are long-winded ways of generating proxy series with certain kinds of temporal and spatial covariance structures, but there’s much to be said for testing methods on some standard data. Their own networks of pseudoproxies is much too “tame” to be adequately realistic, but, if you can’t understand what the tame networks do, you’ll never understand what the “wild” networks do – a lack of understanding presently being demonstrated by Wahl and Ammann and other Hockey Team supporters. Continue reading →

By Stephen McIntyre | Posted in Multivariate, RegEM | Tagged , , , , , , , , , , , , | Comments (72)

Peer Review of Bürger and Cubasch

Jul 4, 2006 – 7:24 AM

The review of referee #2 for Bürger and Cubasch’s article in Climates of the Past is posted up here . From the style, can anyone doubt that the anonymous reviewer was Mann himself? Take a read.

The reviewer makes reference to this being the "2nd attempt by the authors to publish this flawed manuscript" – it had been rejected by GRL. I presume that referee #2 must have participated in the GRL process.

I think that there are many ways that the Bürger and Cubasch article could have been strengthened and one would hope that a reviewer would do so. In terms of comments that would be useful to a writer, TCO’s comments posted up here would have been far more useful than those of referee #2.

Also, not that I have a great deal of experience with reviewer comments, but my own experience has been that reviewer comments are usually a paragraph or two. I don’t object to the length of "Mann’s" review, but merely point out that such detail is atypical. For example, here is the GRL review of the Wahl and Ammann Comment and our Reply:

This paper should not be published in GRL. The exchanges between the present authors and everyone who has criticized their earlier paper have gone on too long. The point has been made over and over. The fact seems to be that there are choices to be made in this kind of analysis but the results seem to not be too sensitive to them when they are carried out in good faith (exactly what it means to be robust is of course a matter of judgement, but I have some experience on this and I think it is). The current authors have had their say in numerous other replies to criticisms of their work. It is time to get on with it!

While there were many reasons to reject the Wahl and Ammann submission to GRL, the reviewer’s comments were simply irrelevant to them. Also the reviewer brought up an unrelated issue. In this case, Famiglietti’s cover letter agreed that the review was "brief", but Famiglietti claimed to have had "significant communication" with the reviewer. He also apologized for delays because "it was very difficult to find a qualified and willing reviewer".

Now think back also to a comment by von Storch about how hard it used to be to get articles published on millennial paleoclimate methods, a comment where he generously credited us with opening the door and being lucky enough not to have had Mann as the reviewer for our article.

So here we see that the reviewer for Climates of the Past is probably Mann. It also seems that he was a reviewer for GRL. Are "Mann’s" review comments the sort of thing that should decide whether to publish or not publish – or are they things that should be dealt with in a Reply?

Update: I have reason to believe that Mann may not have been a GRL reviewer, although the GRL reviewer seems to have been on the Hockey Team. So how did Mann know that the Bürger and Cubasch paper had been rejected by GRL? Did the GRL reviewer – perhaps Schmidt – break confidentiality and inform Mann? Will GRL admonish Schmidt or whoever for breaking confidentiality?

By Stephen McIntyre | Posted in General | Comments (156)

Calibration RE

Jul 4, 2006 – 5:36 AM

The NAS Panel notes the following about several statistics used in proxy studies:

If \hat{y} are the predictions from a linear regression of y on the proxies, and the period of interest is the calibration period, then RE, CE, and r^2 are all equal.

Here’s a result about MBH methods (and applicable to related methods with re-scaling) that has not been reported:

If \hat{y} are the predictions from an estimate of y in which \hat{y} has been re-scaled so that its standard deviation matches the standard deviation of the target, and the period of interest is the calibration period, then
(1) CE=RE = 2*r-1.

I can prove this (the proof is fairly trivial but you have to think a little or else some one else would have observed it already) and have verified it against actual results. Notice that the relation is with r directly and not r^2 . I thought of this when I saw a comment of Bürger and Cubasch about correlations, in which they mentioned that correlation and r2 were inappropriate when scale was involved. It’s not like I’m some advocate of one statistic over another, so much as someone who says that you should look at all aspects. In this case, there’s a reason why correlation links directly to explained variance. OK, Gerd, do you agree? This would be a good puzzle for Rob Wilson (who uses re-scaling procedures) to think about as well.

By Stephen McIntyre | Posted in Statistics | Tagged , | Comments (10)

Famiglietti Strikes Again

Jul 3, 2006 – 8:54 AM

There is another terrific article by Bürger and Cubasch posted up here . I’ve just looked at for a few minutes so far and it will take time to fully digest, but one can tell right away that it is a very interesting and stimulating article. Gerd Bürger notified me of it and I therefore bring it to your attention.

It was rejected by GRL.

I don’t know what reasons Famiglietti gave for the rejection. I was going to make some nasty comment about Hockey Teams and water boys, but thought better of it. I suspect that it will be along the lines that they are bored with the matter. But in terms of really understanding the statistical methods, the study of MBH methods has only just begun. It’s taken a long time to even figure out MBH did. That was only the first step and shouldn’t have taken any time.

Now it’s time to figure out the statistical properties of their method, which they should have explained on day 1. It doesn’t matter that they’ve moved on to some even more obscure method. Insight from understanding the statistical properties of something where the methods are now more or less understood will be a big leg up for the task of figuring out the next method.

By Stephen McIntyre | Posted in MBH98 | Tagged | Comments (39)

Spot the Hockey Stick #15: NOAA and a reply to Jerry Pournelle

Jul 3, 2006 – 1:17 AM

(John A): On Jerry Pournelle’s fascinating weblog, a poster has mentioned Steve McIntyre’s expanded horizons into the world of multiproxy studies and the revealing reply to the NAS Panel by D’Arrigo in praise of cherrypicking proxies because "that’s what you have to do if you want to make cherry pie".

Jerry responds:

The basis assumption of statistical inference is random selection of the sample. If that assumpti0n is not met, other inferential tools are required.

Am I correct in concluding that the Hockey Stick has rather quietly vanished as its assumptions were exposed?

Do understand that exposing the alarmists as having systematically selected data is not the same as proving they are wrong; it does mean they have not accounted for all the data.

Later he adds:

Pity it’s not science yet. I would think it important to turn real science loose on a problem of this importance.

It’s been more than a year since the last post on the climate science version of "Where’s Wally?". The Hockey Stick has hardly vanished. A commenter noticed that NOAA persists in presenting the Hockey Stick as fact:

In the early days of paleoclimatology, the sparsely distributed paleoenvironmental records were interpreted to indicate that there was a "Medieval Warm Period" where temperatures were warmer than today. This "Medieval Warm Period" or "Medieval Optimum," was generally believed to extend from the 9th to 13th centuries, prior to the onset of the so-called "Little Ice Age."

In contrast, the evidence for a global (or at least northern hemisphere) "Little Ice Age" from the 15th to 19th centuries as a period when the Earth was generally cooler than in the mid 20th century has more or less stood the test of time as paleoclimatic records have become numerous. The idea of a global or hemispheric "Medieval Warm Period" that was warmer than today however, has turned out to be incorrect.

This is fascinating to me that "the early days of paleoclimatology" were so recent. It makes me feel old that I can remember such ancient times as if they were yesterday.
Continue reading →

By John A | Posted in Spot the Hockey Stick! | Comments (27)

What was "First" About MBH98?

Jul 2, 2006 – 9:39 AM

The NAS Panel claimed that MBH98 was the "first systematic" multiproxy study. It wasn’t; it didn’t even claim to be, citing Bradley and Jones 1993 and several other studies of the same vintage as predecessors. Crowley was a peer reviewer for the NAS panel, who presumably relied on him to catch this sort of mis-step. He should have caught this, but didn’t. So what was the distinctive contribution of MBH98 – if it wasn’t the “first” multiproxy study?

Rather than going at this from the point of view of statistical methods as I’ve spent most of my time doing recently, I’ve gone back to basics and looked at proxy selection. When you look at the proxy series in the studies leading up to MBH98 (and in the contemporary and somewhat rival Jones et al 1998), it’s remarkable to see how many series were taken directly from Bradley and Jones 1993 and the degree of overlap between the proxy selections. There are a couple of surprising conclusions that emerge from a simple parsing and comparison of selections.
Continue reading →

By Stephen McIntyre | Posted in MBH98, NAS Panel | Comments (30)

Groveman and Landsberg

Jul 1, 2006 – 2:01 PM

Jean S pointed out the following quote from the NAS Report and suggested that this be discussed:

The first systematic, statistically based synthesis of multiple climate proxies was carried out in 1998 by M.E.Mann, R.S.Bradley and M.K. Hughes (Mann et al. 1998); their study focused on temperature for the last 600 years in the Northern Hemisphere. The analysis was later extended to cover the last 1,000 years (Mann et al. 1999), and the results were incorporated into the 2001 report of the Intergovernmental Panel on Climate Change (IPCC 2001)."

Jean S drew attention to the following comment from my report on Hughes at NAS:

"Hughes said that the calculation of a global or hemispheric mean was a ‘somewhat tedious afterthought’ to these spatiotemporal maps. He cited Groveman 1979; and Bradley and Jones 1993 as originating the practice."

Here’s a discussion of Groveman and Landsberg 1979; I’ll discuss Bradley and Jones 1993 on another occasion. The conclusion is that MBH is NOT the "first systematic, statistically based synthesis of multiple climate proxies", as both the studies mentioned by Hughes would fit that description. I’ll discuss what the distinctive "contribution" of MBH actually was on another occasion. Continue reading →

By Stephen McIntyre | Posted in Multiproxy Studies, NAS Panel | Tagged | Comments (32)

Rational Decisions, Random Matrices and Spin Glasses

Jun 30, 2006 – 10:00 PM

Interesting title, no? What if I added Principal Components to this odd concatenation of concepts?

Galluccio et al 1998 published a paper with the above title here, which has led to a number of follow-ups, which you can locate by googling. I’ll try to summarize Galluccio’s basic idea and then tie it back into principal components and multiproxy networks.

I haven’t fully grasped all aspects of this article but the concepts seem intriguing relative to issues that we are working on. Continue reading →

By Stephen McIntyre | Posted in Multivariate, Statistics | Tagged , | Comments (11)

Reconciling Zorita

Jun 30, 2006 – 6:13 PM

One bit of housekeeping that I want to tidy up before more NAS postings: a couple of months ago, Eduardo Zorita kindly sent me comprehensive data from ECHO-G, on which, unfortunately, I’ve so far not been able to spend as much time on so far as I would have liked. So much to do, so little time. Included in the package were supporting calculations for their Comment on MM05 in which they stated:

Our results, derived in the artificial world of an extended historical climate simulation, indicate therefore that the AHS [Artificial Hockey Stick] does not have a significant impact but leads only to very minor deviations. We suggest, however, that this biased centering should be in future avoided as it may unnecessarily compromise the final result.

Obviously this is a different conclusion than we reached and I’ve been anxious to reconcile the different findings. Eduardo and I already exchanged code on our replication of Mannian PCs and I’ve confirmed that the key aspects coincide (although Eduardo did not use detrended standard devations in Mannian PC calculations). Continue reading →

By Stephen McIntyre | Posted in General, Multivariate | Comments (28)