Making Apple Pie Instead of Cherry Pie

Mar 11, 2006 – 8:11 AM

One of the concerns that people are increasingly expressing to us in reaction to demonstrations of methodology problems in Hockey Team multiproxy studies is more or less this: even if the methods used in these various studies are flawed from the point of view of a statistical purist, no one’s presented an alternative interpretation of the proxy data. Here’s a recipe for apple pie instead of cherry pie. Continue reading →

By Stephen McIntyre | Posted in Multiproxy Studies | Tagged , , , | Comments (121)

NAS Panel Reading List

Mar 10, 2006 – 11:30 PM

The minutes for the NAS Panel here includes a reading list. They provide citations for each of the PPT presentations, including ours. They mention a “handout” and “CD” from us.

The “handout” was a formal written presentation to the panel, setting out the points in our PPT. It deserves a citation. I wonder whether they even gave it to the panel members. The reading list includes the Comments on our work by Von Storch-Zorita and by Huybers, but it does not include our Replies (which were also on the CD). I know that it was probably a mistake and no harm was intended, but, at best, it’s amateurish.

I wonder how they made up this reading list? There aren’t any articles on spurious regression or calculating confidence intervals. There aren’t any articles about replication standards or disclosure.

By Stephen McIntyre | Posted in NAS Panel | Comments (2)

Sciencemag on NAS Panel

Mar 10, 2006 – 8:13 PM

Here’s a summary (archive)from Science of last week’s NAS panel.

The heat was on a 12-person National Research Council committee last week as it tackled the politically charged debate over how scientists have gauged temperatures from the past millennium or two. Chair Gerald North of Texas A&M University in College Station kept the audience on a tight leash, including principal protagonists Michael Mann of Pennsylvania State University in State College and his critics, Stephen McIntyre of the University of Toronto, Canada, and Ross McKitrick of the University of Guelph, Canada. House Science Committee Chair Representative Sherwood Boehlert (R-NY) had requested the study in the wake of attacks on Mann’s “hockey stick” temperature curve showing an abrupt, presumably human-induced warming over the last century (Science, 1 July 2005, p. 31).

Mann made himself scarce throughout the proceedings, even abruptly departing as McIntyre stood to make a final comment. Others, however, had already provided independent support for temperature trends resembling Mann’s, and Mann himself pointed out that he had sworn off the criticized analytical method years ago. The committee has promised a report on the science of millennial temperatures in June.

It’s funny how people view things differently. For example, I thought that the take-home issue arising from D’Arrigo was the quandary of the Divergence Problem. Yes, she presented a reconstruction that was generically somewhat hockey-stickish, but if the proxies and methods are problematic, so what?

One thing everyone can agree on is that Mann made “himself scarce”. I didn’t realize that Mann exited when I made a short comment on Friday morning. It was a very short comment saying that I thought that the committee had dealt with verification statistics in a very unsatisfactory way, which they did.

But I talked less than 30 seconds. So he must have moved fast. As fast as a cheetah.

No wonder I couldn’t find him to say hello. I really was going to say hello to him afterwards. I introduced myself to Hughes and chatted about Liverpool soccer – so don’t say that I wasn’t going to.

What would I have said to him? I don’t know, I hadn’t thought about it. I’d probably have thanked him for getting me involved in an interesting project and taken it from there. Maybe he’d have thanked me for interesting commentary and all the good times. Maybe we’d talk basketball – he’s near Philadelphia, what does think about Alan Iverson? Or what about Joe Paterno as Penn State football coach? How does Penn State look for next season? Precious moments…. when will we spend precious moments?

By Stephen McIntyre | Posted in NAS Panel, News and Commentary | Tagged | Comments (28)

IPCC 4AR and Ammann

Mar 10, 2006 – 8:36 AM

Luboà…⟠Motl pointed out that IPCC "needs" Ammann and Wahl in a peer reviewed journal. Let’s re-visit some curious timing issues, which Ian Castles brought up before and which need to be re-examined with the re-submission.

The IPCC WG1 timetable (thanks to Ian for this) says the following:

Third Lead Author meeting, December 13 to 15, in Christchurch, New Zealand. This meeting considers comments on the first order draft and writing of the second order draft starts immediately afterwards.Note. Literature to be cited will need to be published or in press by this time. Copies of literature not available through normal library sources should be sent to the TSU so they can be made available to reviewers if requested.
Meeting of the TS/SPM writing team December 16 in Christchurch, New Zealand

Now let’s look at the UCAR website for Ammann and Wahl, which provides the following information:

Climatic Change
May 10, 2005 — In review
September 27, 2005 — Revised
December 12, 2005 — Provisionally Accepted
February 28, 2006 — Accepted for Publication

The version that was accepted was dated Feb. 24, 2006. It looks like there was a major re-write between December and February with the addition of all the piffle on RE and r2. I wonder what peer review took place between Feb 24 and Feb 28. In my July 25, I’d given a list of statistical references in which RE statistics had been discussed – none of them were cited. Shouldn’t that have been dealt with? My guess is that all the sections of Ammann and Wahl pertaining to RE and r2 were never externally peer reviewed.

Obviously, Ammann and Wahl was neither "published or in press" on December 13-15. In fact, the present version was not even finalized until Feb. 24, 2006. Are there material differences between the versions? Obviously. The differences are not just picky differences. The revised version completely vindicates our claims about MBH verification statistics (however unwillingly), while the earlier version provided to IPCC concealed this.

So under its own rules, is IPCC allowed to refer to Ammann and Wahl [2006]? Of course not. Will they? We all know the answer to that. When they refer to Ammann and Wahl [2006], will they also refer to its confirmation of our claims about MBH verification r2 statistics. Of course not. That information was not available to them in December. But wait a minute, if Ammann and Wahl was in press in December, wouldn’t that information have been available to them? Silly me.

By Stephen McIntyre | Posted in IPCC, Wahl and Ammann | Tagged , , | Comments (61)

Wahl and Ammann – July 2005 Review

Mar 9, 2006 – 4:36 PM

I’m posting up our July 2005 review of Wahl and Ammann. The recently accepted version is here. I’m posting this up for a variety of reasons. Mann relied heavily on Wahl and Ammann in his NAS panel testimony (which wasn’t even online as accepted last week) and so it’s hard to finish off the discussion of NAS without wading through Wahl and Ammann. Also the associated UCAR Media Advisory that all our claims were “unfounded” had been widely cited.

Comparing W&A against my previous comments in a peer review capacity is an interesting object lesson in what peer review is and isn’t. It’s also interesting to speculate as to whether IPCC considerations may have affected how it’s been handled. Continue reading →

By Stephen McIntyre | Posted in Wahl and Ammann | Tagged , , , , , | Comments (40)

Esper on In-Site Cherry Picking

Mar 9, 2006 – 12:56 PM

I noticed the following quote from Esper et al 2003 (reference in earlier post

It is important to know that at least in distinct periods subsets of trees deviate from common trends recorded in a particular site. Such biased series represent a characteristic feature in the process of chronology building. Leaving these trees in the pool of series to calculate a mean site curve would result in a biased chronology as well. However if the variance between the majorities of trees in a site is common, the biased individual series can be excluded from the further investigation steps. This is generally done even if the reasons for uncommon growth reactions are unknown.

I posted up the next quote from Esper previously, but it’s worth repeating in the present context:

However as we mentioned earlier on the subject of biological growth populations, this does not mean that one could not improve a chronology by reducing the number of series used if the purpose of removing samples is to enhance a desired signal. The ability to pick and choose which samples to use is an advantage unique to dendroclimatology.

These statements just make my jaw drop. If the information at a site level is being fiddled with (“adjusted”) in the way that Esper indicates here (and in my opinion, it would be a worthwhile investigation for someone to check whether it is – and not just from the bosses), how can you ever rely on anything? If the field workers know that the boss has a "signal" in mind and their methods do not require 100% of the data to be recorded, you have biases way beyond D’Arrigo cherry picking individual sites. Maybe I’m just interpreting injudicious comments in an adverse light, but the comments are hugely inappropriate for authors of studies being relied on by IPCC and policy-makers and deserve to be looked into.

By Stephen McIntyre | Posted in Esper et al 2002, Multiproxy Studies | Comments (37)

Enron Trial in the News

Mar 9, 2006 – 10:52 AM

Are any of you keeping track of the news on the trials of Enron executives Ken Lay and Jeffrey Skilling? Andrew Fastow, their CFO, was on the stand yesterday. There’s a terrific book about Enron by Kurt Eichenwald, in which the House Energy and Commerce Committee is mentioned (they had a piece of some Enron hearings). I’d like to spend a few minutes explaining what my take is on the actual Enron fraud, since many people (IMHO) don’t understand the differences between the fraud and simply losing money and then ruminate a little on data policy. Continue reading →

By Stephen McIntyre | Posted in Disclosure and Diligence | Tagged , , , | Comments (17)

Peer Reviewing Wahl and Ammann #1 – The Correspondence

Mar 8, 2006 – 4:09 PM

Mann cited Wahl and Ammann’s recently released paper at NAS (which was not available to us in time for the NAS panel, although I’d seen and reviewed an earlier draft.) After reading it, Per said that he thought that the reviewers had done a lousy job.

Now I was only a reviewer for the first draft (after sending in my review, I seem to have been replaced; I never heard anything more from Climatic Change.) I provided many detailed comments that were simply ignored. Obviously I have an interest in the matter, but Schneider knew of my interest and presumably that was one of the reasons that he asked me to review. (I had previously reviewed a submission by MBH to Climatic Change in 2004, which has never seen the light of day. My review was detailed, so Schneider knew how I reviewed things. Actually my review of the MBH submission led to the introduction of a limited data policy at Climatic Change for the first time, so I had a positive contribution.) In this case however, Schneider disregarded most of my comments – which, by and large, pertain to objective things even though there is a controversial edge.

The only materially new sections of the revised article are the discussions of RE versus R2, low frequency versus high frequency, Appendixes 1-3 plus of course the table of verification statistics. Ammann and Wahl say that they have provided the verification statistics so they are “available to the community”. Good and I’m glad that they saw the light. But let’s be clear about this – they were not added as a simple response to a reviewer request. They refused to provide this information to me as a reviewer and Schneider abetted this. Readers of this blog know that, as recently as December AGU, Ammann was still refusing to disclose the verification r2 and similar statistics. It was only because we filed a complaint with UCAR about misconduct in withholding the results and because I’ve got an audience at the blog and have hammered away at Ammann for attempting to withhold adverse results, that they disclosed the adverse results.

I’ll have quite a bit to say about the article itself in a few days. (I’ve still got to bring the NAS panel to the end of last Friday.)

No one should be under the impression that, if I’d been a little “nicer” to Ammann, that he would have done this on his own. I’ve made some very polite offers to Ammann and have been ignored. He fought disclosure to the bitter end. If you don’t believe me, read on. This goes up to our review of Ammann and Wahl, which I’ll post up tomorrow or the next day. I’ve started with some requests from me to Ammann long before the Climatic Change process began. I’ve described my discussions with Ammann at AGU elsewhere. Continue reading →

By Stephen McIntyre | Posted in Wahl and Ammann | Tagged , , , , , | Comments (42)

Hughes at NAS

Mar 8, 2006 – 11:38 AM

I’m going to jump ahead a little and report on Hughes, who spoke on Friday morning. My notes on Hughes are decent by my standards. I’ll come back and describe our presentation next and then get to Mann’s. Neither Hughes nor Mann attended Thursday’s session or reception. I missed meeting Mann on Friday as I spent a few moments after the Friday morning session chatting to people and when I looked up, Mann was gone. I introduced myself to Hughes after the Friday morning session. He was chatting with the BBC reporters about Liverpool soccer and a miraculous comeback last year by Liverpool in Istanbul (which I’d vaguely heard about.) We chit-chatted pleasantly about soccer and administration and such.

In his morning presentation, Hughes said that their mission was to discover something about the climate system. He described the calculation of a global mean as a "somewhat tedious afterthought". He made a very interesting distinction between the Schweingruber and Fritts approaches to disentangling temperature signals from tree ring data. Continue reading →

By Stephen McIntyre | Posted in General, NAS Panel | Comments (8)

Why peer reviewed publication is not enough

Mar 8, 2006 – 10:52 AM

Obviously Climate Audit has captured a small part of the zeitgeist of the scientific world, especially in regards to the obvious failures of peer review to detect bad practice and scientific misconduct. It has been asked by some climate scientists why access to original data and full disclosure is so important, as if proper audit and replication were an invasive medical procedure rather than intrinsic part of the Scientific Method.

From another area of science, which has already had one thumping case of scientific misconduct already, another one appears to be coming to the boil: the claim of tabletop fusion.
Continue reading →

By John A | Posted in News and Commentary, Replication | Comments (12)