Climate Audit

Peter Kuikman, the secretary of the WAB program (which finances the Mitrie project), is reported as saying that the Mitrie project was funded 89,000 euro (US$113,000). Let’s step back for a moment and look at the terms of reference for the project and see if the Dutch government is getting what they contracted for.

The Dutch website of WAB merely states (in an approximate translation) :

Climate Reconstruction
For some time, there has been a scientific discussion resulting from articles of Michael Mann et al., concerning the average annual temperature in the northern hemisphere for the last 1000 years. Precise measurements of air temperature have only been carried out since 1855. It is of utmost importance to get consensus about the interpretation of temperature measurements and proxies, because this information can be used for:

• detection of climate change
• calibration of climate models
• estimating climate sensitivity

In this project we will analyse different interpretation methods used with proxies and the results [of these methods], and if possible, we will try to arrive at the most probable interpretation.Project Leader: M Juckes

Juckes’ Mitrie website states the following:

MITRIE is funded by the Netherlands Environment Assessment Agency (RIVM) http://www.rivm.nl/en/as part of the Netherlands `Scientific Assessment and Policy Analysis’ (WAB) programme. More information about this programme can be found here (in Dutch).

The MITRIE webpage states its objectives as follows:
There has been considerable interest in paleoclimate reconstructions of temperature over the last millennium. A wide variety of techniques have been used, with sometimes obscure details. The interrelation among the techniques is unclear. This project will seek to achieve two distinct but related aims: firstly, to aid public understanding by providing a web site explaining the scientific basis and providing data and software; secondly, to contribute to the scientific debate by reviewing recent work and attempting to isolate the key differences between the different methodologies within a common framework.

Thus while Juckes has recently stated at CA that "I have never advocated that code should always be disclosed" and gives very qualified support to the notion in a recent comment on disclosure at his website, the Dutch government presumably understood that one of the purposes of funding Mitrie was to "provide data and software."

IAMAS 2005
The Mitrie project started up in June or July 2005. This is a climate project and so the first order of business was to go to a conference on another continent – in this case the IAMAS conference in Beijing, which was held from August 2-11 2005, where 841 scientists and students from 54 countries attended the conference and gave over 1000 presentations.

Session C9 was entitled C9: Explaining the Climates of Historic Times: Detection and Attribution of Anthropogenic Influences (ICCL and PAGES) The "First Convener" was Michael E. Mann, Department of Environmental Sciences, Clark Hall, University of Virginia. Presenting from 15:10 to 15:30 was Martin Juckes. Juckes’ presentation, online here told the presumably beaming First Convener Mann that:

Public discussion is heavily influenced by papers by McIntyre and McKittrick, and Soon and Baliunas, which have little credibility in the scientific literature…..

The Mann et al. conclusion that recent high global mean temperatures are exceptional has been verified by many studies…

2005-2006 Chronology
I am not in a position to provide a comprehensive chronology of Mitrie, but will summarize the information on the public record and from my own contacts.

On Dec 12, 2005, a series of short reviews of proxy methodologies was posted at the MITRIE website: on ice cores (Schlosser), early instrumental (Bohm), boreholoe(Pollack), documentary evidence (Brazdil), glaciers (Oelremanns), lake sediment (Bigler)/ Later (Aug 3, 2006), one on ocean sediments (Henderiks) was added. Notably, there is no "short review" on tree rings – the most contentious topic. Schlosser’s short review on ice cores contains the following useful comments:

Although it is clear, that there is a strong relationship between temperature and isotope ratio, the quantitative conversion of d18O to air temperature is a problem. Willi Dansgaard, one of the pioneers in the isotope and ice core business, has always stressed, that only the changes in d18O should be shown in the diagrams, never the change in temperature. Even today, after almost 40 years of ice core research, the problem remains unsolved….

The currently observed phenomenon of “Global Warming” is a widely discussed issue today, and the media love to present as extreme as possible future climate “scenarios”, if not “predictions”. However, as long as we have not completely understood the mechanisms that led to climatic change in the past, we cannot “predict” the climate of the future. We have made large progress with help of the ice cores, but still there are many problems unsolved.

On Jan 5, 2006, without identifying himself as carrying out a study on behalf of the Dutch WAB or identifying the Mitrie program, Martin Juckes sent me the following email (this is presumably the email that Juckes says was sent in December 2005):

Hello Stephen,I’m trying to reproduce some results published by yourself and Mann et al. I wonder if you could send me the R code used in your 2005 Energy and Environment paper, "The M&M critique of the MBH98 northern hemisphere climate index: update and implications". sincerely, Martin Juckes

On Jan 27, 2006, I replied as follows:

Sorry for the delay. Since some time has passed and some new information came out last summer on MBH methods, I thought that I would annotate this a little to show the effects of the new info (which don’t affect the conclusions). I wanted to include both just to avoid any potential confusion. This is taking a little longer than expected (as I’ve been doing some other things as well). I’ll send you both the original code and the update some time next week with any luck. Cheers, Steve

Recently Juckes, in coopering up his allegation that our source code was not "available", said that

Our intended comment (that the code used by MM2005c was not available) was based on the above statement and on an email from Stephen McIntyre to us saying that he would forward the code when it became available. He first informed us of its availability after the publication of our manuscript.

I actually said that I’d try to send it to him the next week – not "when it became available". As things turned out, my plan to add the annotation went down in priority on Feb 7, 2006, when Ross and I received an invitation from the NAS PAnel to make a presentation to them on March 2, 2006. This was an important event for me and I worked very hard over the next few weeks getting ready. However, while I was compiling files for the NAS presentation on March 1, 2006, I remembered the loose end of the MM05(EE) code, which I had intended to archive at the time of publication and forgot to do – shame on me. ( My stale webpage at climate2003.com, prepared in January 2005, stated: "The computer script used to generate the figures and statistics in the E&E here will be located here [in a couple of days] . "

With the big presentation at NAS panel on March 2, 2006, I fixed this loose end – not least of all, because if someone asked me about source code at the NAS Panel, I sure didn’t want to have to make any excuses. Given that I left for Washington on March 1, I must have done it not long before I left for the airport. I was doing a lot of things that day and I forgot to notify Juckes that I’d uploaded the source code. (Of course, Juckes hadn’t reminded me either.)

What was Juckes doing in the meantime? On Feb 3, 2006, he made a list of links at the Mitrie webpage, linking, among others, to Mann’s FTP site, to Ammann’s webpage, to realclimate – but conspicuously not to climateaudit.org – which whether he liked it or not was surely part of the topic that he was considering.

On Feb 14, 2006, he uploaded a list of citations in BibTex format. Conspicuously – and this is highly relevant in light of his subsequent interest in covariance and correlation PCs , neither Huybers [2005] nor our Reply to Huybers [2005] were listed in the citation list (but the VZ Comment and our Reply to VZ are listed.)

The NAS Panel and Wegman Reports were very much in the news between March 2006 and July 2006. I won’t summarize these events as they’ve been amply documented on this blog and elsewhere. However, strikingly, Juckes neither considers nor discusses either report, though both obviously touch on essential items in hockey stick studies.

During spring of 2006, readers of climateaudit were able to observe first hand the highly civilized exchange of code and equally civilized discussion that took place here between Eduardo Zorita and myself, as we reconciled our understandings of Mannian PC methods to ensure that we were discussing the same thing.

In April 28, 2006, Mitrie coauthor Nanne Weber participated in a KNMI workshop on communications in which Marcel Crok also participated. An interesting report on the workshop (in Dutch) is here

It was striking that the participating climate experts could not really refute the criticism on the hockeystick. Koos Verbeek of the KNMI chose to minimise the importance of the Mann hockeystick.

Marcel Crok of NWT interviewed Nanne Weber in the spring (discussed here), in which Nanne Weber indicated that she believed the following (among other points):

· We had failed to show the impact of Mann’s PC methodology on the NH reconstruction even though this was shown in our EE article(which she had not read) …
· When asked about WA confirming that the R2 failure, she told Crok that she was not a specialist on this and that Martin Juckes was responsible for this. When asked by Crok whether she had investigated the R2 failure, she said that she hadn’t and had no intention to do.
· When asked by Crok whether they were planning to calculate the error bars in MBH, she said that she would ask Juckes about this as well…
· she said that she understood why people don’t want to make their data available: they invested so much time in it and it’s so much work to archive everything.
· Crok asked her about the crucial datasets: Polar Urals, Tornetrask, bristlecone pines and foxtails. She told Crok that Tornetrask is only in Esper and Moberg and Polar Urals only in Mann & Jones, Crowley and Lowery and Esper, but would look into it and come back to him on it.

It’s interesting to re-read this list in light of the absence of any consideration of r2/RE issues or MBH error bars in Juckes et al and especially in light of the handling of the stereotyped series: Tornetrask, Polar Urals and bristlecones. I wonder if Nanne Weber actually "looked into it". Maybe she actually thought that Hegerl’s "Norway" series was a different series than "Tornetrask" and "Fennoscandia".

In Sept 2006, I spent nearly an entire day at KNMI, most of it with Nanne Weber and Rob van Dorland. If the Euro Team had any "uncertainty" about what we had done or how we had done it, or inability to locate source code, this was an ideal opportunity to raise the matter. I even discussed my unsatisfactory experience with Ammann and Wahl, who had refused the offer to attempt to jointly list what we agreed on, what we disagreed on and how to resolve the differences, making the same offer to the Euro Team. My notes from that meeting published here mention (inter alis) the following:

We talked about failure of verification statistics. Weber said that Juckes was their statistics guy; he knew about r2 and RE, while she wasn’t much interested in statistics….

It’s pretty easy to predict what their CPD submission will look like. I’ll bet that it has almost exactly the same proxy network as Osborn and Briffa 2006. They will argue that they can get a HS from this network without using PC methods; ergo, everything is fine in Team-world. Anyway, we shall see.

The CPD Submission

On Oct 26, 2006, Juckes et al filed their CPD submission. This was noted up on the blog here later that day (blogtime – 4:29:52 pm; GMT 22 30 GMT) , On Oct 27, Juckes archived some relevant software (though not a unifying script) python.tar.

The Juckes’ submission reviewed the discussion in McIntyre and McKitrick (GRL 2005) of the biased PC method in MBH98, arguing that these criticisms were not "relevant here" and then stated the code was "not, at the time of writing, available".

McIntyre and McKitrick (2005a) [MM2005] continue the criticism of the techniques used by MBH1998 and introduce a “hockey stick index” which is defined in terms of the ratio of the variance at the end of a time series to the variance over the remainder of the series. MM2005 argue that the way in which the stage (1) principal component analysis is carried out in MBH generates an artificial bias towards a high “hockey-stick 5 index” and that the statistical significance of the MBH results may be lower than originally estimated….Thus, the concerns about the latter two points raised by MM2005 do not appear to be relevant here, though the sensitivity to 5 adjustments of principal component may be a cause for concern.

The code used by MM2005 is not, at the time of writing, available, but the code fragments included in their text imply that their calculation used data which had been centred (mean removed) but had not been normalised to unit variance (standardised).

On Oct 29 (Eastern 8.46 am; GMT 13 46), I objected to Juckes about this claim as follows:

As you either know or should know, the code used in McIntyre and McKitrick 2005 is available at the Supplementary Information to the article at ftp://ftp.agu.org/apend/gl/2004GL021750, as is made clear in the article itself. (The code for MM05 (EE) and MM03 (EE) are at http://www.climate2003.com/scripts/MM05_EE and http://www.climate2003.com/scripts/MM03 respectively). The Wegman Report specifically noted that they verified availability of our source code at the time of their report last summer. Previously, both Huybers and Wahl and Ammann had examined the source code, neither of whom required any assistance from me. Huybers annotated the code in his Supplementary Information.

Juckes subsequently tried to cooper up his claim by saying that he really meant to make this claim about the MM05 (EE) article (notwithstanding the fact that the lead-in discussion was about MM05 (GRL)) and then blamed the stale webpage at climate2003.com for his inability to find the code. (BTW there’s no evidence that he consulted the MM05 (GRL) code which was at the GRL website all along.) I’ll make a separate post on how Juckes seems to have spent November 2.

But let’s go back to the Dutch agency that paid for this and ask them to evaluate Juckes’ little hissy fit here:

• on Jan 5, 2006, Juckes asked me for MM05 (EE) code; on Jan 20, 2006, I undertook to try to send an annotated version to him the following week.
• on March 1, 2006, the code was archived, but I forgot to notify Juckes;
• although the code had been archived, the index page at climate2003.com, which was not my primary website, had not been updated since January 2005 and the statement that the code would be archived "in a few days" was still on the index page.
• between Jan 5, 2006 and Oct 27, 2006, I exchanged over 100 emails with 2 different Mitrie coauthors, actually reconciiling code with one of them;
• between Jan 5, 2006 and Oct 27, 2006, I met personally with 2 different Mitrie coauthors, spending nearly an entire day with Nanne Weber, inviting the opportunity to reconcile any problems;
• I have been visibly online almost every day between Jan 5, 2006 and Oct 27, 2006.

Yes, it would have been better if my webpage hadn’t been stale, but let’s pause for a moment and think about how ineffective poor old Martin Juckes was in this process. If he’d told Eduardo Zorita that he had a problem finding the source code, how long does anyone think it would have taken to get him to the correct url? If he’d come online at climateaudit and reproached me, how long does anyone think it would have taken to get him to the right url?

If it was relevant to him to get the MM05 (EE) code, he should have gotten it. A stale webpage isn’t a good enough excuse for not locating the code. There were lots of ways to locate the code and Juckes tried none of them. He sent one email to me, then sucked his thumb. Juckes can point his finger at me all he likes, but he and his group were the ones that got the $112,000. Peter Kuikman, the guy in the Dutch government that turned over $112,000 to Juckes et al, should be the one that’s calling Juckes and Nanne Weber on the carpet.