Juckes and the Mitrie Project

Peter Kuikman, the secretary of the WAB program (which finances the Mitrie project), is reported as saying that the Mitrie project was funded 89,000 euro (US$113,000). Let’s step back for a moment and look at the terms of reference for the project and see if the Dutch government is getting what they contracted for.

The Dutch website of WAB merely states (in an approximate translation) :

Climate Reconstruction
For some time, there has been a scientific discussion resulting from articles of Michael Mann et al., concerning the average annual temperature in the northern hemisphere for the last 1000 years. Precise measurements of air temperature have only been carried out since 1855. It is of utmost importance to get consensus about the interpretation of temperature measurements and proxies, because this information can be used for:

  • detection of climate change
  • calibration of climate models
  • estimating climate sensitivity

In this project we will analyse different interpretation methods used with proxies and the results [of these methods], and if possible, we will try to arrive at the most probable interpretation.Project Leader: M Juckes

Juckes’ Mitrie website states the following:

MITRIE is funded by the Netherlands Environment Assessment Agency (RIVM) http://www.rivm.nl/en/as part of the Netherlands `Scientific Assessment and Policy Analysis’ (WAB) programme. More information about this programme can be found here (in Dutch).

The MITRIE webpage states its objectives as follows:
There has been considerable interest in paleoclimate reconstructions of temperature over the last millennium. A wide variety of techniques have been used, with sometimes obscure details. The interrelation among the techniques is unclear. This project will seek to achieve two distinct but related aims: firstly, to aid public understanding by providing a web site explaining the scientific basis and providing data and software; secondly, to contribute to the scientific debate by reviewing recent work and attempting to isolate the key differences between the different methodologies within a common framework.

Thus while Juckes has recently stated at CA that "I have never advocated that code should always be disclosed" and gives very qualified support to the notion in a recent comment on disclosure at his website, the Dutch government presumably understood that one of the purposes of funding Mitrie was to "provide data and software."

IAMAS 2005
The Mitrie project started up in June or July 2005. This is a climate project and so the first order of business was to go to a conference on another continent – in this case the IAMAS conference in Beijing, which was held from August 2-11 2005, where 841 scientists and students from 54 countries attended the conference and gave over 1000 presentations.

Session C9 was entitled C9: Explaining the Climates of Historic Times: Detection and Attribution of Anthropogenic Influences (ICCL and PAGES) The "First Convener" was Michael E. Mann, Department of Environmental Sciences, Clark Hall, University of Virginia. Presenting from 15:10 to 15:30 was Martin Juckes. Juckes’ presentation, online here told the presumably beaming First Convener Mann that:

Public discussion is heavily influenced by papers by McIntyre and McKittrick, and Soon and Baliunas, which have little credibility in the scientific literature…..

The Mann et al. conclusion that recent high global mean temperatures are exceptional has been verified by many studies…

2005-2006 Chronology
I am not in a position to provide a comprehensive chronology of Mitrie, but will summarize the information on the public record and from my own contacts.

On Dec 12, 2005, a series of short reviews of proxy methodologies was posted at the MITRIE website: on ice cores (Schlosser), early instrumental (Bohm), boreholoe(Pollack), documentary evidence (Brazdil), glaciers (Oelremanns), lake sediment (Bigler)/ Later (Aug 3, 2006), one on ocean sediments (Henderiks) was added. Notably, there is no "short review" on tree rings – the most contentious topic. Schlosser’s short review on ice cores contains the following useful comments:

Although it is clear, that there is a strong relationship between temperature and isotope ratio, the quantitative conversion of d18O to air temperature is a problem. Willi Dansgaard, one of the pioneers in the isotope and ice core business, has always stressed, that only the changes in d18O should be shown in the diagrams, never the change in temperature. Even today, after almost 40 years of ice core research, the problem remains unsolved….

The currently observed phenomenon of “Global Warming” is a widely discussed issue today, and the media love to present as extreme as possible future climate “scenarios”, if not “predictions”. However, as long as we have not completely understood the mechanisms that led to climatic change in the past, we cannot “predict” the climate of the future. We have made large progress with help of the ice cores, but still there are many problems unsolved.

On Jan 5, 2006, without identifying himself as carrying out a study on behalf of the Dutch WAB or identifying the Mitrie program, Martin Juckes sent me the following email (this is presumably the email that Juckes says was sent in December 2005):

Hello Stephen,I’m trying to reproduce some results published by yourself and Mann et al. I wonder if you could send me the R code used in your 2005 Energy and Environment paper, "The M&M critique of the MBH98 northern hemisphere climate index: update and implications". sincerely, Martin Juckes

On Jan 27, 2006, I replied as follows:

Sorry for the delay. Since some time has passed and some new information came out last summer on MBH methods, I thought that I would annotate this a little to show the effects of the new info (which don’t affect the conclusions). I wanted to include both just to avoid any potential confusion. This is taking a little longer than expected (as I’ve been doing some other things as well). I’ll send you both the original code and the update some time next week with any luck. Cheers, Steve

Recently Juckes, in coopering up his allegation that our source code was not "available", said that

Our intended comment (that the code used by MM2005c was not available) was based on the above statement and on an email from Stephen McIntyre to us saying that he would forward the code when it became available. He first informed us of its availability after the publication of our manuscript.

I actually said that I’d try to send it to him the next week – not "when it became available". As things turned out, my plan to add the annotation went down in priority on Feb 7, 2006, when Ross and I received an invitation from the NAS PAnel to make a presentation to them on March 2, 2006. This was an important event for me and I worked very hard over the next few weeks getting ready. However, while I was compiling files for the NAS presentation on March 1, 2006, I remembered the loose end of the MM05(EE) code, which I had intended to archive at the time of publication and forgot to do – shame on me. ( My stale webpage at climate2003.com, prepared in January 2005, stated: "The computer script used to generate the figures and statistics in the E&E here will be located here [in a couple of days] . "

With the big presentation at NAS panel on March 2, 2006, I fixed this loose end – not least of all, because if someone asked me about source code at the NAS Panel, I sure didn’t want to have to make any excuses. Given that I left for Washington on March 1, I must have done it not long before I left for the airport. I was doing a lot of things that day and I forgot to notify Juckes that I’d uploaded the source code. (Of course, Juckes hadn’t reminded me either.)

What was Juckes doing in the meantime? On Feb 3, 2006, he made a list of links at the Mitrie webpage, linking, among others, to Mann’s FTP site, to Ammann’s webpage, to realclimate – but conspicuously not to climateaudit.org – which whether he liked it or not was surely part of the topic that he was considering.

On Feb 14, 2006, he uploaded a list of citations in BibTex format. Conspicuously – and this is highly relevant in light of his subsequent interest in covariance and correlation PCs , neither Huybers [2005] nor our Reply to Huybers [2005] were listed in the citation list (but the VZ Comment and our Reply to VZ are listed.)

The NAS Panel and Wegman Reports were very much in the news between March 2006 and July 2006. I won’t summarize these events as they’ve been amply documented on this blog and elsewhere. However, strikingly, Juckes neither considers nor discusses either report, though both obviously touch on essential items in hockey stick studies.

During spring of 2006, readers of climateaudit were able to observe first hand the highly civilized exchange of code and equally civilized discussion that took place here between Eduardo Zorita and myself, as we reconciled our understandings of Mannian PC methods to ensure that we were discussing the same thing.

In April 28, 2006, Mitrie coauthor Nanne Weber participated in a KNMI workshop on communications in which Marcel Crok also participated. An interesting report on the workshop (in Dutch) is here

It was striking that the participating climate experts could not really refute the criticism on the hockeystick. Koos Verbeek of the KNMI chose to minimise the importance of the Mann hockeystick.

Marcel Crok of NWT interviewed Nanne Weber in the spring (discussed here), in which Nanne Weber indicated that she believed the following (among other points):

· We had failed to show the impact of Mann’s PC methodology on the NH reconstruction even though this was shown in our EE article(which she had not read) …
· When asked about WA confirming that the R2 failure, she told Crok that she was not a specialist on this and that Martin Juckes was responsible for this. When asked by Crok whether she had investigated the R2 failure, she said that she hadn’t and had no intention to do.
· When asked by Crok whether they were planning to calculate the error bars in MBH, she said that she would ask Juckes about this as well…
· she said that she understood why people don’t want to make their data available: they invested so much time in it and it’s so much work to archive everything.
· Crok asked her about the crucial datasets: Polar Urals, Tornetrask, bristlecone pines and foxtails. She told Crok that Tornetrask is only in Esper and Moberg and Polar Urals only in Mann & Jones, Crowley and Lowery and Esper, but would look into it and come back to him on it.

It’s interesting to re-read this list in light of the absence of any consideration of r2/RE issues or MBH error bars in Juckes et al and especially in light of the handling of the stereotyped series: Tornetrask, Polar Urals and bristlecones. I wonder if Nanne Weber actually "looked into it". Maybe she actually thought that Hegerl’s "Norway" series was a different series than "Tornetrask" and "Fennoscandia".

In Sept 2006, I spent nearly an entire day at KNMI, most of it with Nanne Weber and Rob van Dorland. If the Euro Team had any "uncertainty" about what we had done or how we had done it, or inability to locate source code, this was an ideal opportunity to raise the matter. I even discussed my unsatisfactory experience with Ammann and Wahl, who had refused the offer to attempt to jointly list what we agreed on, what we disagreed on and how to resolve the differences, making the same offer to the Euro Team. My notes from that meeting published here mention (inter alis) the following:

We talked about failure of verification statistics. Weber said that Juckes was their statistics guy; he knew about r2 and RE, while she wasn’t much interested in statistics….

It’s pretty easy to predict what their CPD submission will look like. I’ll bet that it has almost exactly the same proxy network as Osborn and Briffa 2006. They will argue that they can get a HS from this network without using PC methods; ergo, everything is fine in Team-world. Anyway, we shall see.

The CPD Submission

On Oct 26, 2006, Juckes et al filed their CPD submission. This was noted up on the blog here later that day (blogtime – 4:29:52 pm; GMT 22 30 GMT) , On Oct 27, Juckes archived some relevant software (though not a unifying script) python.tar.

The Juckes’ submission reviewed the discussion in McIntyre and McKitrick (GRL 2005) of the biased PC method in MBH98, arguing that these criticisms were not "relevant here" and then stated the code was "not, at the time of writing, available".

McIntyre and McKitrick (2005a) [MM2005] continue the criticism of the techniques used by MBH1998 and introduce a “hockey stick index” which is defined in terms of the ratio of the variance at the end of a time series to the variance over the remainder of the series. MM2005 argue that the way in which the stage (1) principal component analysis is carried out in MBH generates an artificial bias towards a high “hockey-stick 5 index” and that the statistical significance of the MBH results may be lower than originally estimated….Thus, the concerns about the latter two points raised by MM2005 do not appear to be relevant here, though the sensitivity to 5 adjustments of principal component may be a cause for concern.

The code used by MM2005 is not, at the time of writing, available, but the code fragments included in their text imply that their calculation used data which had been centred (mean removed) but had not been normalised to unit variance (standardised).

On Oct 29 (Eastern 8.46 am; GMT 13 46), I objected to Juckes about this claim as follows:

As you either know or should know, the code used in McIntyre and McKitrick 2005 is available at the Supplementary Information to the article at ftp://ftp.agu.org/apend/gl/2004GL021750, as is made clear in the article itself. (The code for MM05 (EE) and MM03 (EE) are at http://www.climate2003.com/scripts/MM05_EE and http://www.climate2003.com/scripts/MM03 respectively). The Wegman Report specifically noted that they verified availability of our source code at the time of their report last summer. Previously, both Huybers and Wahl and Ammann had examined the source code, neither of whom required any assistance from me. Huybers annotated the code in his Supplementary Information.

Juckes subsequently tried to cooper up his claim by saying that he really meant to make this claim about the MM05 (EE) article (notwithstanding the fact that the lead-in discussion was about MM05 (GRL)) and then blamed the stale webpage at climate2003.com for his inability to find the code. (BTW there’s no evidence that he consulted the MM05 (GRL) code which was at the GRL website all along.) I’ll make a separate post on how Juckes seems to have spent November 2.

But let’s go back to the Dutch agency that paid for this and ask them to evaluate Juckes’ little hissy fit here:

  • on Jan 5, 2006, Juckes asked me for MM05 (EE) code; on Jan 20, 2006, I undertook to try to send an annotated version to him the following week.
  • on March 1, 2006, the code was archived, but I forgot to notify Juckes;
  • although the code had been archived, the index page at climate2003.com, which was not my primary website, had not been updated since January 2005 and the statement that the code would be archived "in a few days" was still on the index page.
  • between Jan 5, 2006 and Oct 27, 2006, I exchanged over 100 emails with 2 different Mitrie coauthors, actually reconciiling code with one of them;
  • between Jan 5, 2006 and Oct 27, 2006, I met personally with 2 different Mitrie coauthors, spending nearly an entire day with Nanne Weber, inviting the opportunity to reconcile any problems;
  • I have been visibly online almost every day between Jan 5, 2006 and Oct 27, 2006.

Yes, it would have been better if my webpage hadn’t been stale, but let’s pause for a moment and think about how ineffective poor old Martin Juckes was in this process. If he’d told Eduardo Zorita that he had a problem finding the source code, how long does anyone think it would have taken to get him to the correct url? If he’d come online at climateaudit and reproached me, how long does anyone think it would have taken to get him to the right url?

If it was relevant to him to get the MM05 (EE) code, he should have gotten it. A stale webpage isn’t a good enough excuse for not locating the code. There were lots of ways to locate the code and Juckes tried none of them. He sent one email to me, then sucked his thumb. Juckes can point his finger at me all he likes, but he and his group were the ones that got the $112,000. Peter Kuikman, the guy in the Dutch government that turned over $112,000 to Juckes et al, should be the one that’s calling Juckes and Nanne Weber on the carpet.

21 Comments

  1. Posted Nov 4, 2006 at 2:59 PM | Permalink

    Isn’t it fascinating that when asked to investigate Michael Mann’s construction (which is heavily about tree rings) they alter the terms of reference to talk about all reconstructions and Mann is nowhere to be found – the exact same alteration that the NAS Panel used when they were commissioned by Congressional Committee.

    To me, it looks like climate scientists are averting their eyes to MBH98 and refusing to investigate it properly. My guesses would be that they don’t like to be seen to be attacking a paragon of climate alarmism, but more importantly because when it comes to the statistics they have a lot less to say than Steve has already said.

    They refuse to grasp the nettles of spurious regressions, cherry picking and bad methodologies.

  2. John Lish
    Posted Nov 4, 2006 at 3:26 PM | Permalink

    Oh Martin, what tangled webs we weave…

  3. Hans Erren
    Posted Nov 4, 2006 at 4:42 PM | Permalink

    in

    http://www.mnp.nl/nl/themasites/wab/producten/overig.html

    the third bullet item:

    het schatten van de klimaatgevoeligheid

    is best translated as: “estimation of climate sensitivity”, not climate variability.

  4. Barclay E. MacDonald
    Posted Nov 4, 2006 at 5:02 PM | Permalink

    Steve M. you have been persistently and openly critical of Mann and his gang of 42 for a quite awhile. As you are well aware, they are looking for opportunities to discredit you and avenge their fragile egos.

    They will find that you have made mistakes from time to time. But I agree that when their complaints are so misleading and contrived and are undoubtedly there to set you up for future criticism, you should make it appropriately painful for them.

    All we really want from them is truth and honesty. I certainly don’t see it here. Thanks for your continued efforts to “audit” the gang of 42.

  5. Francois Ouellette
    Posted Nov 4, 2006 at 6:11 PM | Permalink

    #4 Most of Steve’s criticisms are posted on this blog, which is not a scientific paper. Mann and his team are free to comment here and respond to those criticisms in real time. To attempt to discredit Steve in a scientific paper, where the dynamics of responding and refuting such allegations is very, very much different, just goes to show that these people can’t and won’t play an honest game. Of course, doing so would mean they would lose… in fact, everybody knows they’ve already lost.

  6. bender
    Posted Nov 4, 2006 at 7:05 PM | Permalink

    Lost what? Is there a game on? You’re delusional. There is no “Team”.
    LOL

  7. jae
    Posted Nov 4, 2006 at 9:37 PM | Permalink

    There is a team, but sadly, no referees.

  8. Posted Nov 5, 2006 at 3:48 AM | Permalink

    Now it’s clear: Steve McIntyre is guilty, he has not got a secretary yet! :-)

  9. Steve McIntyre
    Posted Nov 5, 2006 at 9:08 AM | Permalink

    bender, what would be your over/under on pinned smoothing in Juckes?

  10. bender
    Posted Nov 5, 2006 at 9:33 AM | Permalink

    Well, prior to your asking the question I would have said very low probability. (I’m a generous guy.) Now that you say it, I’m adjusting my Bayesian priors. I’m sure you’re asking this tongue-in-cheek question for a darn good reason … Let me take a look at the data and then give you my revised odds.

  11. Steve McIntyre
    Posted Nov 5, 2006 at 10:32 AM | Permalink

    Juckes hasn’t archived a unifying script for his program – he’s going to do that after it gets accepted. However in his excited comment on the supposed "error" in MM05b, he archived a script here in Fig2p.txt: http://home.badc.rl.ac.uk/mjuckes/mitrie_files/software/mcintyre_mckitrick2005/

    which includes the following smoothing function:

    sm ASSIGN function(x,y){
    l ASSIGN length( x )
    z = c( rep( x[1], y ), x, rep(x[l], y) )
    a = filter( z, rep( 1./(2*y+1), (2*y+1) ), method=”convolution”, sides=2, circular = TRUE )
    a[(1+y):(l+y)]
    }

  12. MarkR
    Posted Nov 5, 2006 at 11:07 AM | Permalink

    How is anyone supposed to “peer review” work if the calculation script isn’t available?

    Federally funded work including code should be made available to other researchers upon
    reasonable request, especially if the intellectual property has no commercial value.

    Wegman Page 5

  13. Steve McIntyre
    Posted Nov 5, 2006 at 11:21 AM | Permalink

    #12. Examination of script is NEVER done in climate peer review. I acted as a peer reviewer for Climatic Change and asked to see script; the editor, Stephen Schneider, said that no one had ever made a similar request in 28 years of editing and refused to even ask the author to provide script.

  14. Posted Nov 5, 2006 at 1:56 PM | Permalink

    And, governments are making climate change policy on 28 years of unevaluated scripts. Where is all this blind science going to lead us? With out CA doing the heavy lifting, we would be blindly following Al Gore off an economic cliff. Thanks Steve for all you do!

  15. Willis Eschenbach
    Posted Nov 5, 2006 at 4:13 PM | Permalink

    I haven’t examined the logic under the function, but the function is clear. “circular = TRUE” defines how it handles the endpoints, and that it handles them by padding the end of the data by mirroring (around x but not y) at the ends of the function. If so, that’s not the best method possible … but on the other hand, it doesn’t pin the endpoints.

    w.

  16. Steve McIntyre
    Posted Nov 5, 2006 at 9:05 PM | Permalink

    #15. NOte that Juckes first pads the series with 10 repetitions of the closing value. So the end point is very heavily weighted and it’s a near-pin.

  17. Willis Eschenbach
    Posted Nov 5, 2006 at 11:19 PM | Permalink

    #16. Ooooh, what does the tricksy Hobbit have in its pockets? Pads it first, then wraps it … nasssty. Makes it look like you haven’t pinned it, when in essence you have. In fact, since it’s padded with “y” repetitions, and the length of the filter is 2*y+1, it never wraps at all.

    This is a very poor choice for endpoints. It is not justified by any mathematical or practical advantage, all it does is tend to pull the average towards the final point. It is equivalent to assuming that the recent trend will continue, which is correct … but only about half the time …

    w.

  18. Posted Nov 6, 2006 at 4:45 AM | Permalink

    re #11 and following: this is taken from code which was trying to reconcile differences between our calculations and Stephens. It does exactly the same as the extend.persist function used by Stephen. This technique is not used in our published study. Following earlier criticism (I thought by Stephen, but perhaps it was someone else, given that he obvbiously used the technique himself in his Energy Environment, 2005 paper) I saw no reason to use it. It is, however, used by McIntyre and McKitrick (2005), so I needed to put it in there to examine the differences between our paper and theirs.

  19. Willis Eschenbach
    Posted Nov 6, 2006 at 4:47 AM | Permalink

    Martin, thanks for the reply. What technique was used regarding the smoothing at the endpoints of your study?

    w.

  20. Steve McIntyre
    Posted Nov 6, 2006 at 5:18 AM | Permalink

    #18. The extend.persist function was used in our study only to extend proxies to 1980 by persistence, emulating a procedure used in Mann et al. It’s not a technique that we endorse. (We noted that a surprising number of MBH98 proxies were so extended in MM03, but did not concern ourselves with this particular issue in our MM05 papers, contrary to one of Juckes’ claims.) Contrary to Martin’s statement here, extension by persistence is used over and over again in Juckes et al. Indeed, Juckes spends a considerable amount of energy arguing that these extensions don’t have any material impact on final reconstructions – an issue that was not in play in any ofd the 2005 papers.

    In our own smoothing by (2M+1) gaussian filters, we padded by the mean of the closing M values, as is easily observed by archived code. On no occasion did we pad a smooth by padding with M endpoint values.

    Juckes did not explain how he dealt with end point padding in his article. It is my surmise that he applied this technique in his article. But perhaps he did something else and, as Willis requested, can explain what he did.

  21. Hans Erren
    Posted Feb 2, 2007 at 5:23 AM | Permalink

    http://mitrie.badc.rl.ac.uk/

    Introduction
    MITRIE is funded by the Netherlands Environment Assessment Agency (RIVM) as part of the Netherlands `Scientific Assessment and Policy Analysis’ (WAB) programme. More information about this programme can be found here (in Dutch). http://www.klimaatportaal.nl/index.htm

    There has been considerable interest in paleoclimate reconstructions of temperature over the last millennium. A wide variety of techniques have been used, with sometimes obscure details. The interrelation among the techniques is unclear. This project will seek to achieve two distinct but related aims: firstly, to aid public understanding by providing a web site explaining the scientific basis and providing data and software; secondly, to contribute to the scientific debate by reviewing recent work and attempting to isolate the key differences between the different methodologies within a common framework.

    Proxy climate reconstructions draw on a wide variety of data sources. Reviews of some of the data sources can be found here. http://mitrie.badc.rl.ac.uk/short_reviews

    NEW:: Our work is now published in a Climate of the Past Discussion paper. Software used for this paper can be downloaded here, the data used is provide in the supplementary material.

    http://www.copernicus.org/EGU/cp/cpd/2/1001/cpd-2-1001.htm

    http://home.badc.rl.ac.uk/mjuckes/mitrie_files/software/mitrie_python.tar

    A comment on an error in McIntyre and McKitrick (2005) [Energy and Environment] here.

    http://home.badc.rl.ac.uk/mjuckes/mitrie_files/pdfs/comment_ee_figure2.pdf

    A comment on the disclosure of software and data can be found here.

    http://mitrie.badc.rl.ac.uk/disclosure_1

One Trackback

  1. […] (Juckes and the MITRIE Project) […]

Follow

Get every new post delivered to your Inbox.

Join 3,422 other followers

%d bloggers like this: