Climate Audit

As I reported here, Juckes et al 2006 stated that the source code for MM05 (GRL) was not "available". On Oct 29, I objected to Juckes and coauthors about this false claim. It turned out that Juckes had been unable to locate the relevant code and had been remarkably ineffective in his efforts to locate the code. Let’s look at how he’s tried to justify this.

On Oct 29 (Eastern 8.46 am; 13 46 GMT) , I emailed Juckes and coauthors as follows:

As you either know or should know, the code used in McIntyre and McKitrick 2005 is available at the Supplementary Information to the article at ftp://ftp.agu.org/apend/gl/2004GL021750, as is made clear in the article itself. (The code for MM05 (EE) and MM03 (EE) are at http://www.climate2003.com/scripts/MM05_EE and http://www.climate2003.com/scripts/MM03 respectively). The Wegman Report specifically noted that they verified availability of our source code at the time of their report last summer. Previously, both Huybers and Wahl and Ammann had examined the source code, neither of whom required any assistance from me. Huybers annotated the code in his Supplementary Information.

On Oct 31 (Eastern 955 am 14 55 GMT), Juckes replied:

Thank you for the correction: The citation at that point in our article should be to the Energy and Environment article (which we refer to as MM2005c) and not MM2005 as it stands. I’m sorry for this mistake. I had an email from you at the start of this year saying the code used by MM2005c was not then available (and that you would inform me when it was), but since the situation has been remedied I will correct the text accordingly.I was, I think, also thrown off the track by your web-site which says "The computer script used to generate the figures and statistics in the E&E here will be located here [in a couple of days]." It might help to avoid further confusion if the page you refer to below was linked to from your site.

However, I note that the code available at http://www.climate2003.com/scripts/MM05_EE requires the function NHbeta in order to compute the reconstructions for figure 1 of your 2005 Energy and Environment paper. This function is not defined in any of the scripts I could find at http://www.climate2003.com/scripts. Can you tell me where to find this function?

On Oct 31 ( yahoo 4.33 pm; 16 33 GMT) , I uploaded the file "ee2005.functions.txt" with the requested function NHbeta to http://www.climate2003.com/scripts/MM05b.

On Oct 31 (yahoo 11.54 pm; 23 54 GMT), I uploaded an edited backup script producing figures and statistics "backup.txt", which included a commentary on MM05 Figure 2, about which I’d heard that Juckes had concerns. I’ll post some more on MM05b Figure 2, but, in brief, the observation about the "perverse" flipping property of the Mannian PC method was correct.

Martin’s Big Day – November 2
Here is a riveting episode of 24.

05 29 GMT (Eastern 12.29 am) – I notified Juckes as follows that the function NHbeta had been uploaded and went to bed.

Your statement, as it stands, is untrue about source code for either article – MM05(GRL) or MM05(EE). Source code for MM05 (GRL) was archived in January 2005. Source code for MM05 (EE) was archived on March 1, 2006, shortly after your original inquiry, but long before the submission of your article to Climates of the Past. If you propose to simply substitute an untrue statement about MM05(EE) code for an untrue statement about MM05(GRL) code, that is unacceptable to me. Further, given that your allegation occurs in the context of a discussion of principal components — an issue treated in MM05 (GRL) to which your text currently refers, your proposed substitution is opportunistic as well as untrue.

In response to your query about the function NHbeta, I have uploaded this function. I have also made a couple of slight edits to the backup script so that references to local directories are replaced by references to online directories to enhance the script so that it is as turnkey as possible. (I have continued to archive the original scripts.) In March 2006, I inadvertently uploaded a file containing the function NHgamma, a slight modification of NHbeta. I apologize for this oversight. However, the differences between NHgamma and NHbeta are immaterial to the issues at hand, as the differences only pertain to the handling of undocumented MBH re-scaling and the relevant methods are documented in NHgamma. The modifications were made in response to new information on MBH methodology, which became available during 2005.

Your paper purports to carry out an intercomparison and evaluation of millennial paleoclimate reconstructions. To our knowledge, ours were the first studies in the field which provided source code so that results could be directly tracked and, to date, are the only studies in this field in which figures and statistical results can be replicated from an online script. (Wahl and Ammann, following our example, have provided considerable documentation, but fall well short of providing an equivalent script that generates their figures and statistics.) None of your coauthors have archived code for any of their articles; in some cases, they have even failed to archive their data as used or to provide accurate data citations for their articles and have refused or failed to provide this information upon request.

If you wish to intercompare and evaluate availability of source code for paleoclimate studies, I strongly welcome the effort. However, any attempt to disparage our efforts in providing source code through untrue or misleading statements or any caviling about these efforts while omitting to discuss the record of your coauthors can only be interpreted as an intentional effort by you and your coauthors to provide a misleading representation of the research record.

By only objecting at this time to this particular allegation, please do not consider that this constitutes acquiescence with any other statements in the article. I have made an immediate objection to this particular allegation because, given our emphasis on code availability, your false claims are both disparaging and defamatory and well outside the bounds of academic give-and-take. I intend to respond to scientific and statistical issues in other venues over the next few weeks.

I re-iterate that I am not prepared to acquiesce in your presentation of untrue and defamatory statements about source code availability or to acquiesce in your presentation of a misleading representation of the research record on availability of paleoclimate source code. I re-iterate my previous notice requiring that you immediately amend your submission so that it contains no such untrue and defamatory statements.

09 48 GMT (Eastern 4.48 am) Juckes sent me an email reply, twitting me about supposed "uncertainty" about what we had done and saying of the secondary website:

I notice that your website still contains the statement: "The computer script used to generate the figures and statistics in the E&E here will be located here [in a couple of days]" which may lead many people to think that the code is not yet available.I appreciate that you have disclosed more code than is normal in this and other fields of research. Our paper is an intercomparison of reconstructions: the statement about unavailability of your code was only there because there is uncertainty about the interpretation of your results which would be easily resolved if the code had been available. As many people would assume, on the basis of your many writings on this issue, that the code was available I thought it was worth pointing out that it was not (though we could of course have found it if we had happened upon the web page you have now disclosed to us).

14 24 GMT (2.24 pm yahoo) – I uploaded a new Index page for climate2003.com removing the stale index page and inserting a link to the MM05b SI.

15 17 GMT (9.17 am blog time) – I posted up new version of Figure 2 at climateaudit here
.

16 04 (Juckes blog time). Juckes archived the file 2004GL021705-NOAMER.txt downloaded from AGU (originally archived by us in January 2005.)

16 04 (Juckes time) Juckes archived ee2005.functions.oct.txt.

16 07 (Juckes time) Juckes made a directory for Mc-Mc files.

16 13 (Juckes time) Juckes saved Fig 2p.txt

16 14 (Juckes time) Juckes saved Fig 2.txt

16.14 (Juckes time) Juckes made a zip file of these above four files.

16 15 Juckes made a pdf directory.

16:15 (Juckes time) Juckes posted up Comment on Figure 2 here , asserting an "Error" in MM05. I guess that he was quivering with excitement. In this Comment, which I will discuss separately, he made no reference to the re-stated Figure 2 posted earlier that day at climateaudit – and explicitly documented in the revised script of Oct 31, 2006 to which I’d directed him.

17 40 (CPD time – presumably GMT) . Juckes posted up his "Correction" at CPD in which he stated:

At the present time this page is not linked to from the main site (www.climate2003.com), which contains the statement that “The computer script used to generate the figures and statistics in the E&E here will be located here [in a couple of days]” Our intended comment (that the code used by MM2005c was not available) was based on the above statement and on an email from Stephen McIntyre to us saying that he would forward the code when it became available. He first informed us of its availability after the publication of our manuscriptIntro page modified at MITIRE to link to new docs.

The Next Day Nov 3, 2006

10 48 GMT (4.34 am blog time) Juckes wrote to CA as follows:

Firstly I’d like to apologise for an error in our manuscript: On page 19, line 6, the manuscript states: “The code used by MM2005 is not, at the time of writing, available”: This comment should have referred to the code used by MM2005c (their Energy and Environment paper) “¢’‚¬? the code used by MM2005 (their GRL paper) was made available at time of publication.We apologise to McIntyre and McKitrick for suggesting that the code for MM2005 was unavailable when it was in fact the code for MM2005c which we had not been able to obtain, and note that they have made efforts to provide code for this and other publications which go beyond the norm in this field.

Further details are on the Climate of the Past Discussion site, where anyone is welcome to post a comment.

Stephen McIntyre has now made the code for their Energy and Environment paper available to us (though his http://www.climate2003.com site still declares that it is unavailable). Interestingly, given the strong emphasis of McIntyre and McKitrick’s 2005 Energy and Environment paper on the importance of centering data for principal component calculations this step appears to have been omitted for figure 2 of their paper, which displays a principal component of uncentred data. Further details are available at mitrie.badc.rl.ac.uk.

Concerning the code, this is provided in the spirit of transparency. I fully agree with the comment above that python should not be judged on the basis of this code.

13 20 GMT (7.20 am blog time) Juckes writes in to CA as follows:

Response to comment 32. The lower two curves in the plot are calculated using the MBH method, centred on the calibration period, which is described in the caption as “decentred”, i.e. not centred on the complet data record. The upper two curves, which are described in the caption as “Above: PC1 using centred calculations” are calculated without centering of any kind. The discussion on page 77 of the paper makes it clear that the authors intend to show the contrast between the MBH PC and a centred PC, but what is plotted in the figure is a comparison between the MBH PC and a PC calculated with no centering.

Response to comment 33: I have never advocated that code should always be disclosed. For the record, I asked for the code for the McIntyre and McKitrick 2005 paper in December 2005, and it was not provided. I checked Stephen McIntyre’s website prior to submission and found a line clearly saying that the code was not yet available. It has now been provided and, as I acknowledge, this goes beyomd what is normal in the field.

13 35 GMT 7.35 am blog time. I wrote at CA:

I will revert on a number of points here more fully. I wish to note for the record that the stale index webpage at http://www.climate2003.com was updated yesterday (November 2, 2006) and that a direct link to the Supplementary Information for MM05 was placed on the index page. This change was made promptly after the matter was brought to my attention. The date of the change is recorded at the website and, if Juckes wishes confirmation, I will provide an access password to coauthor Zorita to confirm this. I presume that Juckes either did not double-check the webpage when he submitted his correction or possibly he may have inadvertently used a cached version.

Second, on the issue of EE Figure 2, when the matter was brought to my attention, I edited the backup script on Oct 31, 2006, including a commentary and a script to produce a re-stated version of Figure 2 using prcomp. I posted this re-stated version at climateaudit on Nov 2 here . Juckes discusses Figure 2, but failed to discuss the comments as of Oct 31, 2006.

14 24 GMT (8.24 am blog time) Juckes replied:

Hello Stephen,As you are no doubt aware, my update to the Climate of the Past Discussion site was posted yesterday, Nov 2nd, not today. Concerning your revised figure 2, would you confirm that you still have a minus sign multiplying the centred PC, and that without this minus sign the effect of the adjustment to the non-BCP is to decrease the PC in the first 50 years, as it does for the MBH PC?

1457 GMT (8.57 am blog time) Juckes again:

See http://www.cosis.net/members/journals/df/article.php?PHPSESSID=fb8156e80bb4fef23d9f9e1c22527f86&a_id=4661 for further details of the correction to our paper. I haven’t responded on this page previously because Stephen McIntyre only drew my attention to the related discussion on http://www.climateaudit.org/?p=886.Regarding comment 83 above, the centred PC would also show what Stephen McIntyre calls `this perverse property of the Mann method’ if Stephen didn’t have a “-” in his code inverting the centred PC. I am sure this is an oversight.

1530 GMT (9.30 am blog) Juckes again:

The sign of the Principal Component is arbitrary, I only mentioned it because you appear to attach importance to the direction of the perturbation.As regards the difference between your centred PC and the Mann et al. PC, would you agree that this is partly due to the fact that you have not normalised the data, so that proxy series with high variance contribute disproportionately to your PC? If the data is normalised (by setting cor=TRUE in the call to princomp) the leading PC does reflect the anomalous growth in the 20th century, though it is still significantly different from the Mann et al. PC. Wouldn’t you agree that using normalised data is appropriate here?

As far as I can tell from your modified site, it was updated after I posted the comment yesterday, but if I’ve got that wrong, and the check I made on your site yesterday got information from a cache for some reason, please accpet my apologies.

Just a brief comment in response to 41: the paper is on a discussion site, the idea is that the site can act as a forum for discussion (hence the name). This is slightly different from traditional printed publications.

15 49 (949 am blog time) Juckes:

So have you contacted the editor of Energy and Environment about the mistake in your published paper

Juckes seems to be absolutely frothing with excitement here.

17 40 (CPD time) Juckes wrote a "Clarification" at CPD as follows:

I’ve been asked to clarify that “‹Å“at the present time’ in the AC S516 refers to the time I was writing the document, not when it was posted a few hours later — by which time the site had been amended to reflect the availability of McIntyre and McKitrick’s code. Detailed discussion on this and related issues can be found at
http://www.climateaudit.org/?p=880 and http://www.climateaudit.org/?p=886

Oddly enough, we have a rather nice little time line of Juckes’ activities on November 2. The webpage was amended at 14 24 GMT and the Correction was online at 17 40 GMT. update – Martin Juckes suggests that the surmise that CPD is GMT is an assumption; at present, the submission is timed only to CPD time, which appears to be GMT time, but might be GMT plus one hour.) So the amendment was over 3 hours (update – perhaps 2 hours) before the submission of AC S516. We have direct evidence of Juckes working on these matters between 16 04 and 16 15 and then submitting the "correction" at 17 40 (perhaps 16 40). My personal belief (although the evidence is merely circumstantial) is that Juckes wrote the Correction after 14 24 and that the amendment had already taken place when he "was writing the document".

Juckes could easily have said that he didn’t think about re-checking the webpage at the time that he was writing the article and apologized. That would have been the simplest course of action. Now after already making a couple of untrue statements about this matter, he’s made one more claim that seems very suspicious to me.