IPCC and the Law Dome Graphic

Apr 5, 2010 – 12:38 PM

Re-reading Climategate and AR4 Review Comments, I noticed an interesting discussion about handling the Law Dome O18 record – a series used in Mann and Jones (2003) and Jones and Mann (2004) with a very elevated MWP. Continue reading →

By Stephen McIntyre | Posted in climategate, IPCC | Tagged , , , , | Comments (44)

A Small FOI/EIR Success

Apr 4, 2010 – 10:07 AM

As CA readers, CRU, the Met Office and a couple of other UK institutions had more or less stonewalled David Holland’s FOI requests.

One of Holland’s particular interests is one that is perhaps a little appropriate/inappropriate for Easter Sunday – Caspar and the resurrected Jesus paper. Holland had tried for some years to determine exactly how IPCC had determined that Wahl and Ammann 2004 2005 2006 2007 met IPCC deadlines.

In a letter to Keith Briffa of April 1, 2008 (circulated by Briffa to various IPCC cadres), Holland had asked:

6. Why is their no mention of the fact that Mann et al. fails R2 verification in the important 15th century step? Who made the decision to reject comment 6-750 on the SOD by the reviewer for the Government of the United States of America who objected to the unpublished paper of Wahl and Ammann because as reviewed it did not include the table of verification data required for it be finally accepted for publication and which actually corroborated the criticism of McIntyre and McKitrick?

To which Briffa had answered (May 15, 2008):

6. Your question conflates a number of complex issues and makes statements with which I do not necessarily concur. However, in answer to the question of who made the decision to “reject comment 6-750 on the SOD”, this decision, as with all decisions related to specific reviewers’ comments, was made by multiple Chapter 6 authors and sanctioned by all of them.

More on this later. SOD Review Comment 6-750 had stated:

6-750 A 29:41 :42 The use of Wahl and Ammann (accepted) does not comply with WG1’s deadlines and all text based on this reference should be deleted. WG1’s rules require that all references be “published or in print” by December 16, 2005. Wahl and Ammann was “provisionally accepted” on that date, and not fully accepted until February 28, 2006, at which time no final preprint was available. Substantial changes were made in the paper between December 16, 2005 and February 28, 2006, including insertion of tables showing that the MBH98 reconstruction failed verification with r-squared statsistics, as had been reported by McIntyre and McKitrick in 2003. These tables were not available in the draft considered by WG1 when developing the second-order draft. [Govt. of United States of America (Reviewer’s comment ID #: 2023-415)]

To which Briffa as IPCC Author had answered:

See response to comment 6-1158.

SOD Comment 6-1158 had made a similar point to Comment 6-750:

6-1158 B 29:41 29:41 Wahl and Ammann 2006 did not meet several publication deadlines. Is it fair to use this study when other studies also not meeting publication deadlines were not used? It was not accepted by December 13-15. TSU did not have a preprint by late February. The version available for review was not the same as the accepted verion – in particular, the version made available omitted critical information that MBH98 failed cross-validation r2 and CE statistics. [Stephen McIntyre (Reviewer’s comment ID #: 309-119)]

To which Briffa as IPCC Author stated:

Rejected- the citation is allowed under current rules.

The Bergen Lead Authors’ meeting in late June 2006 appears to have prompted a decision by WG1 to change their deadlines. On July 3, 2006, the WG1 TSU sent an email to Lead Authors and Reviewers, notifying them that they could propose “additional” articles that met certain criteria as follows ( I received this email):

Dear Colleague,

Following the Government and Expert review of the Working Group I contribution to the IPCC Fourth Assessment Report, the attached guidelines are being provided to clarify how recent scientific literature related to review comments may be included in the final draft. Please feel free to distribute this information among your colleagues.

Thank you for your interest in the work of the IPCC.

Best regards,
WG1 TSU

The guidelines stated as follows;

Guidelines for inclusion of recent scientific literature in the Working Group I Fourth Assessment Report.

We are very grateful to the many reviewers of the second draft of the Working Group I contribution to the IPCC Fourth Assessment Report for suggestions received on issues of balance and citation of additional scientific literature. To ensure clarity and transparency in determining how such material might be included in the final Working Group I report, the following guidelines will be used by Lead Authors in considering such suggestions.

In preparing the final draft of the IPCC Working Group I report, Lead Authors may include scientific papers published in 2006 where, in their judgment, doing so would advance the goal of achieving a balance of scientific views in addressing reviewer comments. However, new issues beyond those covered in the second order draft will not be introduced at this stage in the preparation of the report.

Reviewers are invited to submit copies of additional papers that are either in-press or published in 2006, along with the chapter and section number1 to which this material could pertain, via email to ipcc-wg1@al.noaa.gov, not later than July 24, 2006. In the case of in-press papers a copy of the final acceptance letter from the journal is requested for our records. All submissions must be received by the TSU not later than July 24, 2006 and incomplete submissions can not be accepted.
1 see report outline at http://ipcc-wg1.ucar.edu/wg1/doc/wg1_outline.pdf

I submitted two documents in response to this notice: the Wegman Report and the NAS Report. My submission was acknowledged by the WG1 TSU.

Holland has been trying for a couple of years to verify Briffa’s statement that Wahl and Ammann 2004 2005 2006 2007 had met these revised rules – had “a copy of the final acceptance letter from the journal” been obtained for WG1 TSU records, for example – particularly given the explicit statement by WG1 TSU that “incomplete submissions can not be accepted”.

On November 28, almost immediately after Climategate, Holland re-visited this outstanding question. He had noticed that a July 26 email from WG1 TSU to Coordinating Lead Authors and a July 28 email from Overpeck to Briffa (both in 725. 1154353922.txt) referred to a spreadsheet containing a list of additional articles relevant to chapter 6, submitted under the new rules.

Holland’s new round of requests are online here. Here is the salient portion of his Nov 28, 2009 FOI request:

On 28 July 2006 at 6:32 PM UEA/CRU received an email from Jonathan
Overpeck, to which a spreadsheet was attached “listing: the
submitter, file name of the paper, its acceptance date, and the
chapter and section which the submitter feels is relevant.” These
were expert comments received pursuant to an email sent to
Reviewers on 5 July 2006, soliciting additional reviewers comments.

Please send me an electronic copy of the spreadsheet.

In December 2009, the UEA reverted saying that they would consider the request under EIR (Environmental Information Regulations) – this may be an interesting precedent since the decision between EIR and FOI has been disputed in a number of other cases. Holland’s view is that EIR is applicable; the institutions prefer to respond under FOI which has more exemptions. UEA:

In short, we will consider your request and provide information under EIR, not FOIA.

On January 26, 2010, the UEA refused on the interesting basis that the document was held by the police and they didn’t have a copy:

Your request for information received on 28 November 2009 for a spreadsheet sent as an attachment to an email of 28 July 2006 from Jonathan Overpeck has now been considered, and, upon consideration, it is, unfortunately, not possible to meet your request.

In accordance with Regulation 14 of the Environmental Information Regulations 2004 this letter acts as a Refusal Notice, and I am not obliged to supply this information and the reasons for exemption are as stated below:

Exception
Reason
Reg. 12(4)(a) – Information not held
The requested information was not held at the time of the request

Reg. 12(5)(b) – Disclosure would adversely affect a criminal enquiry

Information is held by the police in connection with a current investigation

We believe that Regulation 12(4)(a) applies to your request because the only location that this information was held on was on a backup server as the original information had been ‘deleted’ some years ago. Pursuant to an investigation carried out by the Norfolk Constabulary, the server upon which the requested information resided was taken from the University grounds on 24 November and now resides with the police
forces conducting an investigation into a possible criminal offence. Therefore, at the time of the request, we did not hold the requested information, and we currently have no access to either the server or any of the material on it.

Regulation 12(5)(b) also applies to the data requested because the requested data is part of a larger set of data that is the subject of an ongoing police investigation. Such information is now under an embargo by the investigating forces and any disclosure would adversely affect the ability of that public authority to conduct the criminal enquiry.

Regulation 12(1)(b) mandates that we consider the public interest in any decision to release or refuse information under Regulation 12(4) and Reg. 12(5). In the case of Regulation 12(4)(a), there really is no consideration of the public interest as we simply did not, at the time of the request, nor do not now, have the requested information.

Turning to Regulation 12(5)(b), we feel that there is a strong public
interest in protecting the ability of police forces to investigate criminal offences and that we should abide by established procedures by which evidence is gathered and used.

Overall, we therefore believe that the public interest in non-disclosure of the information outweighs that in favour.

I should note that whilst we believe that it is possible that this material is already in the public domain due to the illegal penetration and use of University computing facilities, this does not relieve us of our obligations to address any request on its merits under the Regulations.

The above comment is interesting in considering claims that people hadn’t “deleted” emails and documents.

A few days ago (March 26, 2010), the UEA notified Holland that they had located the requested information in another location and sent him a copy. UEA FOI officer Palmer;

Further to my letter of 26 January 2010 responding to your request of 28 November 2009 for a spreadsheet sent as an attachment to an email of 28 July 2006 from Jonathan Overpeck, I am writing to advise you of recent developments in this matter.

By my letter of 26 January, you were informed in good faith that the requested information was not held at the time of the request. It was the University’s belief that the only location that this information was held on was on a backup server and that the original information had been ‘deleted’ some years ago. I further advised that, pursuant to an investigation carried out by the Norfolk Constabulary, the server upon
which the requested information is held was taken from the University’s grounds on 24 November 2009 and is being held by the police forces conducting an investigation into a possible criminal offence.

It has come to the University’s attention that the University does hold the spreadsheet in question otherwise than on the backup server. We only made this discovery very recently.

Accordingly, I enclose with this letter a copy of the spreadsheet in fulfilment of your request of 26 January 2010.

No excuses this time that the provision of the spreadsheet would cause the demise of international relations between the UK and the rest of the known universe. The spreadsheet consisted of four articles:

Submitted by Date Received File Name Chapter/Section Journal Date Accepted / Published
Hermann Held 17-Jul-06 SchneiderVonDemling_etal.pdf 6.4.1.3 Geophys. Res. Lett. 07-Jun-06
Gabi Hegerl 20-Jul-06 Hegerl_etal.pdf 6.? J. of Climate 12-Jun-06
Ellen Mosley-Thompson 23-Jul-06 Thompson_etal.pdf 6.5.2 PNAS 11-Jul-06
David Schnieder 24-Jul-06 Schneider_etal.pdf 6.6 Geophys. Res. Lett. 30-Jun-06

The Thompson et al (PNAS 2006) article prompted some discussion in the Climategate letters (indeed, the WG1 TSU letter is a trailer to this discussion.)

Interestingly, neither the NAS report nor the Wegman Report were forwarded from the WG1 TSU to the Chapter 6 Authors, though both had been properly submitted.

Needless to say, Wahl and Ammann 2004 2005 2006 2007 wasn’t on the list.

By Stephen McIntyre | Posted in climategate, Wahl and Ammann | Tagged , | Comments (44)

“Keith Should Say…”

Apr 2, 2010 – 5:31 PM

I doubt that many inquiries are provided with documents in which the subject of the inquiry not only asks subordinates to delete documents subject to an FOI request, but also states in writing that he expects a subordinate to give an untrue statement to an official. And even rarer that an inquiry would not clarify the subject’s state of mind and knowledge when the expectation was issued.

Jones’ request to delete emails is notorious. His email saying that “Keith should” make an untrue statement to university FOI official Palmer has thus far not been analysed, but is obviously something that the Parliamentary Inquiry should have considered before purporting to blame UEA administration for CRU obstruction of FOI requests. Continue reading →

By Stephen McIntyre | Posted in climategate, FOIA | Tagged , , | Comments (70)

CNN Tomorrow

Apr 1, 2010 – 3:37 PM

John Roberts’ show at 6:20 am to talk about the Parliamentary Committee. Roberts visited CRU in the heat of Climategate. They said that I would be appearing with Michael Mann – anyone heard of him? In the re-confirm, they said that I’d be appearing with Mike MacCracken.

By Stephen McIntyre | Posted in Uncategorized | Comments (31)

Code: The Trick

Apr 1, 2010 – 2:29 PM


load MBHsmooths1.txt % http://www.climateaudit.info/wp-content/uploads/2009/11/mbhsmooths1.txt
% MBHsmooths1 = [ Year AnnualRecon Instrumental MB98SmoothTrick MBH98SmoothNoTrick MBH99SmoothTrick MBH99SmoothNoTrick ]

[B98,A98]=butter(10,2/50); % '50 year lowpass'

in98=MBHsmooths1(401:981,2); % Annual Recon MBH98,
%source:
% http://www.nature.com/nature/journal/v430/n6995/extref/FigureData/nhmean.txt ,
% col 2

Trick=MBHsmooths1(982:996,3); % pad with instrumental (1981..1995) ..
% source:
% http://www.nature.com/nature/journal/v430/n6995/extref/FigureData/nhmean.txt
% col 3

out98t=flipud(filter(B98,A98,flipud([filter(B98,A98,[in98;Trick;zeros(100,1)])]))); %..then smooth
% Note that filter.m initializes with zeros
out98t=out98t(1:596);
out98tr=out98t(26:576); % smoothed reconstruction with the trick

% no trick
out98nt=flipud(filter(B98,A98,flipud([filter(B98,A98,[in98;zeros(115,1)])]))); %..then smooth
out98nt=out98nt(1:596);
out98ntr=out98nt(26:576); % smoothed reconstruction without the trick

% read image, http://uc00.files.wordpress.com/2010/05/mbh985b.png
A=imread('mbh985b.png'); % from MBH98, copied for scientific purposes

XS=1.1691; % x-scale
YS=-365.7848; % y-scale
XB=-1.5749e+003; % x-bias
YB=281.8316; % y-bias
Year=(1400:1995)';

close all
image(A)

hold on
plot(Year*XS+XB,out98t*YS+YB,'r')
plot(Year*XS+XB,out98nt*YS+YB,'g')

% Note that at the begin red and green lines overlap

% MBH98: Mann, M. E., R. S. Bradley, and M. K. Hughes (1998), Global-scale
% temperature patterns and climate forcing over the past six centuries,
% Nature, 392, 779– 787.

% (C) UC 2010

By uc00 | Posted in Uncategorized | Tagged , , | Comments (5)

Der Spiegel

Apr 1, 2010 – 12:16 PM

Long article in Der Spiegel here. (h/t P Gosselin). The proprietor of Climate Audit is discussed. Some interesting new comments from Peter Webster.

By Stephen McIntyre | Posted in Uncategorized | Comments (52)

Tricking the Committee

Mar 31, 2010 – 1:30 AM

In my submission to the Parliamentary Committee, I observed that the “trick” wasn’t “clever” – it was the deletion of inconvenient data.

The IPCC “trick” was not a “clever” mathematical method – it was merely the deletion of inconvenient data after 1960. Post-1960 values were even deleted in the reconstruction archived version at NOAA.

I provided an explanation with a graphic – which wasn’t quoted or cited. They cited a less precise submission by Peter Taylor on this point.

They noted that “UEA interpreted the use of the word “trick” differently” citing the following portion of the UEA submission [discussing the trick email describing the WMO 1999 graphic, not the TAR graphic – June 13, 2010]:

as for the (now notorious) word ‘trick’, so deeply appealing to the media, this has been richly misinterpreted and quoted out of context. It was used in an informal email, discussing the difficulties of statistical presentation. It does not mean a ‘ruse’ or method of deception. In context it is obvious that it is used in the informal sense of ‘the best way of doing something’. In this case it was ‘the trick or knack’ of constructing a statistical illustration which would combine the most reliable proxy and instrumental evidence of temperature trends.

Contrary to UEA’s claims, there is no valid statistical procedure supporting the substitution of tree ring proxy data going the wrong with instrumental temperature data to create a false rhetorical impression of the coherence of the proxy data. Indeed, as I observed in my submission, Mann himself had condemned the merging of instrumental and proxy data as follows:

No researchers in this field have ever, to our knowledge, “grafted the thermometer record onto” any reconstruction. It is somewhat disappointing to find this specious claim (which we usually
find originating from industry-funded climate disinformation websites) appearing in this forum [realclimate].

Unfortunately, NIgel Lawson – who, to my knowledge, does not have in-depth knowledge of Climate Audit issues – was invited by the Committee to testify on Climate Audit issues and made incorrect and exceedingly ill-judged comments on the topic, comments that were seized upon by the Committee as follows:

These [ UEA]interpretations of the colloquial meaning of “trick” have been accepted by even the staunchest of critics:

Lord Lawson of Blaby: The sinister thing is not the word ‘trick’. In their [UEA’s] own evidence they say that what they mean by ‘trick’ is the best way of doing something.
Chairman: You accept that?
Lord Lawson of Blaby: I accept that.

From these ill-judged comments, the Committee concluded:

[The trick] appears to be a colloquialism for a “neat” method of handling data.

This is absurd. The trick was not a “neat” way of handling data, nor a recognized form of statistical analysis. The trick was a clever way of tricking the readers of the IPCC 2001 graphic into receiving a false rhetorical impression of the coherency of proxies – a point understood at the beginning by Jon Stewart of the Daily Show, but now misunderstood due to continued disinformation.

The Committee went on to consider Lawson’s testimony on “hide the decline” as follows:

61. Lord Lawson did, however, describe CRU’s treatment of the data as “reprehensible”, because, in his view, Professor Jones deliberately hid data that demonstrated a decline in temperatures.

62. The data that he believed to be “hidden” are a set of tree ring data that disagree with other data sources regarding temperature trends. Lord Lawson said: “when the proxy series […] departed from the measured temperature series, a normal person will say maybe that means the proxy series is not all that reliable”. In that context he made two specific claims:
• that the tree ring data were flawed because “for a long period before 1421 they relied on one single pine tree”; and
• that the divergence problem was not just for data after the 1960s, “it is not a good fit in the latter half of the nineteenth century either”.

Again, these comments by Lawson were ill-considered. I have no idea what he had in mind in his comment about the lone pine tree nor what his 19th century comment was about. Jones was able to rebut each of Lawson’s comments and the Committee accepted Jones’ rebuttal of Lawson’s comments.

On these points, the Committee did not consider or rebut any of my written points, preferring to deal with easier targets.

The Committee cited the UEA denial that they had ever sought to “hide the decline”:

CRU never sought to disguise this specific type of tree-ring “decline or divergence”. On the contrary, CRU has published a number of pioneering articles that illustrate, suggest reasons for, and discuss the implications of this interesting phenomenon.

Again, as so often, you have to watch the pea under the thimble. It is true that the decline was reported in publications by CRU authors – but this does not mean that CRU never sought to disguise the “decline or divergence”. As I pointed out in my submission (and others also), CRU authors – in their capacity as IPCC contributors – sought to disguise the decline in the influential IPCC spaghetti graph (and elsewhere, through such measures as not archiving post-1960 reconstruction values.)

Given that the IPCC report is the most influential representation of the data, this is hardly a minor incident.

The Committee reached the following conclusion on this matter:

66. Critics of CRU have suggested that Professor Jones’s use of the words “hide the decline” is evidence that he was part of a conspiracy to hide evidence that did not fit his view that recent global warming is predominantly caused by human activity. That he has published papers—including a paper in Nature—dealing with this aspect of the science clearly refutes this allegation. In our view, it was shorthand for the practice of discarding data known to be erroneous. We expect that this is a matter the Scientific Appraisal Panel will address.

Here the Committee either failed to watch the pea under the thimble or is itself moving the pea. Once again, the fact that the decline is discussed in a Nature paper does not justify the deletion of the inconvenient data in the IPCC spaghetti graph in order to provide the false rhetorical consistency that IPCC was seeking. The issues are entirely separate and the Committee should have been able to discern this.

In addition, their suggestion that Jones and others were doing nothing more than “discarding data known to be erroneous” is simply absurd. There was no testimony to the Committee (nor has it ever been suggested) that the tree ring data was measured incorrectly or that the data was “erroneous” – the data is what it is. The tree ring data goes down instead of up – but that doesn’t make it “erroneous”. It only means that the data is a bad proxy – something that was concealed from IPCC readers.

It is discouraging to read such bilge.

By Stephen McIntyre | Posted in climategate, Uncategorized | Tagged , , | Comments (210)

Stringer’s Dissents – a Split Decision

Mar 30, 2010 – 8:51 PM

MP Graham Stringer, widely recognized for insightful questioning of Jones at the Parliamentary Inquiry, voted against acceptance of the Parliamentary Inquiry report, against the inclusion of the Summary and against a number of individual items. In every case, Stringer was opposed by three MPs: Tim Boswell, Evan Harris and Brian Iddon.

Individual dissents were as follows.

Stringer voted against paragraph 47 which stated the following:

47. This has substance if one considers CRU’s work in isolation. But science is more than individual researchers or research groups. One should put research in context and ask the question: what would one hope to find by double checking the processing of the raw data? If this were the only dataset in existence, and Professor Jones’s team had been the only team in the world to analyse it, then it might make sense to double check independently the processing of the raw data and the methods. But there are other datasets and other analyses that have been carried out as Professor Jones explained:

There are two groups in America that we [CRU] compare with and there are also two additional groups, one in Russia and one in Japan, that also produce similar records to ourselves and they all show pretty much the same sort of course of instrumental temperature change since the nineteenth century compared to today.67

[…] we are all working independently so we may be using a lot of common data but the way of going from the raw data to a derived product of gridded temperatures and then the average for the hemisphere and the globe is totally independent between the different groups.68

He voted against paragraph 51 which read as follows:

51. Even if the data that CRU used were not publicly available—which they mostly are—or the methods not published—which they have been—its published results would still be credible: the results from CRU agree with those drawn from other international data sets; in other words, the analyses have been repeated and the conclusions have been verified.

He proposed an amendment to paragraph 66, to leave out from the beginning to “We” in line 6 and insert “We have not taken enough evidence on this matter to come to a final conclusion”.—(Graham Stringer.) Paragraph 66 read as follows:

66. Critics of CRU have suggested that Professor Jones’s use of the words “hide the decline” is evidence that he was part of a conspiracy to hide evidence that did not fit his view that recent global warming is predominantly caused by human activity. That he has published papers—including a paper in Nature—dealing with this aspect of the science clearly refutes this allegation. In our view, it was shorthand for the practice of discarding data known to be erroneous. We expect that this is a matter the Scientific Appraisal Panel will address.

He proposed an amendment to paragraph 132 as follows: to leave out from “science” in line 6 to the end and add “it would be necessary for the whole of climate science to increase its transparency and improve its scientific methodology”. Paragraph 132 reads as follows:

132. Reputation does not, however, rest solely on the quality of work as it should. It also depends on perception. It is self-evident that the disclosure of CRU e-mails has damaged the reputation of UK climate science and, as views on global warming have become polarised, any deviation from the highest scientific standards will be pounced on. As we explained in chapter 2, the practices and methods of climate science are a key issue. If the practices of CRU are found to be in line with the rest of climate science, the question would arise whether climate science methods of operation need to change. In this event we would recommend that the scientific community should consider changing those practices to ensure greater transparency. it would be necessary for the whole of climate science to increase its transparency and improve its scientific methodology

By Stephen McIntyre | Posted in climategate | Tagged , , | Comments (30)

“The Selling of the Police”

Mar 30, 2010 – 3:06 PM

Margaret Beare of York University in Toronto has written for many years on the “selling” of the police. Continue reading →

By Stephen McIntyre | Posted in Uncategorized | Comments (11)

Slingo: Wing and Re-Frame

Mar 30, 2010 – 1:52 PM

I doubt that Julia Slingo of the Met Office knows much about the Hockey Stick controversy based on actual personal knowledge of the issues. Nonetheless, she saw fit to weigh in on the controversy at the UK Parliamentary Committee hearings. A few days after the hearings, I emailed her asking her for citations/references for two highly questionable assertions that she made to the Parliamentary Committee. A couple of days ago, she replied. Instead of providing the requested citations/references, her reply – in all-too-familiar re-framing climate scientist style – failed to answer the question. Rathering than answering the question, she pontificated about irrelevant issues, pontifications that were themselves inaccurate. Continue reading →

By Stephen McIntyre | Posted in climategate | Tagged | Comments (59)