Phil Jones’ written answers to the Muir Russell panel shed interesting light on the insularity of IPCC authors, who see nothing odd about a system in which reviewers do not see either author responses to their review comments or the comments of other reviewers until long after the release of the final document. Jones’ comments were made in connection with questions from Boulton about Jones’ threat to keep McKitrick and Michaels 2004 out of IPCC and McKtrick’s allegation that Jones and other chapter 2 authors, having grudgingly agreed to refer to McKtrick and Michaels 2004, had “fabricated” IPCC’s editorial comment that its results had no statistical significance. This topic is revisited in AR5 (First Draft) where IPCC (with surprising candour) admitted that there was no “explicit” basis for the disparaging claim in AR4.
Jones was interviewed by the Muir Russell panel (actually just Boulton and Peter Clarke) on April 9, 2010.
On April 15, Boulton sent follow-up questions to Jones about his handling of McKitrick and Michaels 2004, an issue that had attracted notoriety as a result of IPCC Coordinating Lead Author Jones’ email saying:
The other paper by MM [McKitrick and Michaels] is just garbage. […] I can’t see either of these papers being in the next IPCC report. Kevin [Trenberth] and I will keep them out somehow — even if we have to redefine what the peer-review literature is!
Despite Muir Russell’s promises at his opening press conference and despite requests
from the Parliamentary Science and Technology Committee, for the most part, the Muir Russell panel failed to create a useful record of questions and answers. Boulton’s questions about McKitrick and Michaels 2004 are a rare exception.
In apparent compliance with Jones’ 2004 threat, the chapter of which Jones and Trenberth were Coordinating Lead Authors contained no mention of McKitrick and Michaels 2004 in the two drafts sent to reviewers (First Order Draft and Second Order Draft). In the final AR4 report, IPCC grudgingly mentioned McKitrick and Michaels 2004 (and the related de Laat and Maurellis 2006), adding the adverse editorial assertion that they ceased to have “statistical significance” when atmospheric circulation was considered – a comment which, according to McKitrick’s strongly worded submission to Muir Russell, was both untrue and which had no support in the peer reviewed literature that IPCC was supposed to draw on.
Boulton’s follow-up questions on April 15, 2010 asked Jones about the basis for the IPCC claim that the McKitrick and Michaels 2004 had no “statistical significance” as follows:
When the final IPCC FAR was published in May 2007, it included a new paragraph in Chapter 3, on page 244, that referred to the McKitrick and Michaels (2004) and De Laat and Maurellis (2006) papers, and that had not been included in either of the drafts shown to reviewers. It is assumed that this was either written by you, or in consultation with Trenberth, but in any case, the two of you, as Coordinating Lead Authors, bear responsibility for its inclusion. It reads:
“McKitrick and Michaels (2004) and De Laat and Maurellis (2006) attempted to demonstrate that geographical patterns of warming trends over land are strongly correlated with geographical patterns of industrial and socioeconomic development, implying that urbanisation and related land surface changes have caused much of the observed warming. However, the locations of greatest socioeconomic development are also those that have been most warmed by atmospheric circulation changes (Sections 3.2.2.7 and 3.6.4), which exhibit largescale coherence. Hence, the correlation of warming with industrial and socioeconomic development ceases to be statistically significant (highlighting added). In addition, observed warming has been, and transient greenhouse-induced warming is expected to be, greater over land than over the oceans (Chapter 10), owing to the smaller thermal capacity of the land”.
Q1: What is the justification for what appears as an ad hoc conclusion not based on published research that summarily dismisses an argument that is based on peer-reviewed research?
Q2: Why were these conclusions not shown to or discussed with expert reviewers during the IPCC Report preparation?
Q3: The references to sections 3.2.2.7 and 3.6.4 of the IPCC Report are misleading since neither section presents evidence that warming due to atmospheric
circulation changes occurs in the regions of greatest socioeconomic
development. Neither section even mentions industrialization, socioeconomic
development, urbanization or any related term. How can they therefore be
used to justify the stance of the above quotation?Q4: No justification is given for the claim of statistical insignificance, which has a precise meaning. Do you have a p value that justifies this statement, and if not,
what does it mean?
I will not discuss all the answers in this note and urge interested readers to consult the original here.
Let me start this quick review with the fourth question of Issue 2, since “statistical significance” is a technical term and IPCC made very specific assertions on this point.
As McKitrick had alleged, Jones did not have a p-value that justified his claim, but argued that “there is no need to calculate a p value for a statement that is based on the laws of physics”.
The pattern of atmospheric-circulation-related warming appears similar to the geographical distribution of socioeconomic development. Such similarity makes it impossible to use purely statistical methods to ascribe patterns of warming trends to patterns of socioeconomic development. It remains possible, however, to ascribe patterns of warming trends to atmospheric circulation because its influence is in accord with the laws of physics and can be detected in day-to-day weather variations, on which timescales socioeconomic trends are infinitesimal. As stated, it is essential to extract the known and understood influences first and then look at the residuals. There is no need to calculate a p value for a statement that is based on the laws of physics.
The latter statement surely raises epistemological issues on which Phil Jones hardly stands as an authority. (For example, I am unaware of any publications by Jones in epistemological literature.) But even if Jones were a qualified epistemological authority (which he isn’t), the statement in question was not “based on the laws of physics”. The IPCC statement in question was “the correlation of warming with industrial and socioeconomic development ceases to be statistically significant” – this statement may be based on an argument from physics, but it is a statistical statement and was not itself based on the laws of physics. Boulton should not have accepted such flannel.
In response to the related first question, Jones denied that the IPCC assertion about the lack of statistical significance was “ad hoc”, but, instead of evidence, merely presented a whinging complaint about too many academic papers:
The fact that MM2004 is in the peer-review literature does not mean it is good science. There are examples of poor science across all areas of science in the peer-review literature. Occasionally scientists submit comments on poor or incorrect papers, but this sadly is something of a rarity. With the plethora of journals it is becoming harder and harder to read and respond to all the literature. One could make a full time job of publishing criticisms of poor or incorrect papers.
Any competent inquiry would have seen through Jones’ flannel, but not Muir Russell. Although Boulton asked supplementary questions, none addressed these issues. Nor did the final report contain anything but flannel on the topic.
By the way, there’s an interesting development on this front in IPCC AR5 (First Draft) which admitted (with surprising candour) that AR4 “provided no explicit evidence” for the claim that the McKitrick and Michaels 2004 (and de Laat and Maurellis 2006) results had no statistical significance as follows (more on this on another occasion):
McKitrick and Michaels (2004) and de Laat and Maurellis (2006) analysed surface air temperature trend fields and assessed potential for biases in terms of national socioeconomic and geographical indicators. Both studies concluded that urbanisation and related land surface changes have caused much of the observed warming. According to the AR4, the correlation of warming with industrial and socioeconomic development ceases to be statistically significant if one takes into account the fact that the locations of greatest socioeconomic development are also those that have been most warmed by atmospheric circulation changes. AR4 provided no explicit evidence for this overall assessment result.
Returning to Boulton’s (sensible) second question – a question that remains relevant with IPCC’s recent efforts to extinguish public discussion of draft documents in real time:
Why were these conclusions not shown to or discussed with expert reviewers during the IPCC Report preparation?
Why indeed?
Jones explained to Boulton (and one can sense an almost Gavinesque sigh) that IPCC reviewers don’t get to see author responses to their comments at each stage (First Draft or Second Draft), that they only saw the author responses long after publication of the final report, that this was an IPCC system and that reviewers “were aware” of and accepted this system as a condition of submitting review comments. Jones:
The comment/response files for each stage were not released after each review, but only released together when the final report was published in May 2007. You seem to be under the impression that expert reviewers saw responses to their
comments at each stage. This has never been the case in any IPCC Report. This was an IPCC decision and all reviewers were aware of this when they made their reviews.
A system in which reviewers do not see author responses or review comments from other reviewers until after (and some months after) the final report surely warrants some introspection even if IPCC reviewers have acquiesced in such a system in the past. Nor does such a system appear to me to comply with (reasonable) public expectations that IPCC procedures be “open” and “transparent” – as IPCC officials like to proclaim.
In addition, I would be surprised if this sort of system is compliant with, for example, US federal standards for “influential scientific assessments”, a failure which may well create problems down the road for US agencies seeking to rely on IPCC reports without triggering fresh public comment and peer review.
Climate Audit 
86 Comments
Steve the links you’re providing to Jones’ answers, Bouldon’s questions, etc. do not seem to work
Thanks again for this accurate investigation ; when were these documents made available?
Steve – fixed. Muir Russell docs became available in July 2010 after the report was published. (But not for comment or rebuttal prior to the report).
Reality check: Did you really mean “First Draft” or “Zero Draft”? Also, shouldn’t M & M “2005” be “2004”, as you have cited it elsewhere in this post?
Steve: 2005 type corrected. “First Draft” is correct.
Boulton translated my submission into a set of pertinent and pointed questions for Jones. But he must have handed them over with a wink, since Jones seems to have blown them off with remarkable indifference, as if he knew that Boulton would accept any old drivel for an answer, as turned out to be the case. The line about not needing to calculate a p-value for a statement based on the laws of physics can be seen to be ridiculous without requiring that Jones have epistemological credentials. Everything that happens in the world is “based on” the laws of physics, so Jones is saying there’s never a need for a p-value ever, even when making a claim about statistical insignificance, as he was doing in this case. This, apparently, is what they teach at the University of East Anglia. I don’t know who looks worse, Jones for making such a crazy statement to an official inquiry where the scientific reputation of the CRU was in the balance, or Boulton for accepting it as a valid argument.
I don’t doubt the Boulton wink – it’s helpful to have laid out the shape of the game played with the McKitrick submission, one of the many mysteries of the Climategate inquiries till now.
The Muir Russell report framed the question this way:
They then stated:
What nonsense. This was exactly their remit: to comment on the rights and wrongs of whether the original exclusion was reasonable or whether it was suppressed because of conduct of Jones and his associates.
They also took the position that Jones’ individual involvement was moot because it was a “team” decision:
The Review Editor in question was Brian Hoskins, who was involved in efforts to counteract Climategate (especially in the Oxburgh “inquiry”). The Muir Russell panel failed to examine relevant documents that might shed light on whether Jones individually or together with David Parker of the Met Office did in fact “dominate” on this issue.
David Holland has obtained non-Climategate documents that show that, contrary to his evidence to the Muir Russell panel, Jones did in fact correspond with Lead Author David Parker about the handling of McKitrick and Michaels 2004 – documents on which Hoskins was copied. These documents are important to the present discussion and I’ve been meaning to report them for some time.
I have read many technical books in which a survey of the literature shows contradictory experiments or analyses and the author says “smith says X but someone else says Y. it is not possible to resolve this contradiction at this time”. The insistence on unanimity in the IPCC report leads to deletion of inconvenient literature rather than including it in the discussion as unresolved issues.
‘Muir Russell panel failed’ it did indeed for the most people . However for ‘the Team’ and as a way to support ‘the cause ‘ it did a dam good job
Not statistically significant…. but no P-value. Yikes!
That is worse then when I here uneducated drug sale reps talk about VERY statistically significant, ALMOST statistically significant, and “a clear trend towards statistically significantance”.
Makes me want to bang my head against the wall….
Interesting to compare this to a pharmaceutical case. Suppose the authors of a report on drug trials claimed that no significant differences were detected in stroke rates between treatment and control groups for an anti-inflammatory painkiller, and on this basis authorized its general use. Then it turned out that the underlying papers did report significant differences, with 2 independent teams reporting significant stroke-related side effects, and the pharmaceutical company simply falsified the evidence. It is easy to suppose there would be criminal consequences. The magnitude of the contrast with the IPCC episode is a good measure of the gap between proper scientific practices and the IPCC process. Just be glad Phil Jones and Geoffrey Boulton aren’t in charge of drug approvals.
Ross,
It is more like ‘the team’ is in charge of developing the drugs as well as getting them approved and running the court that seeks to review them and their drugs.
It’s mind-boggling to me that formal reviewers knew ahead of time that the only feedback from their comments, if any, would be reflected in the final document. That certainly does go against the grain of US funded scientific assessments. It goes against the grain of best scientific practices. It goes against the grain of any sort of technical, academic, legal, whatever sort of review I can imagine.
“Irrespective of the above comments on issues i) and ii), the evidence of the Review Editor underlines the team responsibility for the text, and the unlikelihood that a single voice could dominate on an important issue.”
There is evidence that it was a Team. Here is CG E-mail 1092418712, directly referenced to MM04:
From: Phil Jones
To: “Michael E. Mann”
Subject: Re: Fwd: RE: IJOC040512 review
Date: Fri Aug 13 13:38:32 2004
Mike,
I’d rather you didn’t. I think it should be sufficient to forward the para from Andrew Conrie’s email that says the paper has been rejected by all 3 reviewers. You can say that the paper was an extended and updated version of that which appeared in CR. Obviously, under no circumstances should any of this get back to Pielke.
Cheers
Phil
At 08:11 13/08/2004 -0400, you wrote:
Thanks a bunch Phil,
Along lines as my other email, would it be (?) for me to forward this to the chair of our commitee confidentially, and for his internal purposes only, to help bolster the case against MM??
let me know…
thanks,
mike
At 03:43 AM 8/13/2004, Phil Jones wrote:
Mike,
The paper ! Now to find my review. I did suggest to Andrew to find 3 reviewers.
Phil
I cannot connect the dots to “the chair of [Mann’s and Jones’] committee,” nor to a Pielke, but this does seem to prove that there was behind-the-scenes collusion to keep the paper out of AR4.
Pielke is probably Roger Pielke Sr. who was at one time involved in the IPCC reports and doesn’t take kindly to b*lls**t.
definitely Pielke Sr. The notorious CG1 email came one day after an exchange (384) between Pielke Sr and Mann about Mann keeping McKitrick and Michaels 2004 out of the NRC report on Radiative Forcing for which Mann was a chapter author.
Having been involved in commenting on many US government rules (e.g., endangered species listings, Forest Service planning rules, etc), the procedure is to publish the proposed rule. The general public (ie, everyone) then has a few months to submit comments. On some topics they get thousands of comments. These are all publicly available BEFORE the rule or regulation is finalized.
The comment of Jones about basic physics betrays a world view, that the belief structure of the modelers is “basic physics” (and you will hear this from advocates on the blogs), which is a deductive science and firmly established and does not allow for doubt. The fact that the physics they know requires tricky boundary conditions, parameterizations for subgrip processes, unknown forcing histories which must be guessed at (e.g., aerosols), and numerical kludges is pretended away.
True enough, but in those public proposed rules, the authors/sponsors aren’t typically obligated to respond to every public comment, they only need publish the comment. I thought the AR4 & AR5 review process was more like an ‘internal’ review. For example, if the US Forest Service develops an environmental assessment, they first have authors/investigators produce a draft. They next have internal scientifice and technical peers review the draft. The first authors must typically address those comments to the satisfaction of the reviewers. After all that, there is often an outreach to third affected parties for review comments, perhaps in this example, the US Geological Survey (USGS) or the Bureau of Land Management (BLM) or US Fish and Wildlife Service, (or many other possibilities) if relevant. Those peer review comments also might be mandatory to address before a draft Enivironmental Implact Statement (EIS) is published. Only after publication of the draft EIS are public comments invited. Those comments are typically treated on a more generic level. I’m not an expert on EIS process, but have been involved in some.
To me, an AR4 &or AR5 peer reviewer is selectively invited, as a partner in the internal review (I don’t actually know, but I’m sure someone here can confirm). They are then more like the first or second string of peer reviewers I just mentioned in first paragraph. It would be unusual I believe, for even a draft EIS to be released to public (published) before all internal peer review comments were satisfactorily addressed (or at least mediated). That’s why this seems so extraordinary. A peer reviewer of AR4 or AR5 would have to be very deferential to or trusting of the authors to be willing to proceed in this manner. Or he/she would have to have some means of seeing to it that if their comments were disregarded without cause, that they could bring this to the publics’ attention. Perhaps through a blog. I guess this blog would be a prime example in a way.
I did hear a rumor that the public will be invited to comment on AR5 at some point and all comments would be published. I don’t know if that is true, but if it is, it still doesn’t obligate authors to respond to every comment (nor do I see how that would be practical). Even so, it’s more disturbing to me (if true) that even peer review comments can be disregarded prior to publication.
Michael,
There is an open invitation for expert reviewers for the WG1 First Order Draft here:
https://fod.ipcc.unibe.ch/registration/
Expertise is self-declared, and if you have no published papers you can just enter “none”.
(For those who already saw me say this on the earlier “Another IPCC demand for secrecy” thread, sorry to be repetitive!)
Review comments must be submitted by 10th Feb.
Yes all comments will be published when the AR5 has been published.
Authors are obliged to address every comment, and make a record of the response to the comment – again these responses will be published at the end of the process (individual reviewers will not receive direct responses)
A similar open invitation for reviewers will go out when the WG2 FOD is available in the middle of this year.
Hope this helps!
Richard
ok, thanks. I don’t follow this as close as I should.
Richard, it would be more accurate to say that in principle, Authors are obliged to address every comment. But that is the sort of loose language in the IPCC rules that in practise has been stretched to cover all manner of practises completely antithetical to what is implied. As the record pertaining to this thread shows, authors can and do summarily dismiss comments, often merely writing “rejected” in the spreadsheet. In the case being discussed here, Jones or another author wrote, without evidence, that our paper was “full of errors” and that it was disproven by Parker’s paper, which did not address any of the evidence in our paper. The Review Editor made absolutely no efforts to stop them.
Also, the authors ignored reviewer comments on this matter in the drafts that were shown to reviewers, then after the close of peer review inserted the false claim that our results were statistically insignificant; text that was never shown to expert reviewers. Again, the RE’s signed off with no apparent concern.
In another case, discussed here and here, text cautioning about exaggerated significance of temperature trends was introduced in response to reviewer comments, privately acknowledged in emails among lead authors to be valid, kept in the subsequent drafts even into the government review phase, and then prior to publication, removed and replaced with unreviewed, unsupported text claiming that the problem was non-existent.
Notwithstanding what the rules could, on the most charitable reading, be interpreted to say, in practice IPCC Lead Authors can and do ignore any review comments they dislike, they do so within the structure of the existing rules, and nothing in the review process requires them to do otherwise.
Another example of IPCC authors unilaterally ignoring review comments was Briffa’s refusal to show the decline even when, as a reviewer, I asked that the “decline” be shown in the spaghetti graph. Briffa merely said that it would be “inappropriate” to show the decline and that was that. It was this incident that led David Holland to John Mitchell’s Review Editor comments. The Met Office made untruthful answers to justify their refusals – the untruthfulness of the Met Office being very corrosive to the reputation of that organization at Climate Audit and other blogs. CRU followed the Met Office example of making untruthful excuses to FOI requests, corroding matters further.
When I see handwringing by climate scientists about “climate communications”, the commentary always overlooks one of the most obvious requisites for successful communications: honesty even to critics. Unfortunately, the proven untruthfulness of the Met Office and CRU in something as simple as FOI requests corrodes their credibility on other topics.
This needs to be front and centre as IPCC acolytes seek to paint McIntyre as a gratuitous troublemaker, with refusal of AR5 ZOD takedown requests just the latest example of his intransigence. A fair number of influential people now understand Hide the Decline as ‘data presentation fraud’ – in Burt Rutan‘s useful phrase. The fact that Briffa was able to blow off this, of all review comments, without any consultation, for the AR4 report, says it all.
Steve,
Thank you for your very frank feedback on the Met Office, it is always interesting to see how one’s employer is perceived.
Since you have been so direct with me, I hope I can ask you a direct question in return, prompted by Richard Drake’s remark that
.
Is this picture an accurate one? Are you “a gratuitous troublemaker” or are you just trying to help, albeit in a way which some find uncomfortable?
Indeed what would “success” look like for you at the end of the AR5 process? A Fifth Assessment Report which is as scientifically robust as possible, so that governments can make informed decisions on climate policy, whatever the outcome of those decisions might be, or do you (as some appear to think) have a particular objective to influence these decisions in a particular direction?
I hope these are not impertinent questions. I didn’t really intend to get into such issues, especially not in one of my first contributions to your blog since I only commented to clear up a couple of misunderstandings and answer some specific questions, but your comment and Richard Drake’s has sparked these off in my mind.
Best regards and thank you for your attention,
Richard
Many people think the AR5 process is already irretrievably damaged by the revelations in ClimateGate. I think AR5 should be scrapped and AR4 reinvestigated by teams made of of people on all the sides of the arguments.
It would have to be public. All emails and discussions public. Maybe they should be televised.
The AR5 teams seems to want to keep everything secret without any kind of audit trail. IPCC is a damaged brand.
My wife and friends have never understood why I’m doing this and think that I should go back to doing business.
Mostly I do it because I’m interested in what I write about. I’m not as interested as I was a few years ago, when I woke up every day with 3 or 4 things that I was interested in writing about. There was a lot of energy in my posts a few years ago. I also found the low culture of the Climategate correspondents very distasteful and this has sapped my interest.
Answering your question would take a fairly long essay. It’s late here and I’ll try to reply some time, though I don’t know when.
I’m actually not that interested in “big” policy. I assume that politicians will do what they’re going to do. On a private basis, I tend to think that if climate is a “big” problem, the proffered solutions are probably totally inadequate and that windmills, for example, are like prescribing laetrile for cancer. But that’s not an issue that I deal with at this blog. The only policy that I’ve advocated is much better data archiving. The opposition on this point by climate scientists has been totally insane and the opposition to the prima donnas should have been led by people worried about climate, not by “skeptics”. (As regular readers know, I am not confident enough in my knowledge of all the relevant issues to express an opinion on the “big picture”.) I realize that many readers are interested in policy, but, as an editorial policy, for the most part, I avoid discussions of policy and delete many comments that do not adhere to this policy and urge readers to take such discussions to other blogs (of which there are obvious excellent examples.)
I’ve frequently made suggestions as to how people concerned about the impact of increased CO2 could present their case more effectively. I’ve often used the term “engineering quality report” as an important element in the process. It’s not something that I’ve defined or explained and it would take considerable effort to set out specs for such a document. Most readers get wrongfooted on the phrase and talk about V-and-V or try to point to little articles that they find meritorious, but the sort of thing that I have in mind is more like the sort of document that a professional engineering firm would do for a refinery or a mine or something like that.
I thought that the IAC review should have re-examined the purpose of the IPCC reports – a point that I made to Shapiro in a telephone conversation but which wasn’t considered in their report. It is not at all obvious to me that a literature review of work in the previous 5 years is what is needed.
I obviously have a very low opinion of the standard of work in the 1000-year field. While my criticisms in this area are pointed, I try to write accurately and, when writing well, avoid editorializing as much as possible. If you are aware of any inaccuracies in anything that I’ve written, please bring it to my attention so that I can make appropriate corrections.
Some of my original engagement arose from my astonishment at outright dishonesty that I encountered early on and by the lack of self-policing of such conduct within the field. This was long before Climategate. As a matter of decorum, I’ve established blog policies that do not allow readers or myself to make accusations of dishonesty or “fraud”, but I don’t say everything that I think. I think that the handling of Climategate by the broader climate community has exacerbated what was a difficult situation. Something like the trick to hide the decline should have been disowned in some manner, rather than whitewashed. Financial managers, lawyers, accountants and other professionals are dumbfounded that the climate community is unoffended by such conduct. The various “inquiries” have unfortunately exacerbated the problem through their failure to adhere to even the most elementary principles of public inquiry. These defects are easily understood by non-academics.
There’s lots else that I can say.
In addition to advocating a policy for better data archiving, it’s my impression that you also advocate better governance policies at various institutions (conflict of interest, independent reviews (audits), etc.)
There’s a lot more I could say. Even in my short acquaintance with this blog I am astonished that one person can achieve so much, with so few mistakes, with such good grace. A demonstration of intellectual rigour that sets an example to people far beyond those concerned with the issues discussed here. His wife and friends should know that when he stops he will probably have far exceeded his original aims.
Perhaps if the climate community and scientists in general would accept “failure” such as is done in engineering then many issues would be resolved. Are their analyses such as the one at
http://failures.wikispaces.com/Building+Failure+Cases
in which case studies in the failure of buildings are presented as a means of extending the art? Engineers must accept failure because a collapsed structure cannot be denied so they embrace it.
In mathematics, a refutation is seen as a step forward because it points out inadequacies in current theories. As new knowledge is found, existing theories are adapted through a series of refutations to be made more general and more powerful. The book “Proofs and Refutation” by Imre Lakatos provides an historical description of this in the conception and development of topology from Euler onwards by new knowledge leading to new concepts which showed the inadequacies of existing theories.
Perhaps this is common practice in science and only my ignorance prevents me from seeing it. However from what I have seen of the comments of climate scientists is that they are adamant in refusing to acknowledge any deficiency on their findings and especially will not accept any such finding from others
As an author of in-depth engineering risk assessments myself, I have given considerable thought to what one would look like for climate change. When we are not sure how something will respond to conditions outside of experience, we do a test. Sometimes that is not an option, in which case the logic structure built around the various plausible scenarios cannot be fully resolved. Trying to minimize uncertainty would be a bad idea (leave that for the managers). When I compare the tools available in climate science (or more accurately the lack of tools such as a control sample of the earth’s climate) to the questions that need to be answered…frankly, predicting the future looks hopeless. Climate scientists are now being asked to exhibit the same rigor expected of engineers, driven by folks like Steve. At this point, I cannot shake the notion that some climate scientists have flown too close to the sun (figuratively speaking!), and their wings have melted.
Divergence from standard risk assessment practices is among my main concerns with IPCC as well. Other risk assessments that I have contributed to, such as those for geologic based repositories for nuclear waste, are hybrids of engineering-based and natural sciences – based performance (risk) assessments. They incorporated explorations of climate uncertainty among other questions. They also borrowed greatly from seismic hazard analyses approaches. Those seismic hazard analyses have long attempted to integrate stochastic uncertainty (natural variability) with epistemic uncertainty (trying to understand what you don’t know).
As a part of that, they undertook extensive analyses of alternate conceptual models (ACMs). But AR# products appear to have only one conceptual model for climate change, namely that milankovitch cycles once drove climate (along with various speculative feedbacks) and that now, anthropogenic greenhouse gases are the driver, even though IPCC cannot point to any model which can replicate both. For that matter, at least one of the performance assessments I referred to, clearly rejected Milankovitch as a causal factor for past ice ages. Somehow IPCC breathed new life into that orbital forcing hypothesis without any further dispute. In any case that is but one example of their adoption of a single conceptual model.
Because AR# products don’t appear to attempt to explore epistemic uncertainty except in the most nominal way, they fail to achieve credibility on any level. At least that’s my perception.
Apparently there was some confusion in testing the Milankovitch Hypothesis. Studies were using the glacial ice thickness as a proxy for temperature instead of the rate of change in thickness. Lubos has a nice summary:
http://motls.blogspot.com/2010/07/in-defense-of-milankovitch-by-gerard.html
I find it interesting that the best fit between the model and the observations is a zero lag. Of course, given resolution constraints, the lag could be significantly higher than zero.
Sorry for the OT.
I’ve read that paper a while back, Roe, G., 2006, “In Defense of Milankovitch”, Geophysical Research Letters, Vol. 33, L24703., and have cited it in a poster I recently presented at last December’s AGU. It’s certainly no defense of Milankovitch.
In spite of the title, it’s actually quite useful in REBUTTING Milankovitch.
That’s the problem with citations to blogs about citations, you don’t get to the heart unless you go to the actual source.
Roe himself seems an interesting and otherwise most objective character. I attended a lecture at that same AGU meeting that he presented titled “What can glaciers tell us about climate variability and climate change?” At this talk I learned quite a bit about stochastic glaciology. Enough to feel confident that there is nothing about kilometer scale changes in glacial length over the past century or so, that can be disambiguated from natural variability.
Steve,
just to let you know that is greatly, immensely appreciated, and I’m sure I’m not speaking only for myself …
however, something we disagree on, the “big picture” is quite important too … why do some scientists and most politicians say what they say at this particular moment in time about this particular subject ?
anyway, it’s your blog, and you keep on doing exactly what you want … that’s what my mum, also retired for many, many years always says: “I’ve become way too old to do something I don’t like” … 🙂
Steve,
Thanks for that. I’ll look forward to reading the full essay if/when you find time to do it!
Talking of mines, funnily enough my team now do work for mining companies in the context of risk assessment – some of the multinationals now seem to be regarding an assessment of potential changes in local climate as a necessary aspect of their due diligence for new operations. eg: what are the potential threats to water supplies in the context of the huge demand throughout the lifetime of a new mining operation (many decades), and more importantly, what level of confidence can actually be attached to such assessments of possible future changes? A tough challenge, but it seems there is an emerging market for this kind of advice. And as you say, they can be very demanding clients!
Richard I responded to your comment on mining interest but my reply ended up elsewhere, out of context. probably something I did wrong. however it is at
“Posted Feb 2, 2012 at 3:57 PM”
I wonder if there’s a disconnect with the phrase “scientifically robust as possible”
From what I’ve been reading here, there, and everywhere…Some scientists FIRST definition is:
“the boldest conclusions possible”
others say first and foremost it is:
“the most reliable, valid, and certain conclusions possible”
while yet still others say it is:
“the most reproducible conclusions supported with the greatest cadre of fully available evidence”
I think that all scientists involved consider “scientifically robust” as containing all three of those definitions, but there may be a problem where some are ranking them differently, and implying that those who aren’t doing as they do are being mischevious or are otherwise appearing to work against them.
I think the first definition could perhaps be better understood as:
“the boldest– and/or most actionable, conclusions possible”
Salamano
Interesting. By “robust” I mean something along the lines of your third definition – ie: backed up by evidence, or in the case of future projections, by theory (or models) grounded in evidence. This means being very open about where we are relatively confident and where there are large uncertainties, especially in future projections and most especially at regional scales and for potential impacts.
“I mean something along the lines of your third definition – ie: backed up by evidence, or in the case of future projections, by theory (or models) grounded in evidence. This means being very open about where we are relatively confident and where there are large uncertainties, especially in future projections and most especially at regional scales and for potential impacts.”
Judging from your comment here…allow me to make a few postulations that may or may not be at all accurate (feel free to clarify for me)
From what I gather, Steve also has a high priority for ‘robustness’ in IPCC material as it relates to evidenciary grounding– but instead his area of expertise/interest is different from what it appears yours is (within the penumbra of climate science). He is big on paleoclimatology and temperature reconstruction, particularly as it relates the statistical ‘robustness’ of conclusions that can be established scientifically from the data (or rather that has been established).
Since your area of interest is more forward-looking (ie, ‘interesting’ to the greater climate science community) and Steve’s interest is more in the review of paleo (ie, ‘uninteresting’ to the climate science community), might it be possible that Steve’s pursuits could be viewed as ‘trouble-making’ not only because it’s in an area that many in the climate science community wish to be considered ‘settled’, but also because it has a more foundational element to it that, if allowed to gain traction, can take the focus off the ‘forward-looking’ perspective?
Otherwise, purely examining things from the frame of reference that you have highlighted by the term ‘robust’ (ie, demonstrable evidence that surpasses litmus tests of statistical correlation/significance threshholds), has Steve not raised any valid flags in any of his research activities that he deems interesting and relevant?
Ross, don’t overlook Parker’s role in all of this. Parker was the Lead Author of this section. Jones’ defence to Muir Russell was, to a considerable, the argument that someone else (Parker) was the direct author of the contested language. Muir Russell negligently did not inquire into communications between Jones and Parker. David Holland obtained a damning email between them that I plan to post on.
The first ‘here’ above should be here.
Yes, I realize Parker’s role here gets less attention than it deserves. He was as much in a conflict of interest as Jones on the temperature data issue, and was the author of the email to Jones acknowledging that their ML estimator was likely inadequate for dealing with persistence in the data. So he was right there alongside Jones in a position of responsibility within the IPCC when the AR4 text on these matters was falsified outside the review process. The Met Office-CRU tag team, as it were.
Sure. But there is no rule saying that should a member of the public obtains any draft at any stage in the process, that member of the public can’t circulate the draft and people quote the draft and talk about the contents on a blog, forum, local knitting club meeting or what have you.
The letter from the IPCC to STeveMc isn’t merely saying that they aren’t inviting comments, nor are they saying that authors of the document will not read or respond to discussions outside the IPCC cone of silence, they are trying to say that the public cannot quote, circulate or cite their drafts so as to make it seem the members of the public are not permitted to speak to each other.
That’s quite unusual. It might have a parallel in material that is classified as state secrets. But no one claims the process involved in dealing with state secrets is open and transparent!
Dr. Loehle, you bring up a valid point that supports Steve’s “In addition, I would be surprised if this sort of system is compliant with, for example, US federal standards for “influential scientific assessments”, a failure which may well create problems down the road for US agencies seeking to rely on IPCC reports without triggering fresh public comment and peer review.”
In particular, the EPA is being sued over the proposed 1 hour SO2 rules, and one of the items in the suit is that the EPA took comments that were adverse to their proposed regulations, and changed that part to a different methodology without allowing comment to the changes, and the methodology. Several states are suing that such a procedure violates Federal law by way of violating the approved way for EPA to promulgate regulations. They seek to overturn the regulations until the regulations are promulgated according the Agency’s own rule making procedures.
Note that the legal requirements for US federal rule-making are set up to prevent (as much as possible) capricious and impossible rules from being promulgated. EPA has been known to propose pollution control where no technology exists to meet the rule, or with ambiguous language, or with a standard that even natural waters fail to meet, or with onerous reporting requirements. The public comment period is meant to head off such adverse outcomes. While they are not obligated to respond to each comment per se, when it is pointed out that they are asking the impossible, they quite often revise or withdraw the rule–so they are in fact responsive.
This suit has elements that for some will be a bit ironic. In repsone to the comment that the EPA required a too expensive testing pattern, modelling was proposed. EPA’s position was that current air sampling sites did not measure a small enough grid to catch violations of the proposed limits. The states’ comments was that it was too costly to do it in such small grid sizes and in general most states, except 2, were in compliance. The EPA without comment and review put in that models were to be used. The problem for the states was 2 fold. One is that to model each part of a state on the grid size necessary would be as or more costly as the sampling air monitors. EPA’s contention was to use a larger mesh scale which effectively smeared the 1 hour SO2 all over the map. Test runs on the models indicated that for the size mesh that was affordable, actual SO2 had to be 2 to 4 times lower than the regs required in order for large emitters to be in complaince. But the real issue is that the EPA has required monitoring and data for compliance determination, and the legality of using models as the final determinant is being also challenged. If implemented as was proposed, only 1 state would be in compliance.
There are other issues concerning the “US federal standards for “influential scientific assessments.””
John: missing link on “this suit”? I should have clarified that EPA is “sometimes” response–that is I have seen cases where it is. Other times, cases like you cite occur.
The information was provided verbally by different state agencies to the stakeholders (large emission facilities). Details are available on some state web sites, I assume. I had access one of the discussions. I don’t have the lawsuit docket number handy.
No need to go as far as the 1 hour SO2 rules, EPA ignored their own procedures in citing the IPCC report as a major justification for the GHG endangerment finding. See:
This was EPA’s first step in the regulation of GHGs, especially since the legistlation foundered. They are due to issue draft regulations for GHGs this month, and since they agreed that CCS is not “available,” they have no real technology to address CO2. What a dilema to attempt to regulate an emission for which there is no cure. If they follow form, they will issue a regulation that is filled with errors and unimplementable requirements.
I believe that Phil Jones attitude can be explained by changing one word in his quote
There is no need to calculate a p value for a statement that is based on the my laws of physics
rather disappointingly, the strike html around ‘the’ hasn’t been accepted
There is no need to calculate a p value for a statement that is based on my laws of physics
Nice work! The last paragraph is an important addition and emphasis.
I suspect I am not alone in finding the most extraordinary circularity in this?
“…the locations of greatest socioeconomic development are also those that have been most warmed by atmospheric circulation changes… Hence, the correlation of warming with industrial and socioeconomic development ceases to be statistically significant”
In effect, this is saying that, because we already know that there is more warming in developed areas, it is not reasonable to use that fact to claim that there is a causal link between warming and development.
In other words, we can look for other explanations – but we can’t actually consider the most obvious one, which is that development itself raises the local temperature.
Am I in some way misrepresenting this circularity?
gober – No, you have it just right.
No, gober, I think you have to read the whole reply by Jones, which, although a textbook case of prevarication and misdirection, does cite reasons and sources for the rejection of the inclusion of M and M. And while I’m merely a layman, I’m sure Ross and many others more conversant than I in these issues find them wholly unconvincing and to quote Boulton, “ad hoc.”
In the end, it’s telling how frequently in his response that Jones tries to shift the rational for the decision to the group, which should remind us all of Wegman’s observation that the clique of climatologists producing these reports and papers is essentially a closed self-reinforcing group that will not tolerate dissent.
“We are the group. When we deliver our consensus, debate must end. No need to question our conclusions because they are final. We are the group.”
It is fascinating to consider how committed the IPCC in particular and the AGW community in general are to secrecy, non-disclosed relationships, insider dealings, non-accountability, black box processes, etc. Climategate leaks, the IPCC, day-to-day actions of AGW opinion makers are all consistent with this dysfunction. At best it reflects badly on the standards the AGW community adheres to.
Hi Steve,
I do not frequently comment on blogs – even though some of my papers [de Laat and Maurellis, 2004, 2006] are at the heart of this matter, but in this case I want to because I think it is interesting to note that in AR5 IPCC admits that they did something stupid (sorry, I have no other words for it).
This whole story is an ugly piece of IPCC history, and I have tried for many years – unsucessfully – to get something published about IPCC making up arguments.
I guess IPCC learned, maybe also because some of my direct colleagues are now involved in AR5 WG1 Ch2, and I have made them very clear that I was not pleased with the way AR4 handled this issue, to say the least.
By the way, this does not mean that the findings of our papers are therefor correct, but that is another story.
Best regards,
Jos.
Jos:
One can hope that the text about AR4 making an unwarranted statement will survive the revisions, but don’t hold your breath.
As to whether anything said or written (by Ross or Steve or IPCC or yourself) is correct, I think IPCC is far too quick to believe that they have the right to make that judgement, whereas on such a complex topic it is very very difficult to work through the arguments and analyses and have your judgement turn out 20 yrs later to have been correct.
Hi Craig,
Sure, we’ll have to see if it survives, but at least there are people willing to admit that mistake. For me that was a pleasant surprise.
I think what I wanted say was that also my understanding has evolved. And I have been continuing to work on it and gained some new interesting insights (submitted paper).
But you are right, I don’t believe that at this moment or for the coming years/decades the debate around (surface) temperature trends will easily settle.
J.
Hi Jos… Does your submitted paper have anything to do with trends in particulate emissions at the local/regional level? Steve had an interesting post a month ago about the past use coal in Toronto:
IIRC, the principal driver in Ross’s findings was coal. Then again, I could be wrong.
Not sure, but possible. Europe has warmed faster since the late 1970’s than the global mean. Europe has also seen a drastic decrease in particulate matter since the late 1970’s.
The same appears to apply for the period after the second world war. Europe has seen a drastic increase in particulate matter, temperatures have dropped faster than the global mean.
Since aerosols are important for explaining the lack of warming after WW2 and before the late 1970’s, these temperature trend observations actually fit in nicely with the mainstream paradigm.
I guess that if I would have written that down and used the proper buzzwords the papers would have been heralded rather than disgruntled.
BUT, I am not convinced about this story, and I have a paper in the making that offers yet another perspective.
Cheers, Jos.
The IPCC’s working procedures and reports remind me of the recent banking fiasco in the West: if small organizations blunder they will be dismantled fast, but when the biggest mess up they are allowed to continue without much ado by the Establishment (because that failure also touches their pie(s) or ego(s)).
Is the IPCC too “big” to fail?
The IAC observes:
Climate change assessments | Review of the processes and procedures of the IPCC page 17
The IPCC policies appear to me to be directly opposite that intention. i.e. by prohibiting disclosure of materials accessed does not make them “easily accessible” to reviewers. Easily accessible would be to be free to distribute them for comment and evaluation.
Hiding all reviewer responses and author comments until after the final report is directly opposite to “easily accessible”.
David, here is how the IPCC interprets “all references used in the reports are easily accessible.”
When a reviewer asks the IPCC for an unpublished reference, he is sent a file that is encrypted (“for security reasons”) using a piece of commercial software called LockLizard. He is asked to download and install this software on his computer, then install the license manager. If the reviewer complains that he has Linux and so is unable to install this PC software, he is told “we cannot make exceptions”.
In the end I got the paper I wanted from an author at the Met Office.
Hi Paul
Good to know the Met Office was of some further assistance to you in your role as an IPCC reviewer.
Thanks for mentioning it!
Cheers
Richard
Paul M
If a reference is unpublished, how does the reviewer know it exists, please ?
ianl, this is when the IPCC draft refers to a paper that is submitted to a journal but has not been published yet, and therefore is not generally available. (In some cases, for example the BEST papers, the preprints are posted on the web, but usually not).
Richard B, yes, brownie points to your colleagues at the Met Office!
“It remains possible, however, to ascribe patterns of warming trends to atmospheric circulation because its influence is in accord with the laws of physics and can be detected in day-to-day weather variations, on which timescales socioeconomic trends are infinitesimal.”
So is he saying you he accurately subtract local “weather noise” from the data to reveal a global, gradual and persistent, yet statistically significant underlying CO2 AGW signal. But that the highly localized socio-economic effects are not significant because they are, errr…, gradual and persistent? And he knows they are small because they have to be, because he already subtracted the “CO2 signal” to leave almost nothing. Circular reasoning?
I need a lot more convincing that this is correct. I would have thought that the Watts/square meter effect of CO2 was more highly dispersed, spatially and temporally, than is a conurbation or other socio-economic effects.
The spatial effect in Ross’ paper is not due to climate change, it is the contamination of the temperature record (on land mainly) due to land use change and urbanization. To counter Jones, it is basic physics that urban areas (in locales with forest, but not in desert) will warm due to urbanization vs the countryside.
Its also basic physics that certain LCZ ( local climate zones) in the urban area can be cooler than rural areas. the variability in urban climate zones is remarkable.
can you give an example of this Mosh, is this the evidence for white roofs for everyone?
google stewart oke LCZ
for starters.
‘white’ roofs or albedo differences is one issue. the other things
that modulate the effect are; skyview, roughness; aspect ratio; anthro heat; and heat storage.
for a stark contrast compare compact hi-rise with low density residential.
both urban but big differences.
Mosh: nesting came undone. Of course all the things you say as effects are correct, but there are so many of them (don’t forget people planting trees which grow over time) that it is no longer “basic physics” in the sense that one can simply deduce an answer, but empirical because the complexity prevents deduction from some known law. That is why I was making fun of Jones’ comment [never mind that his own work first denied UHI effects and then found it in China]
Micheal, this is an excellent example of the fallacy of “begging the question”; i.e. imbed the conclusion you are attempting to prove as an assumption in the your argument.
I think the writing team in Ar5 needs to be congratulated for addressing this issue in the way they did.
Steve: unfortunately they undid this bit of candour in their next paragraph. Stay tuned.
I don’t know if I missed any comments on this but I think everyone needs to look at the new Appendix A published on 10 January 2012 and compare it with the old 2008 one published on 13 October 2008. You’d better get a copy of the old one in case it disappears.
These guys have really gone for broke.
It is not just the public that cannot see the comments and authors responses before their governments have to irrevocably accept the final IPCC report – nor can their governments according to the new rules that representatives of our respective countries appear to have agreed to in Kampala sometime between 18 and 19 November 2011.
The last paragraph of the old section 4.1 stated:
In the new Jones-Stocker Appendix A, a new section 4.1 pushed the old one down to 4.2 and the old final paragraph shown above is gone, replaced by the JS confidentiality clause.
Remember the JS clause was sneaked through the previous IPCC Session without either recorded discussion or a vote. I think we should be getting our MPs and Congressmen to ask how this one got passed.
When Renate Christ addressed the IAC she said the idea of the second review stage was for Governments and Expert Reviewers to ensure that the comments on the first draft had been properly taken into account. How can anyone until it is too late?
As importantly how many of you have tried to use the Harvard WGI AR4 archive? You can not search and cite nearly as easily as you can with a pdf. They must have gone to Luther Pendragon.
This is complete madness and I hope Richard Betts and the likes of Judith Curry step up to plate and call it so.
Re: David Holland (Feb 2 09:52),
For anyone that wants to see how the new JS Appendix A came about, the documents for IPCC Session 34 are here and the proposal for the changes is here.
If you look at the IPCC Calendar you wil not find the links but if you go to Session 33 and change 33 to 34 in URL bingo. Not rocket science but nor is it open and transparent.
Brilliant detective work with the URLs David. It’s tempting to take yet another pot shot at those responsible for such opaqueness in the web chronology but, much more important, in getting past it you’ve shone a giant spotlight on how far the IPCC has marched backwards on transparency with AR5, even for governments, and all this through the Jones-Stocker gambit. It is Orwellian, in the worst possible sense. As you rightly say, someone must step up to the plate.
I don’t know if anything has changed since Feb 2nd but if you go to Meeting Documentation on the menu on the left and then select IPCC & WGs Sessions underneath, they have the Session 34 documentation at the top.
Can I assume this was not there a week ago? Certainly bad form not to have it on the Calendar, especially since they actually have a section for these links.
This really does look like an Anthony Jay script for “Yes, Minister” – but with real consequences. I doubt that History will look kindly on the likes of Jones and Stocker
Last year at the Global Mining Water Initiative in Las Vegas Nv, I watched a high-ranking EPA administrator lecture to all assembled that alarming climate change was all but certain and that mining companies better get with the program. That’s among the reasons you are likely seeing interest from such companies.
Michael,
Thanks, that’s interesting. Our involvement goes back a couple of years at least. In many cases (and this is a generalisation across several industry sectors) we find that a company we have been providing weather services to for a while starts to ask about longer-term climate issues, if they are in a sector where multi-decadal scenario planning and risk assessment is needed.
Richard thanks for your communications and patience. I find that your engagement with skeptics and other parties here is most refreshing compared with my experiences attempting to communicate with local climate-change authorities.
But regarding your providing multi-decadal scenario planning and risk assessment to mining customers, I’m curious how that translates into a novel service that adds value.
Hydrologists already condense historical climate variability data into technical products that are multi-decadal and are of great use to mining customers and many others. For example, the often – used “50 year flood”, or the PMF (probable maximum flood) are two well-documented prognostics that come to mind (and more than fit within the lifetime of a typical mine).
They incorporate not only climate variability but typically other phenomena and features, such as groundwater variation, topography (of course!), vegetative patterns, upstream water resource management issue, etc. They also typically are understood to be ‘works in progress’ and are subjected to calibration exercises and empirical closure towards the goal of minimizing uncertainties.
My guess is that an organization that promotes some sort of added certainty in climate change (at current state of the art)would be tempted to apply deterministic ghg-forcing GCMs and the like, to predict future conditions of drought and flooding likelihood. My problem with that is that the state of the art in that modeling field has failed to validate in any meaningful way to the types of tests that would be most relevant. I feel that aside from divergence of current temp records from C02 increases, there is also the prime problem of simulating even a single 100K yr glacial cycle. It’s never been done.
Thanks Michael. Indeed – we are very careful to attach clear health warnings to such work, since different models (or even different variants of the same model) can give a wide range of different projections, and moreover, as you say, they simply cannot be directly validated for the timescales and forcing scenarios for which the projections are made. The models can be evaluated in other ways, but in essence the future projections remain simply a best estimate based on current understanding as expressed in the form of a model. Some people still find this useful, but admittedly some do not!
fair enough! Thanks
There ya go Steve! Now you can mix business with pleasure. Just find a hydrologist, mix in a model, and produce assessments that have the highest likelihood for generating future cash flows! 😉
That commonly used dismissive “basic physics” phrase begins to irk when IPCC authors just ignore the basics when it suits them; eg that a warming ocean is somehow a net CO2 sink, that the deep ocean is somehow hiding the “missing heat” without first heating up the ocean surface, or that cold weather events are somehow triggered by global warming.
Of course that double standard doesn’t irk quite as much as bold statements from IPCC authors with no theory behind them whatsoever; like the whopper that a 4% increase in water vapour causes droughts in Russia or Texas, floods in Pakistan and every other event previously known as natural, random or jetstream related.
“One could make a full time job of publishing criticisms of poor or incorrect papers. ”
Is this not a common practice? And if not, why not?
One Trackback
[…] Although the IPCC calendar webpage doesn’t link to session documents of the 34th session (Kampala Nov 2011), David Holland has alertly located the documents – see here. […]