The U.S. EPA just released (Apr 17, 2009) “Proposed Endangerment and Cause or Contribute Findings for Greenhouse Gases Under Section 202(a) of the Clean Air Act” url together with a Technical Support Document url. In Canada and most countries, governments just implement these sorts of policies without the huge regulatory process that delays everything in the U.S. (including nuclear plants.) So I’m not opposed to governments making decisions, even if I don’t agree with the decision.
The EPA said that it primarily relied on prior due diligence by IPCC, CCSP and the US National Research Council, taking the seemingly plausible position that these reports complied with EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Informationurl.
The present report is clearly “influential information” under EPA policies. EPA has legislative guidelines for the quality of “influential scientific, financial, or statistical information”, while I’m unaware of corresponding guidelines for NRC reports and IPCC reports (and perhaps CCSP reports). While EPA seems to take it as a given that the due diligence for these reports is sufficient to comply with legislated EPA guidelines, my own experience with NRC and IPCC due diligence is not totally reassuring. This experience has been discussed from time to time over the past few years at Climate Audit. I’ll link to a few relevant threads, but there are many other disappointing incidents.
NRC Report on Surface Temperature Reconstructions
The NRC Report on Surface Temperature Reconstructions is cited in the EPA Technical Support Document. In a CA thread here, I quoted comments by the panel chairman, Gerry North, in which he stated that they “didn’t do any research”, that they got 12 “people around the table” and “just kind of winged it.” He said “that’s what you do in that kind of expert panel”. A clip of North’s remarks is online here. This sort of casualness offended me at the time as being inconsistent with expectations of legislators who reasonably expected a bit more seriousness.
This casualness was further exemplified by North’s response to my online question to him during a Colloquy (threaded at CA here)
Question from Stephen McIntyre:
The NRC Panel stated that strip-bark tree forms, such as found in bristlecones and foxtails, should be avoided in temperature reconstructions and that these proxies were used by Mann et al. Did the Panel carry out any due diligence to determine whether these proxies were used in any of the other studies illustrated in the NRC spaghetti graph?
North’s answer was as follows:
There was much discussion of this matter during our deliberations. We did not dissect each and every study in the report to see which trees were used. The tree ring people are well aware of the problem you bring up. I feel certain that the most recent studies by Cook, d’arrigo and others do take this into account. The strip-bark forms in the bristlecones do seem to be influenced by the recent rise in CO2 and are therefore not suitable for use in the reconstructions over the last 150 years. One reason we place much more reliance on our conclusions about the last 400 years is that we have several other proxies besides tree rings in this period.
After the NAS panel said that bristlecones should be “avoided” in reconstructions and North here saying that they are “not suitable for use in the reconstructions over the last 150 years”, the NRC used bristlecones in their spaghetti graph, which is now carried forward into the EPA Technical Support Document.
I’ve observed on many occasions that IPCC does not itself carry out any due diligence. A clear statement of this occurs in Mann’s 2003 answers to questions from Inhofe (noted up at CA here). There were a series of questions, starting with:
30. Did IPCC carry out any independent programs to verify the calculations that you made in MBH98 or MBH99? If so, please provide copies of the reports resulting from such studies.
It is distinctly against the mission of the IPCC to “carry out independent programs”, so the premise of the question is false. However, the IPCC’s author team did engage in a lively interchanges about the quality and overall consistency of all of the papers as the chapter was drafted and revised in the course of review.
Whether a “lively interchange about the quality and overall consistency of all of the papers” is sufficient to comply with EPA guidelines to ensure and maximize the quality, objectivity, utility and integrity of information is surely a question that merits attention by somebody.
The supposed “transparency” of the IPCC process – pointed to by EPA in support of reliance on IPCC reports – has been the topic of many threads here as we follow the lugubrious stonewalling by IPCC and UK agencies, particularly with regard to Ammann’s secret comments to IPCC and Review Editor Mitchell’s secret review comments. IPCC principles state clearly:
All written expert, and government review comments will be made available to reviewers on request during the review process and will be retained in an open archive in a location determined by the IPCC Secretariat on completion of the Report for a period of at least five years.
However, Caspar Ammann of NCAR in the US submitted secret comments about IPCC AR4, which he, CRU and IPCC have collectively refused to disclose. Most recently, CRU stated in response to an FOI request:
In regards the correspondence from Mr. Ammann, s.41 is applicable as we have consistently treated this information as confidential and have been assured by Mr. Ammann that he believes it to be confidential and would expect it to be treated as such. The public interest in withholding this information outweighs that of releasing it due to the need to protect the openness and confidentiality of academic intercourse prior to publication which, in turn, assures that such cooperation & openness can continue and inform scientific research and debate.
“Confidentiality of academic intercourse prior to publication” may be OK for journal publication, but IPCC reports are not journal publications and IPCC policy required that Ammann submit on-the-record comments.
AR4 Chapter 6 Review Editor John Mitchell has likewise refused to disclose his review comments, providing a bizarre sequence of untrue prevarications. First, he said that he had destroyed all the review comments on the basis that he had no obligation to preserve them (in face of express IPCC regulations otherwise); then he said that they were his “personal” property. When he was asked whether the Met Office had paid for his time and travel to IPCC meetings, a new excuse merged. Disclosing the comments would interfere with relations with an exempt sovereign international institution (IPCC), which appears to me to be exempt from any FOI anywhere in the world and which refused to permit the Met Office to disclose Mitchell’s review comments.
So let’s see what EPA says about its reliance on these reports. The draft finding states:
EPA has developed a technical support document (TSD) which synthesizes major findings from the best available scientific assessments that have gone through rigorous and transparent peer review. The TSD therefore relies most heavily on the major assessment reports of both the Intergovernmental Panel on Climate Change (IPCC) and the U.S. Climate Change Science Program (CCSP). EPA took this approach rather than conducting a new assessment of the scientific literature. The IPCC and CCSP assessments base their findings on the large body of many individual, peer reviewed studies in the literature, and then the IPCC and CCSP assessments themselves go through a transparent peer review process. The TSD was in turn reviewed by a dozen federal government scientists, who have contributed significantly to the body of climate change literature, and indeed to our common understanding of this problem. The information in the TSD has therefore been developed and prepared in a manner that is consistent with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency.( U.S. EPA (2002), EPA/260R-02-008 url
Furthermore, relying most heavily on the assessment reports that reflect the scientific literature more broadly guards against an overreliance on and narrow consideration of individual studies.
In their TSD, they say:
This document relies most heavily on existing, and in most cases very recent, synthesis reports of climate change science and potential impacts, which have gone through their own peer-review processes including review by the U.S. Government. The information in this document has been developed and prepared in a manner that is consistent with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency. …
These core reference (Table 1.1) documents include the 2007 Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC), Synthesis and Assessment Products of the U.S. Climate Change Science Program (CCSP), National Research Council (NRC) reports under the U.S. National Academy of Sciences (NAS), the EPA annual report on U.S. greenhouse gas emission inventories and the EPA assessment of the impacts of global change on regional U.S. air quality. …
EPA is relying most heavily on these synthesis reports because they 1) are very recent and represent the current state of knowledge on climate change science, vulnerabilities and potential impacts; 2) have assessed numerous individual studies in order to draw general conclusions about the state of science; 3) have been reviewed and formally accepted by, commissioned by, or in some cases authored by, U.S. government agencies and individual government scientists and provide EPA with assurances that this material has been well vetted by both the climate change research community and by the U.S. government; and 4) in many cases, they reflect and convey the consensus conclusions of expert authors. Box 1.1 describes the peer review and publication approval processes of IPCC, CCSP and NRC reports. Peer review and transparency are key to each of these research organizations’ report development process. In compliance with the U.S. EPA’s information quality guidelines, this document relies on information that is objective, technically sound and vetted, and of high integrity.
EPA’s own guidelines define “influential” information as follows:
“Influential,” when used in the phrase “influential scientific, financial, or statistical information,” means that the Agency can reasonably determine that dissemination of the information will have or does have a clear and substantial impact (i.e., potential change or effect) on important public policies or private sector decisions.
This particular policy would obviously rise to being “influential information”. Such information must meet a higher degree of quality:
EPA recognizes that influential scientific, financial, or statistical information should be subject to a higher degree of quality (for example, transparency about data and methods) than information that may not have a clear and substantial impact on important public policies or private sector decisions. A higher degree of transparency about data and methods will facilitate the reproducibility of such information by qualified third parties, to an acceptable degree of imprecision. For disseminated influential original and supporting data, EPA intends to ensure reproducibility according to commonly accepted scientific, financial, or statistical standards. It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed. It is also important that the degree of rigor with which each of these factors is presented and discussed be scaled as appropriate, and that all factors be presented and discussed. In addition, if access to data and methods cannot occur due to compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections, EPA should, to the extent practicable, apply especially rigorous robustness checks to analytic results and carefully document all checks that were undertaken. Original and supporting data may not be subject to the high and specific degree of transparency provided for analytic results; however, EPA should apply, to the extent practicable, relevant Agency policies and procedures to achieve reproducibility, given ethical, feasibility, and confidentiality constraints.
In my own experience with IPCC and NRC, would I be able to issue an opinion that the various Team articles relied upon by the IPCC comply with EPA standards for “influential information::
It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed. It is also important that the degree of rigor with which each of these factors is presented and discussed be scaled as appropriate, and that all factors be presented and discussed.
The idea is laughable.
The unfortunate thing is that governments have to make decisions. Life moes on. As I’ve said before, if I were making a decision in Canada (under less rigorous Canadian law), I would be guided by the advice of major institutions regardless of what I might think as an individual.
What annoys me throughout is the prima donna behavior by so many climate scientist, in total disregard for how this misbehavior taints the decision-making process.
How can an NRC panel think that “just winging it” is an adequate way of discharging their duties? Did the panelists realize the legal weight that would be placed on their report? Or did they think of it just like any old journal literature review? (This is what I expect.) Did NRC or its President Ralph Cicerone make any effort to explain to the legal weight of their report and their responsibility to be diligent? I doubt it.
Or the IPCC and Met Office. When they refuse to release Ammann’s comments or Mitchell’s comments (in breach of their “transparency” obligations), do they give any consideration to the fact that this taints subsequent reports that rely on IPCC complying with its transparency obligations. And, of course, no one in the “community” cares or criticizes Ammann, CRU and IPCC.
Every time that Mann, Lonnie Thompson, Esper, Briffa or whoever stonewall a data request or methodology request, they taint the process.
I’m not suggesting that the tainting is sufficient to throw out the report, only that it leads to a very disappointing situation.
And it’s annoying to read declarations that IPCC and NRC reports satisfy rather severe EPA guidelines for “influential scientific, financial, or statistical information” satisfying a
higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed.
when our own experience is precisely the opposite.