The recent report of the EPA Office of Inspector General(OIG) contains a remarkable dispute between the OIG on the one hand and EPA and the Office of Management and Budget (OMB) on the other as to whether the Technical Support Document (TSD) for the Endangerment Finding was a “highly influential scientific assessment”, a defined category under OMB peer review policy. It would doubtless seem self-evident to most readers that, if any scientific assessment were to meet any criteria of being “highly influential”, the TSD for the Endangerment Finding would meet such criteria.
But readers should never under-estimate the capacity for institutional mendacity. The EPA and OMB have both vigorously argued that the TSD was NOT a “highly influential scientific assessment”. The OIG report includes a fascinating series of appendices in which EPA and OMB gradually articulate this seemingly improbable doctrine – a doctrine rejected by the OIG.
The dispute arises because the EPA peer review procedures did not meet U.S. standards for a “highly influential scientific assessment” – as clearly stated in the recent OIG report. Thus EPA and OMB have resorted to the improbable argument that the Endangerment Finding was not a “highly influential scientific assessment”.
I remind CA readers that the issue of whether EPA peer review procedures complied with federal policies for a “highly influential” scientific assessment (and with related EPA handbook policies) was first raised at CA shortly after the release of the Endangerment Finding – see tag and here. My submission to EPA directly addressed this topic and included a review of relevant authorities.
OMB Final Information Quality Bulletin
The term “highly influential scientific assessment” is defined in the OMB Final Information Quality Bulletin (Federal Register section III(1), page 11, as follows:
III. 1. Applicability: This section applies to influential scientific information that the agency or the Administrator determines to be a scientific assessment that:
(i) Could have a potential impact of more than $500 million in any year, or
(ii) Is novel, controversial, or precedent-setting or has significant
To meet the definition, either of the clauses suffices. It seems unarguable that the Endangerment Finding meets both clauses (i) and (ii). So how did EPA and OMB argue that the TSD for the Endangerment Finding was not a “highly influential scientific assessment”?
OIG Appendix B
The OIG examination of the EPA peer review process was prompted by questions from Inhofe (see Appendix B) which appear to have been submitted in July 2010.
For the record, until the publication of the OIG report, I did not know that Inhofe’s office had submitted any questions to the EPA OIG or that any investigation was taking place. Nor do I know whether Inhofe’s office was aware of the points raised in my 2009 submission or not. If they were, it was through reading Climate Audit rather than direct contact. Nor do I know whether the OIG was aware of my submission on these points. While the report discusses some issues in my submission, other important issues were unaddressed and I therefore presume that they did not consider my submission to EPA, despite it being on point.) Nor had the EPA addressed these issues in their Response to Comments in December 2009 (see here), where my submission had been ignored.
Appendix D – OIG Request
On August 11, 2010 (see Appendix D – OIG Request to OMB for Clarification of the Definition of a Highly Influential Scientific Assessment), the OIG informed OMB that:
the EPA OIG is currently evaluating EPA’s development of its endangerment and cause or contribute findings for greenhouse gases. Our objective is to determine whether EPA followed key federal and Agency regulations and policies in developing and reviewing the technical data used to support and make its endangerment finding …
Among other questions, they asked:
To what extent should a Federal agency review another organization’s data quality and peer review processes (e.g., document all processes, document and test processes, test selected processes, etc.) before disseminating information from a peer reviewed scientific assessment published by that organization?
Appendix E – Sept 2010 OMB Response
On Sept 10, 2010, (see Appendix E- OMB Clarification of the Definition of a Highly Influential Scientific Assessment, page 55), the OMB Assistant General Counsel provided fairly straightforward responses to the question:
If an agency uses another organization’s data or analysis to support their policy, they are disseminating that information. As such, that information becomes subject to the Agency’s Information Quality Guidelines and the Bulletin for Peer Review.
They also stated:
The second step is for the agency to determine whether a specific piece of information is subject to the Bulletin for Peer Review. Not every article cited in a preamble or risk assessment, for instance, is subject to the Bulletin. Only scientific information that is important to the conclusion being drawn by the agency is subject to the Bulletin. For information that is subject to the Bulletin, the agency must either conduct a new peer review of that information or determine that the prior peer review meets the requirements of the Bulletin for either a peer review or alternative process.
The OIG also asked:
Is a document that summarizes the results and conclusions of other peer reviewed scientific assessments, but offers no new analysis or conclusions, considered a scientific assessment according to OMB’s definition?
The OMB responded (reasonably):
An annotated bibliography would generally not be considered a scientific assessment; however, a document summarizing the ‘state of the science’ would be, as it implicitly or explicitly weighs the strength of the available evidence.”
Appendix F – Apr 2011 First OMB Response to Draft Report
In April 2011, the OIG provided the EPA and OMB with its draft report stating that the EPA had not complied with peer review requirements for a “highly influential scientific assessment”. In their April 15, 2011 response, the OMB Deputy General Counsel now argued that the Endangerment Finding was NOT a “scientific assessment” (see Appendix F, page 57):
With respect to the relationship between the Endangerment TSD and OMB’s Bulletin, the threshold question is whether EPA should have determined that the TSD met the definition of a “highly influential scientific assessment.” If so, the Endangerment TSD would have been subject to the stricter minimum requirements for peer review contained in Section III of the OMB Bulletin.
OMB believes that EPA reasonably determined that the Endangerment TSD itself (as opposed to the underlying peer-reviewed scientific assessments of the NRC, IPCC, USGRCP identified and discussed in the TSD) did not have the impacts or characteristics required to meet the OMB Bulletin’s definition of a highly influential scientific assessment.
If the document does not meet the definition of a “highly influential scientific assessment,” but meets the definition of “influential scientific information,” then it is subject to more discretionary requirements contained in Section II of the OMB Bulletin. OMB believes that the EPA complied with those requirements.
Already one can see the development of a fictional account of the EPA deliberation process. There is no evidence that, at the time, EPA had consciously made a decision that the Endangerment Finding was merely “influential scientific information” as opposed to a “highly influential scientific assessment”. Had there been such a decision, it would have been recorded in contemporary documents and it isn’t.
Appendix G – Jun 2011 EPA Response to Draft Report
On June 11, 2011, the EPA provided its own justification (see Appendix G – Agency Comments on Draft Report and OIG Evaluation of Agency Comments) which is accompanied by inline OIG comments in blue boxes. They also argued that the TSD was not a “scientific assessment” e.g. as follows:
A “scientific assessment” (a prerequisite for being a HISA [highly influential scientific assessment]) is defined in OMB’s Peer Review Bulletin as “an evaluation of a body of scientific or technical knowledge, which typically synthesizes multiple factual inputs, data, models, assumptions, and/or applies best professional judgment to bridge uncertainties in the available information.” The TSD did not conduct such an evaluation. No weighing of information, data and studies occurred in the TSD. That had already occurred in the underlying assessments, where the scientific synthesis occurred and where the state of the science was assessed. The TSD is not a scientific assessment, but rather summarized in a straightforward manner the key findings of the NRC, the USGCRP and IPCC. EPA is confident that a comprehensive review of this issue leads to the conclusion that the TSD is not a HISA, but rather ISI.
As to whether IPCC peer review procedures met EPA standards, EPA merely observed that the peer review processes of IPCC are “well known and accepted” by the U.S. government. In their inline response, the OIG observed that “the only organization for which OMB guidance specifically allows federal agencies to presume findings and conclusions to be adequately peer reviewed is NAS”.
All of Appendix G merits reading in full.
Appendix H – Jun 2011 OMB Response to Draft Report
On June 17, 2011, concurrent with the EPA response (Appendix G), the OMB submitted its formal comments on the draft report (Appendix H – OMB Comments on Draft Report and OIG Evaluation of OMB Comments), again with OIG comments in blue boxes. This letter contained the cutphrase later excerpted by EPA in its responding press release:
OMB believes that EPA reasonably interpreted the OMB Bulletin in concluding that the particular TSD that EPA prepared in this case did not meet the Bulletin’s definition of a “highly influential scientific assessment”:
The letter contains remarkable contortions trying to reconcile the Sept 10 OMB Response requiring (Section III) peer review with their present position that the TSD was not a “scientific assessment”.
They argued that the predecessor reports by IPCC and others were the “scientific assessments”:
Section 1(7) of the OMB Bulletin defines a “scientific assessment” as “an evaluation of a body of scientific or technical knowledge,” including “state-of-science reports.” In this case, EPA concluded that it was the separate, pre-existing and peer-reviewed assessments by IPCC, USGCRP, and NRC that constituted such evaluations of the state of the science.
…rather than requiring interested persons to read the entirety of these lengthy assessments – EPA included in the TSD a reader-friendly version of those passages (from those pre-existing peer-reviewed assessments) on which EPA was relying for making its determination.
The TSD accompanying EPA’s decision provided a condensed form of the three underlying peer- reviewed assessments and, with respect to the key conclusions in the Endangerment Finding, the TSD is in many respects simply a word-for-word transcription of the summary conclusions that are contained in those peer-reviewed assessments.
The OIG sensibly rejected these arguments and adhered to the obvious position that, the TSD was a “scientific assessment”:
However, in synthesizing the findings, conclusions, and other information from these assessment reports (and other sources) in its TSD, EPA was evaluating the state of science and producing an entirely new and separate document that also met OMB’s definition of a “scientific assessment.”
They rejected the idea that the TSD was simply a “reader friendly version” of the earlier assessments:
Nowhere in the TSD does it state that the purpose of the document is to “provide a reader friendly version” of the underlying assessments.
The OIG stated:
EPA classified its endangerment and cause or contribute findings for greenhouse gases as a Tier 1, significant regulatory action because it raises novel policy issues. The TSD provided scientific and technical information to support that action. For that reason, and because the TSD synthesizes information from multiple sources (beyond just those of the USGCRP, IPCC, and NRC assessments), the TSD itself should be considered a highly influential scientific assessment, subject to the applicable OMB peer review requirements for that type of information.
The OIG also observed that EPA did not take the position that the TSD was merely “influential scientic information” during the review process itself, but first advanced this argument (opportunistically) in response to the OIG review:
EPA did not characterize the TSD as influential scientific information (or as a highly influential scientific assessment) during the action development process. EPA first characterized the TSD as influential scientific information in response to our draft report.
The OIG report was issued on Sept 28, 2011. In its press release, EPA took comfort in OMB support for the seemingly improbably claim that the TSD was not a “highly influential” scientific assessment:
OMB in response to our draft report stated that OMB believes that EPA reasonably interpreted the OMB bulletin in concluding that the TSD did not meet the bulletin’s definition of a highly influential scientific assessment.
The reaction of the climate scientists (as quoted by Pielke Jr) were that EPA’s failure to comply with its procedures for a highly influential scientific assessment didn’t matter, because the “science” was right.
On the other hand, it seems to me that this can hardly be the end of the story in legal terms, as there must surely be legal ramifications to EPA’s present argument that they did not observe required peer review protocols because they had decided that the TSD was not a “highly influential scientific assessment”.
As I’ve observed in the past (and specifically in connection with my own EPA submission), in Canada, there would not be an equivalently tortuous regulatory process. I do not contest the right of governments to make decisions on these matters. I do object to what I described in my original post as “the implicit laundering of past stonewalling and obstruction” by IPCC authors in respect to data and methods and to structural defects in the IPCC review process. This criticism was written six months before Climategate provided further confirmation of the tainted IPCC review process. (In respect to the issue that particularly concerned me, the assessment of the Hockey Stick dispute, the Climategate emails e.g. the surreptitious Wahl-Briffa exchange of summer 2006 that Jones asked to delete, that the IPCC review process did not meet EPA standards.
As I’ve observed in the past, rather than breaching important policies and then arguing that the breaches don’t “matter”, climate scientists should ensure that all processes, including IPCC, are truly open and transparent. If this is necessary to comply with EPA standards, that’s simply one more reason for something that should be done anyway.
If EPA were to carry out a highly influential scientific assessment on its own, then proponents would have to turn over data and methods. IPCC does not have corresponding requirements. Indeed, they threatened to expel me as an AR$ reviewer for asking for data. The pernicious practice of data withholding continues (as in the ongoing FOI requests to CRU for the secret Yamal/Urals regional chronology.)
IPCC should ensure that its policies to remove every possible non-compliance with EPA standards. Their failure to implement the conflict of interest policy for AR5 is a mistake.