The U.S. EPA just released (Apr 17, 2009) “Proposed Endangerment and Cause or Contribute Findings for Greenhouse Gases Under Section 202(a) of the Clean Air Act” url together with a Technical Support Document url. In Canada and most countries, governments just implement these sorts of policies without the huge regulatory process that delays everything in the U.S. (including nuclear plants.) So I’m not opposed to governments making decisions, even if I don’t agree with the decision.
The EPA said that it primarily relied on prior due diligence by IPCC, CCSP and the US National Research Council, taking the seemingly plausible position that these reports complied with EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Informationurl.
The present report is clearly “influential information” under EPA policies. EPA has legislative guidelines for the quality of “influential scientific, financial, or statistical information”, while I’m unaware of corresponding guidelines for NRC reports and IPCC reports (and perhaps CCSP reports). While EPA seems to take it as a given that the due diligence for these reports is sufficient to comply with legislated EPA guidelines, my own experience with NRC and IPCC due diligence is not totally reassuring. This experience has been discussed from time to time over the past few years at Climate Audit. I’ll link to a few relevant threads, but there are many other disappointing incidents.
NRC Report on Surface Temperature Reconstructions
The NRC Report on Surface Temperature Reconstructions is cited in the EPA Technical Support Document. In a CA thread here, I quoted comments by the panel chairman, Gerry North, in which he stated that they “didn’t do any research”, that they got 12 “people around the table” and “just kind of winged it.” He said “that’s what you do in that kind of expert panel”. A clip of North’s remarks is online here. This sort of casualness offended me at the time as being inconsistent with expectations of legislators who reasonably expected a bit more seriousness.
This casualness was further exemplified by North’s response to my online question to him during a Colloquy (threaded at CA here)
Question from Stephen McIntyre:
The NRC Panel stated that strip-bark tree forms, such as found in bristlecones and foxtails, should be avoided in temperature reconstructions and that these proxies were used by Mann et al. Did the Panel carry out any due diligence to determine whether these proxies were used in any of the other studies illustrated in the NRC spaghetti graph?
North’s answer was as follows:
There was much discussion of this matter during our deliberations. We did not dissect each and every study in the report to see which trees were used. The tree ring people are well aware of the problem you bring up. I feel certain that the most recent studies by Cook, d’arrigo and others do take this into account. The strip-bark forms in the bristlecones do seem to be influenced by the recent rise in CO2 and are therefore not suitable for use in the reconstructions over the last 150 years. One reason we place much more reliance on our conclusions about the last 400 years is that we have several other proxies besides tree rings in this period.
After the NAS panel said that bristlecones should be “avoided” in reconstructions and North here saying that they are “not suitable for use in the reconstructions over the last 150 years”, the NRC used bristlecones in their spaghetti graph, which is now carried forward into the EPA Technical Support Document.
IPCC
I’ve observed on many occasions that IPCC does not itself carry out any due diligence. A clear statement of this occurs in Mann’s 2003 answers to questions from Inhofe (noted up at CA here). There were a series of questions, starting with:
30. Did IPCC carry out any independent programs to verify the calculations that you made in MBH98 or MBH99? If so, please provide copies of the reports resulting from such studies.
Mann:
It is distinctly against the mission of the IPCC to “carry out independent programs”, so the premise of the question is false. However, the IPCC’s author team did engage in a lively interchanges about the quality and overall consistency of all of the papers as the chapter was drafted and revised in the course of review.
Whether a “lively interchange about the quality and overall consistency of all of the papers” is sufficient to comply with EPA guidelines to ensure and maximize the quality, objectivity, utility and integrity of information is surely a question that merits attention by somebody.
The supposed “transparency” of the IPCC process – pointed to by EPA in support of reliance on IPCC reports – has been the topic of many threads here as we follow the lugubrious stonewalling by IPCC and UK agencies, particularly with regard to Ammann Wahl’s secret comments to IPCC and Review Editor Mitchell’s secret review comments. IPCC principles state clearly:
All written expert, and government review comments will be made available to reviewers on request during the review process and will be retained in an open archive in a location determined by the IPCC Secretariat on completion of the Report for a period of at least five years.
However, Caspar Ammann’s co-author, Eugene Wahl, in the US submitted secret comments about IPCC AR4, which he, CRU and IPCC have collectively refused to disclose. Most recently, CRU stated in response to an FOI request:
In regards the correspondence from Mr. Ammann, s.41 is applicable as we have consistently treated this information as confidential and have been assured by Mr. Ammann that he believes it to be confidential and would expect it to be treated as such. The public interest in withholding this information outweighs that of releasing it due to the need to protect the openness and confidentiality of academic intercourse prior to publication which, in turn, assures that such cooperation & openness can continue and inform scientific research and debate.
“Confidentiality of academic intercourse prior to publication” may be OK for journal publication, but IPCC reports are not journal publications and IPCC policy required that Ammann and/or Wahl submit on-the-record comments.
AR4 Chapter 6 Review Editor John Mitchell has likewise refused to disclose his review comments, providing a bizarre sequence of untrue prevarications. First, he said that he had destroyed all the review comments on the basis that he had no obligation to preserve them (in face of express IPCC regulations otherwise); then he said that they were his “personal” property. When he was asked whether the Met Office had paid for his time and travel to IPCC meetings, a new excuse merged. Disclosing the comments would interfere with relations with an exempt sovereign international institution (IPCC), which appears to me to be exempt from any FOI anywhere in the world and which refused to permit the Met Office to disclose Mitchell’s review comments.
EPA Procedures
So let’s see what EPA says about its reliance on these reports. The draft finding states:
EPA has developed a technical support document (TSD) which synthesizes major findings from the best available scientific assessments that have gone through rigorous and transparent peer review. The TSD therefore relies most heavily on the major assessment reports of both the Intergovernmental Panel on Climate Change (IPCC) and the U.S. Climate Change Science Program (CCSP). EPA took this approach rather than conducting a new assessment of the scientific literature. The IPCC and CCSP assessments base their findings on the large body of many individual, peer reviewed studies in the literature, and then the IPCC and CCSP assessments themselves go through a transparent peer review process. The TSD was in turn reviewed by a dozen federal government scientists, who have contributed significantly to the body of climate change literature, and indeed to our common understanding of this problem. The information in the TSD has therefore been developed and prepared in a manner that is consistent with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency.( U.S. EPA (2002), EPA/260R-02-008 url
Furthermore, relying most heavily on the assessment reports that reflect the scientific literature more broadly guards against an overreliance on and narrow consideration of individual studies.
In their TSD, they say:
This document relies most heavily on existing, and in most cases very recent, synthesis reports of climate change science and potential impacts, which have gone through their own peer-review processes including review by the U.S. Government. The information in this document has been developed and prepared in a manner that is consistent with EPA’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility and Integrity of Information Disseminated by the Environmental Protection Agency. …
These core reference (Table 1.1) documents include the 2007 Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC), Synthesis and Assessment Products of the U.S. Climate Change Science Program (CCSP), National Research Council (NRC) reports under the U.S. National Academy of Sciences (NAS), the EPA annual report on U.S. greenhouse gas emission inventories and the EPA assessment of the impacts of global change on regional U.S. air quality. …
EPA is relying most heavily on these synthesis reports because they 1) are very recent and represent the current state of knowledge on climate change science, vulnerabilities and potential impacts; 2) have assessed numerous individual studies in order to draw general conclusions about the state of science; 3) have been reviewed and formally accepted by, commissioned by, or in some cases authored by, U.S. government agencies and individual government scientists and provide EPA with assurances that this material has been well vetted by both the climate change research community and by the U.S. government; and 4) in many cases, they reflect and convey the consensus conclusions of expert authors. Box 1.1 describes the peer review and publication approval processes of IPCC, CCSP and NRC reports. Peer review and transparency are key to each of these research organizations’ report development process. In compliance with the U.S. EPA’s information quality guidelines, this document relies on information that is objective, technically sound and vetted, and of high integrity.
EPA’s own guidelines define “influential” information as follows:
“Influential,” when used in the phrase “influential scientific, financial, or statistical information,” means that the Agency can reasonably determine that dissemination of the information will have or does have a clear and substantial impact (i.e., potential change or effect) on important public policies or private sector decisions.
This particular policy would obviously rise to being “influential information”. Such information must meet a higher degree of quality:
EPA recognizes that influential scientific, financial, or statistical information should be subject to a higher degree of quality (for example, transparency about data and methods) than information that may not have a clear and substantial impact on important public policies or private sector decisions. A higher degree of transparency about data and methods will facilitate the reproducibility of such information by qualified third parties, to an acceptable degree of imprecision. For disseminated influential original and supporting data, EPA intends to ensure reproducibility according to commonly accepted scientific, financial, or statistical standards. It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed. It is also important that the degree of rigor with which each of these factors is presented and discussed be scaled as appropriate, and that all factors be presented and discussed. In addition, if access to data and methods cannot occur due to compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections, EPA should, to the extent practicable, apply especially rigorous robustness checks to analytic results and carefully document all checks that were undertaken. Original and supporting data may not be subject to the high and specific degree of transparency provided for analytic results; however, EPA should apply, to the extent practicable, relevant Agency policies and procedures to achieve reproducibility, given ethical, feasibility, and confidentiality constraints.
In my own experience with IPCC and NRC, would I be able to issue an opinion that the various Team articles relied upon by the IPCC comply with EPA standards for “influential information::
It is important that analytic results for influential information have a higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed. It is also important that the degree of rigor with which each of these factors is presented and discussed be scaled as appropriate, and that all factors be presented and discussed.
The idea is laughable.
The unfortunate thing is that governments have to make decisions. Life moes on. As I’ve said before, if I were making a decision in Canada (under less rigorous Canadian law), I would be guided by the advice of major institutions regardless of what I might think as an individual.
What annoys me throughout is the prima donna behavior by so many climate scientist, in total disregard for how this misbehavior taints the decision-making process.
How can an NRC panel think that “just winging it” is an adequate way of discharging their duties? Did the panelists realize the legal weight that would be placed on their report? Or did they think of it just like any old journal literature review? (This is what I expect.) Did NRC or its President Ralph Cicerone make any effort to explain to the legal weight of their report and their responsibility to be diligent? I doubt it.
Or the IPCC and Met Office. When they refuse to release Ammann’s coauthor’s comments or Mitchell’s comments (in breach of their “transparency” obligations), do they give any consideration to the fact that this taints subsequent reports that rely on IPCC complying with its transparency obligations. And, of course, no one in the “community” cares or criticizes Ammann, Wahl, CRU and IPCC.
Every time that Mann, Lonnie Thompson, Esper, Briffa or whoever stonewall a data request or methodology request, they taint the process.
I’m not suggesting that the tainting is sufficient to throw out the report, only that it leads to a very disappointing situation.
And it’s annoying to read declarations that IPCC and NRC reports satisfy rather severe EPA guidelines for “influential scientific, financial, or statistical information” satisfying a
higher degree of transparency regarding (1) the source of the data used, (2) the various assumptions employed, (3) the analytic methods applied, and (4) the statistical procedures employed.
when our own experience is precisely the opposite.
![Reblog this post [with Zemanta]](https://i0.wp.com/img.zemanta.com/reblog_e.png)
Climate Audit 
55 Comments
snip – too much editorializing
snip –
Folks, unless you are prepared to read the various guidelines and comment technically on compliance issues in regulatory terms, please save your breath.
I’m going to snip and delete venting and angry outbursts.
Re: Steve McIntyre (#3),
Steve,
Like it or not, I don’t think you can really suggest that your EPA posting is merely about issues of compliance or regulatory guidelines and that comments should address those aspects only. The cat is out of the bag for everyone to see, I’m afraid.
snip
Re: Steve McIntyre (#3),
You have seen more versions of global temperatures than most people and indeed I think you even question the meaning of a global temperature, as do I. I seek guidance. If I were to make a submission to the EPA, which has a link that goes gas emission > climate change including temperature > human health, which graph would you use to demonstrate temperature change in the instrumental period?
This is a problem that the EPA must take on if it is to be ethical in its action. It is so easy for them to choose a convenient one, so maybe they should be steered to one that has reasons for justification.
It’s a snip trap for me isn’t it?
I’m surprised by this. Treating the IPCC, UNRC ,EPA…on and on like they don’t have a card in the game is not a ……’useful’ proposition. There are reasons these organizations are not thorough and have so often ignored your points. There’s a reason it’s a bad career choice for Ammann to work with you.
Steve: Jeff, I realize that you’re relatively new to this snake pit, but I’ve stated this position on many occasions in the post. It’s what I think. It’s also a position that I don’t want to spend a lot of time debating as my view on the obligations of a policy maker really doesn’t “matter”.
Re: Jeff Id (#4),
I understand your point. Without too much brown nosing, I just think a lot of people respect your scientific views above those of many others. The openness of it leads to an understanding which is as we’ve seen, often superior to the work promoted by those other organizations. As a test, put an error up sometime and see how long it sits unnoticed. You’d probably get a personal phone call from Gavin within the hour. Yet I still see hockey sticks and fantastic confidence intervals in that technical paper linked above.
They have some proxy based reconstructions in the technical document above which have the same basic shape as the Mann08 CPS method produced. Several of them had to come from similar math just by the shape. Also, the temperature plot has trend lines which are meant to imply to the observer increasing trends. On CA I learned what created the Mann08 reconstruction, I also learned quite a bit from the Santer posts, although I don’t think I left a single comment on any of them I spent hours on it. Antarctic stuff is still in progress but I know what happened to Antarctic temps now and the real trend is likely under half Steig et al. Hurricanes haven’t increased and that’s leaving out a dozen other items that I’ve learned on the threads here. The policy makers clearly don’t have the best quality data in this document. So I and others here wonder how come?
I don’t think they’ve done a fair job of following their policy.
Steve,
Too polite by a mile as usual, but then again politeness is always a virtue.
snip
I am sure there will be a good number of organizations in the US that will argue along the lines of your demonstration above during the 60 day hearing period that now follows. To what effect remains to be seen.
Seems like guidelines and recommendations are being ignored. Perhaps sending a list of guideline and recommendation violations to the necessary politicos, with the violations clearly described would get some of these matters cleared up ?
I have believed for some time that the requirements for “information quality” could be the Achilles heel of US government AGW policy. All the departments and agencies of government are required to have guidelines similar to those of EPA outlined in the post above. NASA’s guide lines include requirements for all information include a requirement for “lack of bias”, Specifically,
snip- I’m getting OT and at risk of a snip.
With respect to “influencial information” NASA requirements state (and I’ll bet EPA’s are very similar) that information must meet strict standards for “reproducibility”, specifically:
How hard would it be to prove that critical elements of the information upon which EPA is relying its policies can not meet these standards?
snip
Steve, can I ask you in most polite and non-angry manner, what is the basis for your assertion that decision maker should follow advice of “major institutions” even if he is sure that that advice is demonstrably wrong or politicized? The final consequence of this view would be that you as a Canadian decision maker would follow advice of Michael Mann, instead of Steve McIntyre, even if you believe that McIntyre, and not Mann is right about paleo-reconstruction. That’s not make much sense to me. At the contrary, it seems that “major institutions” have very strong vested interests in creating panic nad overstating dangers to sustain their own funding and prestige. I would as policy maker believe them IN PRINCIPLE much less than individual scientists.
Steve: In my opinion, the obligation of a policy maker is to do what he can to improve the performance of the institutions, not to substitute his own scientific judgment for theirs. I’m not interested in debating this further.
What Steve leaves un-said about Canada’s system is that any government with a parliamentary majority (unlike the current one) can repeal regulations as easily as they are adopted.
The process in the US to repeal a regulation is as cumbersome as the process that adopts one. Rules in the US take years to adopt. That is why after 8 years of the Bush administration very few rules adopted under the Clinton administration were changed.
Repealing a law in the US is also much more difficult. Unlike Canada, having a majority in the Congress of the same party as the administration is a much rarer occurrence. Also the parties in the US aren’t as unified as they are in Parliamentary systems, where the loss of a single vote can bring down the government.
Re: deadwood (#10), you say:
Not so. In my post, I said:
I’m far from convinced that the American system makes any sense here. It seems to me that a government should be able to implement policies without going to court (as they can in Canada.)
However, having been stonewalled on so many occasions, it grates me for the system to be characterized as meeting EPA standards for data and methodology transparency.
Re: Steve McIntyre (#11),
Steve,
I think the statement that you just made is precisely why the US system operates as it does. It prevents knee-jerk reactions to “reports, publications, etc”, that otherwise might be implemented without due diligence. Policy makers are always on the lookout to do “something” to prevent some calamity or othersuch. There have been very few issues, if any, that absolutely required immediate action. And in such cases, the machinery can move fast enough. Otherwise, it prevents exactly the sort of thing that can occur in other countries, to the detriment of the populace and/or businesses associated with the policy change. This is another reason why the EPA is required to do economic impact analysis of the policy changes as well, so that full disclosure of the costs can be made public. This allows the public to the decide if they are willing to pay the costs for the proposed benefits.
Re: Steve McIntyre (#11),
I think, maybe, you misunderstand the American political system – and why your post is so important.
The rules on making rules which can involve the courts and our Constitution exist to protect unpopular minorities [racial, political. regional, et Al.] from a tyrannical majority.
Whether this is good or not, you’re wrong in thinking our government can’t implement policies without going to court.
The CONGRESS can write a greenhouse gas law whenever it wants. As in Canada, or other parliamentary governments, these policymakers are elected by the people and must answer to them. For better or worse, Congress has opted not to write such a law.
The EPA, and other agencies, are not elected policymakers. These are civil servants, unaccountable to the people and with near life-tenure under our civil service laws. There is ZERO provision for these agencies under our Constitution – they have ZERO inherent power to make rules and regulations.
Foreigners [and most Americans…] seem to have trouble understanding that the President doesn’t have the power to make laws. He appoints agency heads; through them he can speed up or slow down rule-making. His job is to implement the will of Congress.
The Congress created EPA and delegated power to it, with the provison that EPA act within certain guidelines [Administrative law – i.e., the Administrative Procedures Act, FOIA].
So, whether EPA’s actions meet IPCC’s or Nature’s standards are irrelevant; whether they fail to meet EPA’s standards is relevant; but, if they fail to meet Congress’s proscribed standards the rules must fail in a court challenge. The agencies can’t use more power than Congress allocated to them. ALL national law-making authority flows from Congress.
Congress pays EPA and its scientists to study an issue and come to an independent conclusion, not to simply defer to the President, Exxon, the UN, or another third party. If EPA hasn’t done their own legwork, their finding will fail in court for arbitrariness.
Popularly elected policymakers are allowed to be as arbitrary as they wish in the US, civil servants are not.
Re: jim edwards (#20),
Mr. Edwards is precisely correct that EPA has a obligation to do their homework in taking this administrative action.
This is the point in time when there is an opportunity to identify all the shortcomings of the EPA decisional document through public comment. Thus, to the extent that EPA relies on faulty IPCC involving bristlecones, that needs to made a matter of record.
Similarly, if the studies relied upon by EPA fail to meet quality assurance standards, that defect must be noted in a submitted comment.
Court challenges to the EPA action may take some time to occur, as the EPA has proposed no regulations as yet, but they will occur.
I am have very little expertise on climate science or statistics, but am prepared to file comments on the defects in the process, with appropriate acknowledgement to Dr. McIntyre. The comment cycle is fairly short — I believe the period ends in mid-May. The comments submitted can have a significant effect on how this matter unfolds in the judicial review process. I urge all CA readers to get involved.
Re: Steve McIntyre (#11),
Steve,
You snipped key parts of by previous two comments. As always, your blog, your prerogative. There was neither venting nor anger nor heavy political comment in either of them.
Comments 11, 15, 18 and in particular 20 essentially argue what I posted in my # 5 and # 9. Maybe you don’t like tetris’ comments for whatever reason. Pls have another look at # 20 and maybe by way of simple courtesy paste back in what you snipped out of #9.
We all understand that you feel “grated” [ref your comment @ #11] at being stonewalled for years. That said, there is a fine line between snipping for venting, anger, etc. which I’m fine with and for which my comments certainly do not qualify and you blindly walking your blog into a trap very similar to the one in effect at say RC.
Pls have another look at what I posted. It is simple common sense comment about the politics of the EPA process.
Keep up the good work.
.
Some background, clarification and rationale from the US Office of Management and Budget (OMB) concerning these guidelines.
.
Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies
.
The problem is that everyone wants to ascribe an engineering-quality certainty to the body of work about climate, but no one actually wants to spend the money or do the work to make that happen.
What the EPA should have done is to commission an independent body of scientists, engineers, and statisticians to evaluate the data, methods, and conclusions of the most important papers in a much more detailed fashion than peer review allows. Until this is done, a proper risk/benefit analysis cannot be done, and, thus, the optimal mitigation efforts cannot be determined.
Quite honestly, this is something that should have been done quite some time ago.
A great real-world example of how groupthink and failure to use independent analysis can lead to disastrous results is here:
http://www.nasa.gov/columbia/caib/html/start.html
This is the report of the Columbia Accident Investigation Board. It’s an easy read, but also very detailed and specific. I would recommend it to anyone here. While not directly related to climate science, I think anyone who reads it would agree that the thought process used to justify continuation of poor practices and to explain away inconsistencies is remarkably similar to what Steve posted above.
The failings of NASA described in the report are nearly universal to all large, politicized organizations. I’ve actually used the report extensively (including the video) for doing company-wide training sessions on the importance of validation/verification and process control.
Anyway, just thought I’d recommend it. It’s a good read, and if you don’t end up seeing strong parallels between what happens in the report and the IPCC or EPA, I’ll eat my shirt. An important lesson to take away, too, is an understanding of why this happens – and the answer is not deliberate malfeasance (despite what the outward appearance might be).
Re: Ryan O (#13),
Ryan, IMO, it’s more that the people involved have no idea what an “engineering-quality” study looks like. Responses by climate scientists to my long standing request are always to tiny little articles, which, however meritorious they may be as articles, look nothing like engineering documents and merely demonstrate that the climate scientist in question doesn’t have a clue about what engineering does. Computer programming readers (of whom there are a lot) don’t help much in the explanation because they veer off into criticism of lousy and undocumented code, which is an issue, but a different issue.
Re: Steve McIntyre (#19), WRT the climate scientists, you’re probably right. I imagine the same could be said for most pure research fields. When it comes to the EPA, however, they should have no excuse for the same behavior. As an organization, the EPA knows full well what an engineering quality study looks like – as do many government agencies.
Re: Steve McIntyre (#19),
When I read this, I thought of very large scientific projects such as the Large Hadron Collider. Aside from the accelerator itself there are a number of detectors each of them costing tens of millions of dollars. The planning of such a facility must take into account the types of experiments that are possible and the cost of the infrastructure that is needed to perform them. This is essential so that decision makers can select those experiments that are of sufficient value to warrant the massive expenditure required to support them. From those decisions would flow the engineering plans for a multi-year construction cycle. For this to go ahead with any chance of success full scale engineering plans must have been drawn up. In the case of the LHC, there was a major mistake in the design of the superconducting magnets that cause an explosion when they were first powered up. The release of helium was of such extend that the building had to be evacuated. This is an example of why adequate engineering studies are necessary for such a project.
For my reading here, climate science projects are miniscule compared to the LHC, the Hubble telescope, multiple Mars launders etc. However the capacity for doing proper engineering planning must exist in the scientific community. To build the large physics experiments that they do, such capability is essential. The IPCC with its plans to reshape the world economy cannot plead ignorance of engineering planning capability. Such capabilities must be widespread among physicists, astronomers etc. If the IPCC chooses not to perform the necessary planning, it not for lack of capability in the physics community.
snip – venting
Rep. Edward J. Markey is going to hold four days of climate change hearings next week. Now that we’re playing for keeps we can only hope that the transparency problems that Steve and others have encountered will get aired therein. In addition to the ones Steve lists above, the bizarre story of the Ammann “Lazarus” paper needs to be told and heard. It’s a damning example of how the IPCC’s own rules were at best circumvented, and at worst, subverted.
Maybe this new finding will actually lead to more transparency and cooperation. The lives of hundreds of millions of people are going to be significantly affected now, so no matter what your opinion on this subject you should want and demand full transparency. To me this loophole is unacceptably large:
All an uncooperative scientist has to do is allege that “privacy, trade secrets, intellectual property” would be compromised and then still hide data and methods from scrutiny. Worse, the can just throw up vague, unspecified “other confidentiality protections” and it’s business as usual: No data or code for you. We need to demand that any paper for which all of the data and code is not available for any reason be excluded from any report that is going to be used as the basis for policy and rule-making.
We need this protection from not only claims of “confidentiality” but also to prevent the kind of “dog ate my homework” excuse that Steve highlights in his previous post.
Re: Not sure (#17),
I don’t think that this “loophole” is “the” problem here simply because the studies and analyses were done under different due diligence regimes (with much lower standards) where stonewalling was tolerated.
To my knowledge, this post is the first time that anyone’s pointed out the inconsistency between EPA transparency requirements and the reliance on studies from stonewalling regimes.
Re: Not sure (#17), You said:
From the OMB guidelines …
Re: James Erlandson (#24),
That doesn’t make any sense. How can you possibly say anything about the validity or “robustness” of any study when you can’t get the same results from the same data using the same methods? All you’re doing is discussing the validity of a hypothetical study that using hypothetical data and methods. Whatever the merits of this academic endeavor, I don’t see how it can justifiably be used as the groundwork for regulations that will adversely affect the lives of hundreds of millions of people.
.
Reproducibility is such a ridiculously low standard that I just can’t see how it imposes any kind of significant burden on researchers. Gone are the days when med student Michael Crichton had to spend hours photocopying data to mail off to interested researchers. Nowadays firing off an email with attachments takes minutes at most. There are many publicly available online archives for data sets too large to email.
.
Reproducibility is also essential to any kind of validity analysis. How can you tell the effect of a few proxies in a study if you can’t reproduce the study? This goes to the heart of what Steve calls “stability” and what I think the Team would call “robustness.” Do the results of the study change dramatically if the data is changed only slightly? You simply can’t even begin to answer that question if you can’t replicate.
.
The rest of the paragraph you quote is just bureaucratic rationalizations to justify whitewash by blue-ribbon panel a la NAS panel. All the EPA has to do is assemble a few eminent pro-AGW scientists to just “wing it” and wave their hands over the study and voila! It is rigorous and robust.
Promulgated regulations can be litigated in a way in which laws cannot be. Some should be preparing to be expert witnesses in the inevitable court cases which will come if EPA regulations rather than Congressional action is the way in which the US government proceeds with carbon encumbering.
=========================================
Steve wrote:
Steve, you have expressed something extremely important here. I hope you don’t snip me for quoting you, but “total disregard” for the impact of one’s “misbehavior” is serious stuff in any profession or trade. And since, as you wrote, it “taints” the process, it ought to be the focus of a serious review. Professional misbehavior always impeaches the credibility of the experts to some extent, especially when the stakes to innocent people are large as they are in the present case.
This is more than a matter of mere “annoyance”. To repeat, the scientists at issue have engaged in behavior which impeaches their credibility. Were they testifying as experts in a jury trial, it would be wholly appropriate to point out to the jury that the cavalier attitude toward their ethical responsibilities made them untrustworthy. That is an obvious conclusion. For some reason, you don’t want your readers to make this observation in comments, even though it is a conclusion which flows naturally from the facts and your characterization of them.
At the most basic level, you are saying that these scientists are in breach of a critical aspect of the scientific method. When scientists do not follow the scientific method, their conclusions are unworthy of respect. That’s the case that you have made. Snipping your readers when they say “four” after you have added 2 and 2, doesn’t avoid the conclusion that you have added to four.
Steve: There are many situations where behavior may place a study outside EPA standards for “influential information” without being academic misconduct or necessarily “impeaching their credibility”. Too many readers are always trying to pile on without considering nuances and I get impatient with this.
As a follow on to Jim Edwards (#20) EPA is strictly limited in what it can and can not do:
EPA has started the official process of writing the regulations in the case of CO2 = pollution under the Clean Air Act. The way we got to the current situation is Massachusetts sued the EPA claiming the CO2 was causing global warming and that it constituted a threat to public health. The US Supreme Court agreed that CO2 could be classified as a pollutant and therefore was now eligible to fall under the Clean Air Act. The new top level administrators appointed by the Obama administration have aggressively pushed to start the formal process (public comment) required for setting the rules for regulating CO2. The point is that all this could be changed if Congress passes tomorrow a one paragraph law that effectively states “Under the Clean Air Act CO2 can not be classified as a pollutant”. At that point the whole problem goes away because EPA can only write regulations to existing laws.
Everyone should participate in the public comment process and indeed raise all the points mentioned above, however great pressure should also be put on Congress to rewrite the Clean Air Act.
I’ve got twenty bucks that says the EPA will do exactly what it wants to, regardless of whatever comments are received.
snip – moralizing
Steve: You can snip this, but what I wrote was not moralizing. It was my assessment of what I thought was going to happen. The EPA will change nothing unless and until a court orders it.
Steve: just take it easy, OK?
OK
This is somewhat tangential, but here is what the specter of climate change hath wrought in California:
http://info.sen.ca.gov/pub/07-08/bill/sen/sb_0351-0400/sb_375_cfa_20080818_153416_asm_comm.htm
In it you will find the same kind of thinking that went into the EPA finding.
Link was incomplete by a letter. Try this:
http://info.sen.ca.gov/pub/07-08/bill/sen/sb_0351-0400/sb_375_cfa_20080818_153416_asm_comm.html
snip – OT
From the EPA document cited in the opening –
Puts Daimler back in history a few centuries.
It’s petty, I know. However, it relects a poor understanding of the exactitude that important scientific documents need. In turn, this refects on a general inability of the EPA to understand science.
America sneezes again and Australia catches a cold. You USA scientists have an urgent duty to prepare your submissions.
Very meaty post Steve and food for thought. Worth reviewing this site too for legal opinion on the EPA strategy and more links.
Counted 6 skeptical articles in The Australian on the weekend. I’ll scan them up and make a post on it. Would be great to get a feature based around your post in that paper Steve – very big readership. I might have a go myself if I get a chance, and it works out.
Re: David Stockwell (#36),
David – yes, The Australian gave some space to Ian Plimer et al at last. But typically, the SMH and ABC simply ignored it all. Interesting dichotomy.
Can the EPA be sued? If it makes regulations based on conclusions that could and should have been regarded as dubious at the time, and these regulations cause financial or other damage, can aggrieved parties sue if those conclusions are later repudiated?
Can legislators render agencies like EPA immune from such suits? (Either before or after the agency acts?)
“…like a circle in a spiral
Like a wheel within a wheel.
Never ending or beginning,
On an ever spinning reel”
Everyone just assumes that someone else has done the legwork because – well they are all reputable people after all. But it’s one big lazy loop from a society that only ever reads the bottom line.
I predicted that North’s report would be sloppy and ambiguous and I underestimated how bad it was. The NRC Report was an exercise in the ancient magical arts of speaking out of both sides of one’s mouth, a practice usually reserved for White House press secretaries.
And yes, I would say that to his face…
Folks, please stop simply piling on.
If Steve had chosen from the beginning only to make public his efforts, together with Ross and others, in the peer reviewed journals, and even assuming a portion of it had been accepted, my guess is that the majority of individuals, including many of the scientists, who visit this site on a regular basis, would never have heard of him or his work. As an attorney, I certainly never would have. Think what would have been lost.
What effect would it have had on the Team or the IPCC? Probably nothing. As it is, his work has been and is being widely disseminated and a hell of a lot of people know about it, including the climate science community. I think he made an extremely wise choice.
Per usual, I think that Steve M points to US government agency rules and apparent requirements for handling information that somehow seems not to work as prescribed when tested by him.
Jim Edwards per usual gives an excellent view of how, ideally, our US system of government is supposed to handle these issues.
Re: jim edwards (#20),
However, I think when Jim Edwards says the following:
One must condition all this with how the laws were originally written which is usually with enough vagueness not to hamstring the regulating agency and to allow them room for decision (arbitrary within the law and sometimes otherwise) making without having to frequently go back to congress or to the courts. Another condition would necessarily involve the judgment in a court of law and who is doing the judging. A third would be the wherewithal that a complainant has to pursue a supposed wrong. A single person would be at a distinct disadvantage and particularly one where the case appears to espouse an unpopular cause that would not interest groups such as the ACLU or GreenPeace or other groups that take on such cases.
After all is said and done, regulation in the US can be arbitrary and arbitrarily applied and with held.
Not to be cynical, but realistic, regulating agencies rules for information handling and even open discussions for impending regulations are as much for appearances as they are for substance.
Re: Kenneth Fritsch (#43),
Thanks, and I don’t disagree with you that regulations are arbitrary in effect. As with statutes, there is no requirement that regulations make practical sense. Most don’t, I’m sure we’d agree. The courts also give deference [“Chevron deference”] to agencies, when they use their discretion to come to a determination.
My basic point is discretion requires some work, however.
EPA can come to the ‘wrong’ conclusion after reviewing the evidence. The courts have and will uphold absurdly bad regulations. Agencies can’t just flip a coin or refuse to look at contrary evidence that’s been presented to them – that doesn’t count as the proper use of discretion and the courts will not defer to that. That’s why the EPA lost in the Supreme Court last time.
What Steve has spotlighted is that EPA has, apparently, opted not to do its job. It appears that EPA is deferring to an impressive array of technical synthesis reports, and hoping to get away with it by asserting that the reports are just as good as anything the Congress would expect EPA to produce on its own.
EPA fails to mention the politicization of this issue which might make one pause when evaluating third-party reports. There are so many documents presented that, even if one is shot down, EPA can ‘plausibly’ rely on the remaining reports to develop regulations.
If it could be demonstrated that EPA’s claims about third-party synthesis reports [or sections thereof] were false, then EPA would be forced to essentially replicate the IPCC report – under the lens of FOIA.
Re: jim edwards (#46),
Now that you have boiled the issue down to the its essentials and assuming that third party findings could be shown to be, at least in part, false (or one sided on the whole?) how would an individual like Steve M practically go about seeking justice.
When reputable organizations such as the ACLU get involved they get attention because the responders know that they have a line to the media and that they are not an individual crackpot. How frequently is an individual without an organization like the ACLU behind them able to get their day in court and win the day?
Steve: Ken, puh-leeze. Give me a break. I’m not trying to start a crusade on this thing. I’m just commenting on a topical issue, because I think that the “community” should realize that stonewalling, obstruction and non-transparency could have a price.
Re: Kenneth Fritsch (#49),
Good rhetorical question. I’m as cynical as the rest. Steve’s a Canadian; I wouldn’t presume to require him to play the role of Don Quixote if we Americans go off the deep end. [He has testified before the Senate and worked on 4AR – he’s got to be at least slightly better than your average crackpot.]
I think that individuals can present the truth and be prepared to be ignored. The truth is ammunition, however.
Some group or individual with higher profile might pick up the ammunition and have better success with it. Those petitions from thousands of scientists disagreeing with the consensus seem to have been wasted on the NY Times. They’re probably well-suited as an EPA public comment to dispute ‘consensus’.
The Obama administration appears to be reacting to the Tea Parties, seeking $100 million in cuts from their multi-trillion dollar spree. An individual can assert the truth and trust to fate that a Senator, organization, and/or movement might pick up the torch.
Politics is driving this rush to judgment; politics can slow it down.
Quoting from you Steve after the November Election:
I guess you were wrong about what was first on the Obama Administration’s CO2 agenda. 😉
As Notsure says in post #17, the EPA is is having hearings next month. Perhaps we will get another chance to see what is brought to the table and how it is accepted or rejected. It (may) be an opportunity to test the system as well. This may also transfer into Canada. Is Canada going to follow the USA or work independently? A committee has been set up chaired my MP James Bezan, a Manitoba member. It is also likely that they will cross the country having hearings for the public.
With the American system, I think that something is lost once it gets into the courts.
An observation:
No poster yet here has yet proclaimed the NRC study really debunked the tree ring proxies.
With six other specially selected proxy studies the NRC validated, for promotional purposes, the hockey stick symbol. They “just kind of winged it.”
That was the gravamen of the NRC “study” then, now we can see its ultimate utility, support for cap and trade legislation. The tree ring problem was a minor tiff to them that was artfully circumvented. Post-normal science.
This thread on EPA’s tech findings is quite probably Steve’s most important contribution to the concept of a final conclusion linking AGW theory to direct action on CO2 remediation. Whatever EPA finally decides, it’s technical findings are subject to the Administrative Procedures Act. Under this law EPA has a very heavy, very specific burden to show that CO2, or any other GHG, actually cause real harm to human health. Citing studies performed by others, and not controlled directly by EPA, as its sole basis for technical conclusions is very unlikely to meet explicit procedural requirements of the law. Additionally, EPA is required to respond specifically to comments on its proposal. Like it or not, EPA’s gamble to rely excusively upon IPCC et al, data on an issue of this significance is not going to pass court review. Bottom line, the American public is paying EPA to conduct its own studies on AGW and provide for detailed public comment on each study.
Steve, as an engineer having been involved in bringing new technology to fore and acquiring quite a few scars to show for it, I have found invariably that when research samples or engineering samples are not released but claims of success are made complete with supporting data, the claims are an attempt to swing opinion but the samples are withheld so indepdendent measurements do not swing it back. The situation in which I see you entangled is uncomfortably familiar. Human nature does not change.
Steve, that was a rhetorical Steve M to which I referred and not necessarily you. I suspect that you have more faith in these apparent contradictions being ironed out under the current systems whereas my view would be more inline with changes coming from a change in the consensus thinking and not from the process working “right”.
My point is that, unless one has a popular “unpopular” cause to pursue and thus can obtain the help of a large organization in sympathy with the cause, all these contradictions will not be ironed out on a case by case basis.
It is important for people to point to these apparent contractions so we can better understand how the system works in practice as opposed to theory.
As the following link to the Wall St. Journal indicates, the EPA ruling has nothing to do with science, and everything to do with politics. Obama doesn’t even have the votes in his own party to implement a CO2 cap and tax, so he’s prepared to use the EPA and the courts to do it.
5 Trackbacks
[…] Steve McIntyre has a quieter discussion of the issues in this document at Climate Audit. […]
[…] considered this specific issue in considerable detail. See submission here, covering post here and first post on the topic […]
[…] considered this specific issue in considerable detail. See submission here, covering post here and first post on the topic […]
[…] considered this specific issue in considerable detail. See submission here, covering post here and first post on the topic […]
[…] couple of months ago, I posted on the EPA Endangerment Finding. In Canada, the government would just go ahead and pass the […]