NOAA: HadCRU3 data not "influential"

A CA reader recently wrote to NOAA (which distributes HADCRU3 gridded data) asking them whether the data and methods had been reviewed pursuant to NOAA Quality of Information guidelines. (It is nice to see a reader complete such an exercise himself rather than writing on a thread that I should do it.) NOAA replied that they had not done so because the data came from a third party and that HADCRU3 data “do not meet the definition of influential”. One wonders what sort of data NOAA regards as “influential”. Here’s the correspondence:

The CA reader wrote:

I believe that the HADCRU3 dataset is influential scientific information, which although generated by a 3rd party, is of such importance that it is important that it be of known quality and consistent with NOOA’s information quality guidelines.

Has NOAA reviewed the raw data and processing methods in accordance with NOAA’s Quality of Information standards?

Does NOAA have available the raw station data and metadata and processing algorithms behind the HADCRU3 gridded data? If so, I request a copy of such data, metadata, and processing information.

I would also like to know the nature and description of any specific checks used by NOAA to ensure the objectivity of the HADCRU3 dataset.

Best Regards,

Reader

NOAA replied:

Dear …:

Thank you for your inquiry regarding the HADCRUT3 gridded data available on the National Oceanic and Atmospheric Administration’s (NOAA) Office of Oceanic and Atmospheric Research (OAR) website: http://www.cdc.noaa.gov.

NOAA OAR’s Climate Diagnostics Center is now part of the Earth System Research Laboratory, Physical Sciences Division (ESRL/PSD). ESRL/PSD makes available many climatological and meteorological datasets, as a service to the research community and to the general public. ESRL/PSD is not the originator of these datasets.

In the case of the datasets in question, the originator is the United Kingdom’s Meteorological Office’s Hadley Centre for Climate Change. Please refer to the page you reference in your inquiry and you will note the Hadley Centre is identified as the “original source”. Links to the Hadley Centre are also offered on the ESRL/PSD web page: http://hadobs.metoffice.com/hadcrut3/

The HADCRUT3 datasets are third party information. NOAA’s policies regarding dissemination of third party information are specified in the NOAA Information Quality Guidelines:
http://www.cio.noaa.gov/Policy_Programs/IQ_Guidelines_110606.html, as well as the OMB Peer Review Bulletin. According to NOAA IQ Guidelines, when the agency disseminates information supplied by a third party, the agency should review the OMB Peer Review Bulletin for applicability. According to the provisions of the OMB Peer Review Bulletin, dissemination of an information product is not covered unless it represents an official view of the agency. The Hadley Centre datasets do not represent an official view of NOAA. Further, even if the datasets did contain an official agency view, the peer review requirements would apply only if the dissemination of information is “influential”; which is defined to mean, “the information will have or does have a clear and substantial impact on important public policies or private sector decisions”. NOAA OAR has determined that the Hadley Centre datasets do not meet the definition of influential since the data will not have a clear and substantial impact on important public policies or private sector decisions.

ESRL/PSD makes every effort to ensure that the informational content of HADCRUT3 datasets available on NOAA/OAR’s website are the same as the original data disseminated by the Hadley Center. The NOAA IQ Guidelines provide that the accuracy of original and supporting data within an acceptable degree of imprecision or error, are by definition within the agency standard, and therefore presumed to be considered accurate. Because the HADCRUT3 datasets constitute original data, this standard applies and the datasets are presumed to be accurate under the NOAA IQ Guidelines.

As a re-disseminator, and not the originator of the data, ESRL/PSD does not maintain raw station data, nor is ESRL/PSD the correct source for information regarding the processing algorithms used to create the datasets. That information can be found on the Hadley’s Centre’s website and also in the published literature, referenced and/or linked on the web page.

Thank you, and good luck with your inquiries.

Sincerely,

Nick Wilde

Dr. Nick Wilde, Senior IT Manager,
Physical Sciences Division, Earth System Research Laboratory


40 Comments

  1. Steve McIntyre
    Posted Aug 8, 2009 at 7:37 AM | Permalink

    The NOAA official stated:

    According to the provisions of the OMB Peer Review Bulletin, dissemination of an information product is not covered unless it represents an official view of the agency.

    Can anyone locate the precise language/clause of the OMB Peer Review Bulletin, that says this?

    • MikeU
      Posted Aug 8, 2009 at 8:39 AM | Permalink

      Re: Steve McIntyre (#1), “Can anyone locate the precise language/clause of the OMB Peer Review Bulletin, that says this?”

      From this document, page 8:

      THE REQUIREMENTS OF THIS BULLETIN

      This Bulletin addresses peer review of scientific information disseminations that contain findings or conclusions that represent the official position of one or more agencies of the federal government.

  2. jae
    Posted Aug 8, 2009 at 8:14 AM | Permalink

    Another government brush-off. If this data is not considered “influential,” just what WOULD be considered such??

  3. Posted Aug 8, 2009 at 8:15 AM | Permalink

    “the information will have or does have a clear and substantial impact on important public policies or private sector decisions”. NOAA OAR has determined that the Hadley Centre datasets do not meet the definition of influential since the data will not have a clear and substantial impact on important public policies or private sector decisions.

    I’m very unhappy to learn that temperature data will have no effect on policy. If there’s going to be a policy for climate change, it would be best if they used the data.

    Re: Steve McIntyre (#1),
    I don’t have the slightest idea.

  4. MikeU
    Posted Aug 8, 2009 at 8:58 AM | Permalink

    Not sure if there’s an updated Bulletin for the new administration or not – I’ll keep looking. Note that they’re talking about the peer review of information coming to a federal agency. As far as I can tell, this Bulletin says nothing about the disseminations of said information to the public…

    They do talk about “routine statistical information” coming from another federal agency in their Exemptions section, but I wouldn’t think HADCRUT3 would count in that category since it’s not coming from a U.S. federal agency.

  5. William McQuiddy
    Posted Aug 8, 2009 at 9:09 AM | Permalink

    I’ll bet Hadley Center will be surprised.

  6. MikeU
    Posted Aug 8, 2009 at 9:11 AM | Permalink

    Hmm, I should probably post only after my first cup of coffee. They’re talking about the peer review of information disseminations coming from a federal agency as part of an “official position”. Parsing Bureaucrat is probably an exercise best left to lawyers.

  7. pjm
    Posted Aug 8, 2009 at 9:11 AM | Permalink

    and good luck with your inquiries.

    Do you think he’s thinking you’ll need it?

  8. pjm
    Posted Aug 8, 2009 at 9:21 AM | Permalink

    a clear and substantial impact on important public policies or private sector decisions

    As NOAA is American, I imagine they expect that these policies and decisions will be based on the American product (GISS).

  9. Posted Aug 8, 2009 at 9:31 AM | Permalink

    Perhaps NOAA has a secret “confidentiality agreement” with CRU?

  10. Soronel Haetir
    Posted Aug 8, 2009 at 9:56 AM | Permalink

    Given that there are several such data sets, these agencies can claim with a straight face that none of them individually will have a clear effect on policy. Only the aggregated information of the various data sets is influential, but no one cares what goes into the sausage so long as the end product is consistent.

  11. Calvin Ball
    Posted Aug 8, 2009 at 10:05 AM | Permalink

    …dissemination of an information product is not covered unless it represents an official view of the agency.

    What’s an “official view”? How can data be a “view”?

    • Neil Fisher
      Posted Aug 8, 2009 at 9:48 PM | Permalink

      Re: Calvin Ball (#12),

      What’s an “official view”?

      The official view is the view taken by the official official responsable for the official view.

      How can data be a “view”?

      Exactly the point!

  12. Roger Pielke, Jr.
    Posted Aug 8, 2009 at 11:53 AM | Permalink

    Steve- In reply to #1

    Is here:

    http://www.whitehouse.gov/omb/memoranda/fy2005/m05-03.pdf

    Section that you asked about:

    “This Bulletin addresses peer review of scientific information disseminations that contain findings or conclusions that represent the official position of one or more agencies of the federal government. ” at p. 8

    See also p. 10:

    “An information product is not covered by the Bulletin unless it represents an official view of one or more departments or agencies of the federal government. Accordingly, for the purposes of this Bulletin, “dissemination” excludes research produced by government-funded scientists (e.g., those supported extramurally or intramurally by federal agencies or those working in state or local governments with federal support) if that information is not represented as the views of a department or agency (i.e., they are not official government disseminations). ”

    Since the IPCC represents an official US gov’t view then any science presented therein would necessarily be covered by the OMB PRB.

  13. theduke
    Posted Aug 8, 2009 at 12:18 PM | Permalink

    The Hadley Centre datasets do not represent an official view of NOAA. Further, even if the datasets did contain an official agency view, the peer review requirements would apply only if the dissemination of information is “influential”; which is defined to mean, “the information will have or does have a clear and substantial impact on important public policies or private sector decisions”.

    This is as flagrant an example bureacrat double-talk as ever I’ve witnessed. How can a “dataset…represent an official view of NOAA?” Rather than just say “HADCRU has warned us in advance to not cooperate with you,” they invent all this transparent legal-sounding nonsense to make it sound like they are defending some important principle. In effect, they are saying, “The data is unimportant to us so we are not going to give it to you.” Absurd.

  14. Shallow Climate
    Posted Aug 8, 2009 at 12:41 PM | Permalink

    As I (or someone) posted on another thread, in another time warp, “You can delegate authority, but you cannot delegate responsibility.” And as someone far, far greater than I once said, “It is not what goes into a man that defiles him, it’s what comes out of a man that defiles him.” But NOAA is happy to say that they are not responsible for what comes out of them. And so mankind descends further and further into the abyss, with (once again, in my opinion) the climate science Community proudly leading the way. (Is that sadness or bitterness I detect here, or both?)

  15. Posted Aug 8, 2009 at 1:10 PM | Permalink

    “…ESRL/PSD does not maintain raw station data, nor is ESRL/PSD the correct source for information regarding the processing algorithms used to create the datasets. That information can be found on the Hadley’s Centre’s website and also in the published literature, referenced and/or linked on the web page.”

    Apparently they believe the raw data and processing algorithms are available to the public. I guess they’ve never asked for them, or they would know better.

  16. Andrew
    Posted Aug 8, 2009 at 2:00 PM | Permalink

    That’s rather amusing. One wonders whether we can detect a bit of jealousy on the part of NOAA-after all, one would be hard pressed to find a more influential data base than HadCrut-GISS may be close, but NOAA is clearly grouchy that almost nobody even pays attention to NCDC!

  17. mpaul
    Posted Aug 8, 2009 at 3:28 PM | Permalink

    hmm, here’s an email I received from Senator Feinstein of California:

    Dear …:

    Thank you for writing to me to express your opposition to establishing a cap-and-trade system to curb greenhouse gas emissions. I appreciate the time you took to write, and I welcome the opportunity to respond.

    I understand that you are concerned that a cap-and-trade system could result in the loss of American jobs and increase energy costs. I believe, however, that the cost of failing to act to address climate change far exceeds the cost of taking the necessary steps to reduce greenhouse gas emissions and make a permanent shift towards clean, renewable energy and energy efficient technologies. A recent study by researchers at Tufts University found that inaction could cost the United States economy as much as 3.6 percent of the gross domestic product, or $3.8 trillion annually, by 2100.

    Please know that I appreciate hearing your concerns, and I will keep your thoughts in mind as I continue working in the Senate to advance measures that address climate change, invest in our economy and increase our energy security.

    Again, thank you for writing. If you have further questions or comments, please contact my office in Washington, D.C. at (202) 224-3841. Best regards.

    Sincerely yours,
    Dianne Feinstein
    United States Senator

    Here’s a link to the Tuff’s University study that Senator Feinstein states influenced her policy decision:

    http://www.nrdc.org/globalwarming/cost/cost.pdf

    Footnote 27 of the study states:

    To estimate the effects of the business-as-usual scenario,
    we increased regional temperatures every decade by the
    expected temperature change from the Hadley CM3 climate
    model
    .

    On page 9, the study states:

    Our projection of a business-as-usual climate future is
    based on the high end of the “likely” range of outcomes
    under the Intergovernmental Panel on Climate Change’s
    (IPCC’s) A2 scenario

    In fact, they cite IPCC 28 times in the study.

    This would seem to suggest that HadCRU3 data has directly influenced US public policy.

    All bold font treatments are mine.

  18. henry
    Posted Aug 8, 2009 at 4:52 PM | Permalink

    NOAA OAR has determined that the Hadley Centre datasets do not meet the definition of influential since the data will not have a clear and substantial impact on important public policies or private sector decisions.

    Nice way to tell Jones that his 25 years of work will “not have a clear and substantial impact on important public policies or private sector decisions”.

    Good title for a headline “NOAA says Hadley Centre datasets not influential.”

    New question, then – If GISS and HadCRU use data from the same stations, and the HadCRU datasets do not meet the definition of influential, then what additional data allows GISS to meet the definition of influential?

    • Geoff Sherrington
      Posted Aug 8, 2009 at 6:48 PM | Permalink

      Re: henry (#19),

      Yes, and what about US Oak Ridge funding of CRU for all those years? Does this not give any IP rights to the US Govt in a roundabout way?

      • Harry Eagar
        Posted Aug 10, 2009 at 5:28 PM | Permalink

        Re: Geoff Sherrington (#20),

        Well, if we’re not getting anything we can use out of Hadley, maybe Oak Ridge should consider not funding it?

        I gotta say that this is hilarious — in a serious, grownup way, of course — and mpaul’s post cracked me up.

  19. Alan Wilkinson
    Posted Aug 8, 2009 at 6:49 PM | Permalink

    Nice headline here: “It’s Raining Bonuses at the Met Office”

    http://www.timesonline.co.uk/tol/news/politics/article6788803.ece

    Piers Corbyn, owner of the independent forecasting business WeatherAction, said: “The Met Office forecasts for the last three summers have been completely and ridiculously wrong. They can only give themselves bonuses because they decide how they mark themselves.”

    A Met Office spokesman said despite poor publicity over seasonal forecasts, its short-term forecasts were “very, very good” and increasingly accurate.

    He said the Met Office had met three out of its four targets for forecasts in the year ending in March.

  20. Craig Loehle
    Posted Aug 8, 2009 at 6:55 PM | Permalink

    I think “official” data would the consumer price index, the official inflation rate, EPA car mileage numbers, the official unemployment rate, etc. They will probably insist that Hadley CRU and GISS data are simply research products not subject to this type of review.

  21. rafa
    Posted Aug 9, 2009 at 1:13 AM | Permalink

    For those of us who do not have English as their mother tongue, the phrase “good luck with your inquiries” seems to be full of sarcasm. Is that so? or, instead, is just an informal farewell idiom?.

    best

    • TAG
      Posted Aug 9, 2009 at 5:30 AM | Permalink

      Re: rafa (#25),

      For those of us who do not have English as their mother tongue, the phrase “good luck with your inquiries” seems to be full of sarcasm. Is that so? or, instead, is just an informal farewell idiom

      It is not really sarcasem but it indicates that the inquirer can expect no further assistance from them. It is a final good bye as in “Good Bye and Good Luck.”

  22. Geoff Sherrington
    Posted Aug 9, 2009 at 4:26 AM | Permalink

    What’s in a word like “Influential?”

    Last month an IPCC survey was conducted by the UN. The only question asked was:-

    “Would you please give your honest opinion about solutions to the warming crisis in the rest of the world?”

    The survey was a huge failure because of the following:

    In Eastern Europe they didn’t know what “honest” meant.
    In Western Europe they didn’t know what “crisis” meant.
    In Africa they didn’t know what “warming” meant.
    In China they didn’t know what “opinion” meant.
    In the Middle East they didn’t know what “solution” meant.
    In South America they didn’t know what “please” meant.
    In the US they didn’t know what “the rest of the world” meant.

  23. Geoff Sherrington
    Posted Aug 9, 2009 at 4:44 AM | Permalink

    Steve,

    On re-reading your header, maybe the following is stale cheese, but it has CRU monthly and annual temperatures from late 1800s to 1992 or so, for several hundred stations, as well as code (but not in R) and propeties other than temp.

    http://cdiac.ornl.gov/ftp/ndp020/

    with contained files like
    jonesnh.dat 24-Jun-1993 17:04 8.2M and
    jonessh.dat 24-Jun-1993 17:04 1.2M
    ndp020r1.txt 04-Aug-2009 11:57 29K (note date)

    There are dozens of files, as if they were used to work up a paper. I’ve cross-posted to Roman M on “A 2002 request to CRU”.

    Steve: I’m familiar with the 1993 files which Jones sent to CDIAC. It’s been referred to on a number of occasions as evidence that there were either no confidentiality agreements from the 1980s or Jones disregarded them and the data is no longer confidential. The existence of an Aug 4, 2009 file is a surprise – I’ll look at it. Update: IT seems the same as the Wayback Machine version except for insertion of “DOI: 10.3334/CDIAC/cli.ndp020″ in the text. Timing might be a coincidence.

  24. MetMole
    Posted Aug 9, 2009 at 7:21 AM | Permalink

    Re: Rafa #24

    For those of us who do not have English as their mother tongue, the phrase “good luck with your inquiries” seems to be full of sarcasm. Is that so? or, instead, is just an informal farewell idiom?

    Further to TAG (at #27), the additional beauty of it is that under the circumstances, with a well-known history of (officious) institutions denying “denialists” access to data and code, it clearly could be taken to be sarcastic but the form of it is such that the writer could easily deny any such intention. All he has to say is that he meant exactly what he wrote. And in truth, who is to say he didn’t?

    Besides, you could either accuse him outright of being underhand sarcastic and be labelled paranoid. Or you could ask him if he meant what he wrote, in which case good luck with the reply you get.

    And, Rafa, for the avoidance of doubt that last clause was intended as a joke. :)
    .
    Keep lucky.

  25. Calvin Ball
    Posted Aug 9, 2009 at 1:02 PM | Permalink

    Rafa, the lawyers have a term “plausible deniability”. That may have been sarcastic, but it’s plausibly deniable.

  26. Ron H.
    Posted Aug 9, 2009 at 4:17 PM | Permalink

    mpaul #18

    Damn! I got exactly the same letter from Senator Feinstein. I am hurt. I thought the nice lady was speaking directly to me personally. >sarc/off

    I suppose that what she is really telling us is that no matter what we want, she plans to vote for that Cap & Tax pig.

  27. Mick
    Posted Aug 9, 2009 at 10:22 PM | Permalink

    Hello everybody!

    Hi, Dr Nick!

    It shouldnt be suprising that Dr Nick has moved into climate science…

  28. Posted Aug 10, 2009 at 2:08 AM | Permalink

    NOAA OAR has determined that the Hadley Centre datasets do not meet the definition of influential since the data will not have a clear and substantial impact on important public policies or private sector decisions.

    Is Dr Nick Wilde not aware of the fact that HADCRUT3 is the data set used by the IPCC???!!!

  29. Scott
    Posted Aug 10, 2009 at 1:56 PM | Permalink

    Brush off. This reply is coming from the IT Manager not the FOI officer or any of the scientists who manage this data.

    • Charlie
      Posted Aug 10, 2009 at 9:20 PM | Permalink

      Re: Scott (#33), The Chief Information Officer is the person most commonly assigned the task implementing the Information Quality regulations. It is a different system than the FOIA. I guess in this case, the CIO uses the title IT Manager instead. I submitted the original request to the contact e-mail on the archive data page. That person obviously forwarded it to the IT Manager. All told, they took only 11 work days to reply.

      Although I don’t agree with the reply, he did make a substantive reply rather than just burying the request or sending back a form letter saying something like “doesn’t meet the FOIA or Q of I inquiry requirements”.

      A key statement in the response is that ESRL/PSD section of NOAA doesn’t do any review of HADCRU3 other than making sure the data is an accurate copy of the Met submission.

      My next step is to inquire with the main CIO of NOAA. I will politely point out the importance and influence of this data; the lack of transparency at CRU and the Met Office; and try to convince them to conduct an independent review.

      I may even suggest that someone with the initials SM might be convinced to accept a contract to review HADCRU3. :)

      A parallel inquiry will be an inquiry under FOIA to the Secretariat of NOAA, to find out whether a group other than ESRL/PSD has done a due diligence sort of review of HADCRU3. Each little step helps a bit. Calling their attention to the lack of verification of HADCRU3 and then NOAA having to come back and say “we don’t have any documents on verification of HADCRU3 because we never did any” may help them decide that it is a project worth doing.

  30. Geoff Sherrington
    Posted Aug 11, 2009 at 5:09 AM | Permalink

    From the site

    http://cdiac.ornl.gov/ftp/ndp020/

    I’ve extracted more Australian annual temp data from CRU and placed it in context with a few other authorities. I have no idea if this CRU version is the one that is still used. Ditto for most of the others. It makes me wonder how investigators can try to relate physical effects with temperature. Pick-a-graph.

    This site, Mount Gambier in South Australia, has WMO #948210 and Australian # 126021. It is about 25 km north of the Southern Ocean and 210m asl. Some metadata from BOM –
    “Meteorological observations commenced in Mt Gambier in 1861, when daily rainfall readings commenced at the Post Office. The Bureau of Meteorology opened an office at the Mt Gambier airport in 1941.
    Observations continued from the control tower on the airport up until 1963, when the office was moved to its present location on a small rise overlooking and adjacent to the airport. The office is about ten kilometers north of the City of Mt Gambier.”

    Please email me if a more clear graph would help. Recall that the blue diamonds at top, BOM 1993, are the earliest and the rest are adjustments thereof. I do not know what happens with yellow GISS. Sometimes at the top, sometimes at the bottom. (This version of GISS is raw, not homogenised).

    • Charlie
      Posted Aug 13, 2009 at 3:49 PM | Permalink

      Re: Geoff Sherrington (#36), “This version of GISS is raw, not homogenised”

      Discussion on the starting topic of this thread has run its course, so I don’t feel too bad about hijacking of the thread to discuss Mt.Gambier time series adjustments.

      Geoff — Have you been able to determine from the documentation of each data source what sort of adjustments have been applied in each case. It is clear from looking at the graph, that the different organizations have chosen different values for corrections. It also looks like some organizations identified breakpoints that needed correction in places where others treated the data stream as homogeneous. I’m a novice in this area, with most of my knowledge coming from reading about the USHCN Version 2 adjustment process.

      My suggestion for a graph would be to pick one series as the reference, and then plot the differences between it and each of the others. If needed for clarity, you could add in an additional offset of 0.1 or 0.5C between the difference series. Since BOM2009 is available over the entire period, it would probably be the best as the “reference series”.

      An additional plot line of the actual value of the BOM2009 could be added above the difference plots to provide context. It would probably be best to have the Y axis of the difference series expanded as compared to the BOM2009 actual values.

  31. curious
    Posted Aug 11, 2009 at 10:12 AM | Permalink

    Geoff – rather than a bigger graph can I suggest you tile it as individual plots? Maybe with each one having BOM1993CD as the original for comparison?

    • Geoff Sherrington
      Posted Aug 12, 2009 at 3:57 AM | Permalink

      Re: curious (#37),

      There are several options. I can make difference graphs. Some writers here known how to click on the smaller graph to get a larger one. I do not. Help please?

      The objective here is to show that there is no global temperature estimate which can lay claim to be the thoroughbred line.

  32. Tony Hansen
    Posted Aug 12, 2009 at 5:54 AM | Permalink

    Geoff Sherrington #36,
    Could you show the difference between BOM ’93 and BOM ’09 for the years they overlap?

One Trackback

  1. […] NOAA: HadCRU3 data not “influential” […]

Follow

Get every new post delivered to your Inbox.

Join 3,421 other followers

%d bloggers like this: