The "Confidentiality Agreements"

CRU refused my FOI request for CRU data stating that:

Regulation 12(5)(f) applies because the information requested was received by the University on terms that prevent further transmission to non-academics

I asked to see the precise language of the underlying agreements because I very much doubted that agreements specifically prohibited “further transmission to non-academics”. If there were such a term in an agreement, it seemed far more likely that the term would be for “academic use” or something like that, and, given that my interest was scholarly rather than commercial, I doubted that the language of any applicable agreement would be applicable.

CRU has only managed to locate three documents pertaining to their agreements with NMSs: an application to Spain and letters from Norway and Bahrain, all from 1993-4. They also include a letter from CRU to the Met Office and, inexplicably, a copy of a current webpage from NERC governing Met Office data.

In their discussion of data availability, they state (in contorted logic) that:

In some of the examples given, it can be clearly seen that our requests for data from NMSs have always stated that we would not make the data available to third parties. We included such statements as standard from the 1980s, as that is what many NMSs requested.

Let’s examine the agreements. If the effect of these and other agreements is to prohibit them from supplying me the data, it is under language that has far more broad reaching consequences than merely preventing the delivery of data to me.

The British Territories Agreement
Here is the language from the CRU request to the UK Met Office regarding information about British Territories. It asks for data in connection with the construction of climatological normals; it makes no mention of the construction of a gridded temperature index nor the construction of a merged land-sea index. It says that the data would be used “unauthorized for any project” – which would obviously include its use in CRUTEM and HadCRU unless such authorization had been obtained. It doesn’t prohibit delivery of the data to “non-academics”; it prevents delivery of the data to “third parties” -which includes all the recipients of Advance 10K data, those people who downloaded cruwlda2 or newcrustnsall and any other academic.

If this language is representative of agreements with NMSs, then it prohibits the delivery of the data to CDIAC at the US Department of Energy (who published versions in the mid1980s and placed a version online in the early 1990s.) For that matter, it would prohibit the delivery of NMS data to the Met Office for use in HadCRU as the Met Office is a “third party” to CRU.

The Norway Correspondence
The Norway correspondence is also from 1993-1994 (pre WNO Resolution 40) and also is about 1961-1990 climatological normals, rather than temperature data. But viewing this as an example of the “lost” agreements, as CRU invites us to do, it says that CRU must not give the data “to a third party”. It doesn’t say “non-academic” – it says “third party.” If CRU seriously holds that this clause is in effect, then, as above, it prevents them from delivering data to the Advance-10K academics, to CDIAC and to the UK Met Office for use in HadCRU.

The Bahrain Correspondence
The language may well not constitute a binding agreement. But let’s stipulate that it does. Again it doesn’t refer to “non-academics”; it refers to “third parties”, including academics, CDIAC and the Met Office.

The Spanish Correspondence

As others have noted, the Spanish correspondence contains no prohibition on delivery to third parties – it merely asks for citation. I suppose that CRU might argue that there was an implied term of the agreement regarding further transmission of the data – but on the available evidence, such implied term would extend to all “third parties” and not merely to Climate Audit readers.


  1. jim edwards
    Posted Aug 12, 2009 at 2:19 PM | Permalink

    Agree with your characterization of any restrictions on distribution as being absolute, and not subject to the whim of the data holder.

    If anything can be implied from the Spanish agreement, it’s a waiver of any condition on use of the requested information.

    They asked Hulme to sign his request, agreeing to their proposed terms, they also asked him to list his planned use.

    Hulme DIDN’T sign the agreement, and failed to provide ANY planned purpose.

    The Spanish opted to give him all of the data, anyway. That smells like waiver of conditions of use.

  2. jryan
    Posted Aug 12, 2009 at 2:24 PM | Permalink

    This will no doubt lead to claims of verbal waivers being in place for academics.

  3. Craig Loehle
    Posted Aug 12, 2009 at 3:26 PM | Permalink

    Steve, it is not fair the way you use that “logic” thingy to hoist them by their own petards (yes I’m quoting Shakespeare incorrectly again, but I’m at a climate skeptics workshop in Missouri at the hotel computer…)

    • Scott Brim
      Posted Aug 12, 2009 at 4:13 PM | Permalink

      Re: Craig Loehle (#3)

      … but I’m at a climate skeptics workshop in Missouri …

      Isn’t Missouri known as the “show me” state?
      As in, “I’m from Missouri. Show me.”

  4. Pedro S
    Posted Aug 12, 2009 at 3:32 PM | Permalink

    Roger Pielke Jr has also put in an FOI request. He is an academic so they can’t reject him on the ‘not an academic basis’.

  5. Follow the Money
    Posted Aug 12, 2009 at 3:45 PM | Permalink

    You all should find pertinent leads about the IACGEC and others’ data policies and data warehousing policies in this article:

    Space Policy
    Volume 16, Issue 1, 29 February 2000, Pages 45-59

    European perspectives on trends in public sector Earth observation data policies

    Ray Harris and Nicola Olby

    Note, for one, the European concern for “re-investment” as support for clamming up on the science.

  6. Third Party
    Posted Aug 12, 2009 at 5:19 PM | Permalink

    What about transmission to 4th , 5th, 6th, … parties?

    What if all the data providers are invited to the party?

  7. Ryan O
    Posted Aug 12, 2009 at 6:02 PM | Permalink

    Well, if they won’t show me the data, how about showing me who won’t let them show me the data?

    Mr. Palmer,

    I request, under FOI/EIR, a list of the individual scientists, governments, NMSs, and any other organization that has been solicited to provide temperature data for inclusion in the CRU land temperature index by any person acting in an official capacity at CRU or the University of East Anglia. I additionally request, as a separate entity, a list of the individual scientists, governments, NMSs, and any other organization that is known to have provided temperature data for the aforementioned purpose.

    The reason for my request is to determine what parties have or may have contributed information in order to lobby these parties to provide the University of East Anglia written statements of unconditional or conditional release (on satisfactory terms for academic and scientific research) of this information; or, alternatively, written statements that the information is already considered public domain.

    It is in the public interest for the temperature information to be freely available in order for the scientific processes of replication and sensitivity analysis of the CRU land temperature index to be conducted. As your organization has offered the position that it lacks the resources to complete this arduous undertaking itself, it is therefore greatly in the public interest that the individuals and organizations that are apparently preventing release of the information be provided such that they can be lobbied through the appropriate channels.

    Ryan O’Donnell

    • steven mosher
      Posted Aug 12, 2009 at 10:49 PM | Permalink

      Re: Ryan O (#9),


    • Posted Aug 13, 2009 at 8:22 AM | Permalink

      Re: Ryan O (#9),

      This should tie in nicely with my request (see this comment at Lucia’s). We ought to get a list of organizations where we don’t need to ask as well as a list of ones where we do

      • Ryan O
        Posted Aug 13, 2009 at 10:53 AM | Permalink

        Re: FrancisT (#18), I saw that. Very nice.

      • Kenneth Fritsch
        Posted Aug 13, 2009 at 11:06 AM | Permalink

        Re: FrancisT (#18),

        Is not there some confusion here about obtaining, upon request the raw data, that the CRU sources currently provide to CRU and the original raw data (before the 1980s when the lose of these records occurred) that were provided originally and then adjusted by CRU. I suspect that CRU could show “value added” after the 1980s and thus they might have a legitimate data series from that time forward.

        Of course, since we have the satellite series for that period, I would have to say that UAH rules – or maybe RSS rules.

  8. DaleC
    Posted Aug 12, 2009 at 6:12 PM | Permalink

    “It says that the data would be used “unauthorized for any project” – which would obviously include its use in CRUTEM and HadCRU unless such authorization had been obtained.”

    Missing a ‘not’ somewhere?

  9. AnonyMoose
    Posted Aug 12, 2009 at 10:14 PM | Permalink

    It looks like the Met’s data quality control procedures have missed the detail about having access to data. Maybe next month’s Met data will only have UK data. On the bright side, it will be the global dataset which is of the highest quality.

  10. Posted Aug 12, 2009 at 11:58 PM | Permalink

    Slightly off topic, here is a FoI request to the Met Office to see if they have assessed the quality control procedures used in creating CRUTEM.

    We have not undertaken any in-house assessment of the effectiveness of the CRUTEM3 quality control and homogenisation process. The only analysis in this regard which we have been involved in is that detailed in Brohan et al., 2006. The procedures are well documented within the peer reviewed literature over the last twenty+ years – a full listing of which is available from http:/ We have always concluded that the dataset construction processes are adequately documented in the peer reviewed literature. It should also be noted that HadCRUT has always been a collaborative effort with the Hadley Centre contributing the sea surface temperature component and the Climatic Research Unit providing and maintaining ultimate responsibility for the land component.

  11. anonymous
    Posted Aug 13, 2009 at 1:40 AM | Permalink

    “We have always concluded that the dataset construction processes are adequately documented in the peer reviewed literature.”

    So their dataset construction processes are adequately documented, but they have lost the raw data and don’t know exactly what they did to get their “value added” version – not a caveat i’ve noticed on studies using the data. Climate Science – it’s an ongoing joke isn’t it ? You really couldn’t make half this stuff up.

  12. 2dogs
    Posted Aug 13, 2009 at 1:47 AM | Permalink

    Does this explain the sudden withdrawal of access on July 31? That is, when prompted to check the actual agreements, CRU realised what they were currently doing was in breach?

  13. Manfred
    Posted Aug 13, 2009 at 4:15 AM | Permalink

    The L.A. police department possibly has a vacancy for your skills, after Inspector Columbo left them in 2003.

  14. Ron Cram
    Posted Aug 13, 2009 at 5:16 AM | Permalink

    Steve, can anything be done when an FOI request is rejected for untrue reasons? Now that you have evidence no restriction exists toward “non-academics,” is there an FOI police that you can report this too? I would think that for FOI to have any teeth, some entity would have to be in charge of enforcing it.

  15. steven mosher
    Posted Aug 13, 2009 at 11:57 AM | Permalink

    The response to my request:

    Dear Mr. Mosher


    (Our Ref: FOI_09-69)

    Further to your request for information received on 24 July 2009, I have consulted relevant units within the University and, pursuant to my obligations under section 16 of the Act to provide advice and guidance, I am writing to request clarification of several aspects of your request. Apologies for the delay in responding to you on this matter, but as you may know, we have received a large number of requests for information under the Act recently and it is taking some time to deal with each request.

    In your request, you have asked for a copy of policies and procedures regarding employee responsibilities regarding entering into confidentiality agreements and verbal agreements, and for a copy of policies and procedures regarding employee responsibilities regarding the preservation of written agreements.

    The University does not have one, overarching policy or procedure regarding entering into confidentiality or verbal agreements. Each division within the University has policies and procedures specific to their area of work . This also applies to the preservation of written agreements. In order to answer your question for all of the University, it is highly likely that we would exceed the statutory appropriate limit of 18 person hours to locate, retrieve & review the requested information.

    In order to avoid this situation, I would therefore ask you to clarify what aspect of the University’s work would be the focus of your request for such policies and procedures. For example, given the nature of the other components of your request, are you simply interested in policies in relation to research activities?

    Please note that the statutory timescale of 20 working days as defined by the Freedom of Information Act 2000 will be ‘suspended’ until such time as we receive clarification of your request. Once that is received, the ‘clock’ will recommence, your request considered, and you will receive the information requested within the statutory timescale, subject to the information not being exempt or containing a reference to a third party. You will be informed of any exemptions or references to third parties.

    I trust this is to your satisfaction and look forward to your reply.

    Yours sincerely

    David Palmer

    Information Policy & Compliance Manager

    University of East Anglia

  16. jeez
    Posted Aug 13, 2009 at 3:17 PM | Permalink

    I received the same response as mosh.

  17. andymc
    Posted Aug 24, 2009 at 6:40 AM | Permalink

    Antagonish (with apologies to Hughes Mearns)

    I get requests from god knows where,
    For missing files that are not there,
    They were not there again today,
    I wish that you would go away,

    Although I know this can’t be proved,
    I’m sure we had them when we moved,
    We packed them well, but in our haste,
    We put them in a box marked ‘waste’

    We do not have a single datum,
    I told you lads, the doggy ate ‘em.
    Secrecy agreements too,
    A thieving monkey from the zoo?

    Don’t ask me now, don’t ask me later,
    There’s just no sign of that raw data
    Use Met data that’s been padded,
    That’s data that’s had value added,

    I put the files upon the stair,
    I looked last week, they were not there.
    I couldn’t find them come what may,
    Maybe they just blew away

3 Trackbacks

  1. […] The “Confidentiality Agreements” […]

  2. […] this article recognizes the key aspect of what Steve McIntyre has spent years now working on  HERE as an example, access to the data and code behind research projects.  Others may agree with NAS recommendations […]

  3. […] “small document” is online at CRU here. It was reported at CA on August 11, 2009 here, here and here (among […]

%d bloggers like this: