The CRU Gong Show: Refusing Ross McKitrick

Today brought in some CRU refusals- their rejections of Ross Mc, Roman M, myself. (They’re going to have to re-do their Roger Pielke rejection, since they replied to the wrong request in his case.) Each one deserves to be savored. So today I’ll post up their obstruction of Ross McKitrick.

FOI officer Palmer denied the request on the grounds that the request is “manifestly unreasonable” as the data is “available elsewhere”, that its disclosure would have an “adverse effect on international relations” and would have an adverse impact on the institutions supplying the data.

CA readers will recall that I requested the same version of CRU station data as was sent to Peter Webster and that they refused on the grounds that they had “confidentiality agreements” (all of which have been destroyed or lost other than stale agreements with Norway and Bahrain and and an agreement with Spain that does not require confidentiality) with parties that they can no longer identify, but the one thing that they were certain of was that these agreements prohibited the delivery of the data to a “non-academic”.

Ross McKitrick is obviously an “academic”. And aside from being an “academic”, he even has relevant publications in the field. Here is Ross’ original request:

Pursuant to the Environmental Information Regulations, I hereby request:

1. A copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009

2. A copy of any instructions or stipulations accompanying the transmission of data to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009 limiting its further dissemination or disclosure.

I write as an academic with publications in peer-reviewed journals and an ongoing research program on the subject of surface climate measurement. With respect to #2, please be aware that restrictions on data disclosure may disqualify any research arising from this data set from being published in many peer-reviewed journals, therefore I require a complete response as to whether any such instructions accompanied the data.

Thank you for your attention,

Here is the CRU response in full:

Your request for information received on 24 July 2009 for a “A copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009” and “a copy of any instructions or stipulations accompanying the transmission of data to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009 limiting its further dissemination or disclosure” has now been considered and it is, unfortunately, not possible to meet your request.

In accordance with Regulation 14 of the Environmental Information Regulations 2004 this letter acts as a Refusal Notice, and I am not obliged to supply this information and the reasons for exemption are as stated below:

Exception Reason
Reg. 12(4)(b) – Request is manifestly unreasonable. Information is available elsewhere

Reg. 12(5)(a) – Adverse effect on international relations. Release would damage relations with scientists & institutions from other nations

Reg. 12(5)(f) – Adverse effect on the person providing information. Information is covered by a confidentiality agreement

We believe that Regulation 12(4)(b) applies to your request for the data because the requested data is a subset of data already available from other sources; namely the Global Historical Climatology Network (GHCN ) , and the Climatic Research Unit already makes requested information available on it’s website in a gridded format. We believe, following DEFRA guidance, that it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available.

In regards Regulation 12(5)(a), much of the requested data comes from both individual scientists and institutions from countries around the world. If this information were to be released contrary to the conditions under which this institution received it, it would damage the trust that other national scientists and institutions have in UK-based public sector organisations and would likely result in them becoming reluctant to share information and participate in scientific projects in future. This would damage the ability of the University and other UK institutions to co-operate with meteorological organisations and governments of other countries.
Regulation 12(5)(f) applies to the data requested because the data was received by the University on terms that limits further transmission. We believe that there would be an adverse effect on the institutions that supplied data under those agreements as it would undermine the conditions under which they supplied the data to the Climate Research Unit.

In regards your request for any stipulations accompanying the transmission of the data to academics at Georgia Tech, no such instructions or stipulations are held by the University.

All the agreements that we do hold in relation to the requested data are available on the Climate Research Unit website at: http://www.cru.uea.ac.uk/cru/data/availability/
Regulation 12(1)(b) mandates that we consider the public interest in any decision to release or refuse information under Regulation 12(4). In this case, we feel that there is a strong public interest in upholding contract terms governing the use of received information. To not do so would be to potentially risk the loss of access to such data in future as noted above. In regards Regulation 12(4)(b), we believe it is not in the public interest to divert public resources away from other work to provide information that is available elsewhere. Finally in regards Regulation 12(5)(a), we feel that there is a clear public interest in neither damaging nor restricting scientific collaboration between UK-based scientists and institutions with international colleagues.

I should note, however, that the University is commencing work, in concert with the Met Office Hadley Centre, to seek permission from data suppliers in advance of the next update of the CRUTEM database in 2010 in order to provide public access to this data. This work has been announced on the CRU website and further updates on it’s progress will be available there.
I apologise that your request will be met but if you have any further information needs in the future then please contact me.

162 Comments

  1. Old Dad
    Posted Aug 13, 2009 at 6:15 PM | Permalink | Reply

    The coverup is always worse than the crime.

  2. JS
    Posted Aug 13, 2009 at 6:15 PM | Permalink | Reply

    I love the first one – it becomes a catch all for rejection of a request for any data:
    1) We refuse to provide you with the reasons for the rejection of your application for political assylum because they are a subset of the reasons that have been used for other rejections.
    2) We refuse to provide you with the currencies and exchange rate(s) that have been used in our calculation of your fees because they are a subset of the foreign exchange data which is already publically available from multiple sources.
    3) We refuse to provide you with the definition of “academic” that we use in judging requests for data by “academics” because our definition is a subset of the set of all definitions that are already publically available.

    The needle is in the haystack – you find it!

  3. Ryan O
    Posted Aug 13, 2009 at 6:16 PM | Permalink | Reply

    I’m wondering how they reconcile the transmission of data to Webster with the apparently devastating results that occur upon transmission of data.

    • Scott Brim
      Posted Aug 13, 2009 at 6:53 PM | Permalink | Reply

      Re: Ryan O (#3),

      I’m wondering how they reconcile the transmission of data to Webster with the apparently devastating results that occur upon transmission of data.

      They know the sound a boomerang makes as it is thrown and are thus able to dodge it successfully when it comes back.

  4. Ross McKitrick
    Posted Aug 13, 2009 at 6:22 PM | Permalink | Reply

    Just slowly read these sentences out loud.

    Regulation 12(5)(f) applies to the data requested because the data was received by the University on terms that limits further transmission. We believe that there would be an adverse effect on the institutions that supplied data under those agreements as it would undermine the conditions under which they supplied the data to the Climate Research Unit. In regards your request for any stipulations accompanying the transmission of the data to academics at Georgia Tech, no such instructions or stipulations are held by the University.

    One more time

    Regulation 12(5)(f) applies to the data requested because the data was received by the University on terms that limits further transmission. We believe that there would be an adverse effect on the institutions that supplied data under those agreements as it would undermine the conditions under which they supplied the data to the Climate Research Unit. In regards your request for any stipulations accompanying the transmission of the data to academics at Georgia Tech, no such instructions or stipulations are held by the University.

    If your tea has not yet snorted out your nose, read again. It will hit you one of these times.

    • Pat Frank
      Posted Aug 13, 2009 at 7:39 PM | Permalink | Reply

      Re: Ross McKitrick (#4), Right. There are stipulations on the data, with the exception that there are no stipulations on the data.

      Obviously, under such nonstipulated stipulations, we cannot supply the requested data.

    • Posted Aug 13, 2009 at 8:29 PM | Permalink | Reply

      Re: Ross McKitrick (#4),

      It’s good to know someone finally found a use for triplespeak.

      Only in climatology.

      • JS
        Posted Aug 13, 2009 at 8:42 PM | Permalink | Reply

        Re: Jeff Id (#25), is it Orwellian or Kafkaesque? I think Kafka is more relevant than Orwell here. But doublethink is undoubtedly an advantage when writing such letters.

      • Ryan O
        Posted Aug 13, 2009 at 9:33 PM | Permalink | Reply

        Re: Jeff Id (#25), Don’t you know? This method of communication is insensitive to the sign of the input.

  5. curious
    Posted Aug 13, 2009 at 6:22 PM | Permalink | Reply

    I think this is a stalling tactic whilst they stomach pump the pup. IMO this will need an onside UK politician to expedite it – otherwise it is just going to be climb aboard the merry-go-round of back and forth requests which would ultimately require a formal complaint and all the delay and aggro that would entail.

    Claiming:

    We believe, following DEFRA guidance, that it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available.

    when all they have to do is resend the data already sent to Webster is pathetic. The EIR states they have to make every effort to assist. Or am I missing something – are they claiming that the website version is what they supplied to Webster?

    And I’m not sure about the last line – is that a cut and paste thing or as per the original?:

    “I apologise that your request will be met but …”

    I await the next instalment with interest!

  6. curious
    Posted Aug 13, 2009 at 6:26 PM | Permalink | Reply

    Ross – kettle on!! 8<~0

  7. Ross McKitrick
    Posted Aug 13, 2009 at 6:31 PM | Permalink | Reply

    If you enjoyed the first installment of this British farce, here’s the second.

    We believe that Regulation 12(4)(b) applies to your request for the data because the requested data is a subset of data already available from other sources; namely the Global Historical Climatology Network (GHCN ), and the Climatic Research Unit already makes requested information available on it’s website in a gridded format. We believe, following DEFRA guidance, that it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available.
    In regards Regulation 12(5)(a), much of the requested data comes from both individual scientists and institutions from countries around the world. If this information were to be released contrary to the conditions under which this institution received it, it would damage the trust that other national scientists and institutions have in UK-based public sector organisations and would likely result in them becoming reluctant to share information and participate in scientific projects in future. This would damage the ability of the University and other UK institutions to co-operate with meteorological organisations and governments of other countries.

    We can’t send you the data since it’s already available in so many places, including the CRU web site. Furthermore, we can’t give it to you since we’re forbidden from making it available.

    Maybe Hugh Laurie and Stephen Fry could work up a sketch around this.

    • Scott Brim
      Posted Aug 13, 2009 at 7:28 PM | Permalink | Reply

      Re: Ross McKitrick (#7),

      … We believe that Regulation 12(4)(b) applies to your request for the data because the requested data is a subset of data already available from other sources; namely the Global Historical Climatology Network (GHCN ), and the Climatic Research Unit already makes requested information available on it’s website in a gridded format…

      Is there an implication here, deliberate or otherwise, that the processed, value added data transmitted by CRU to qualifying academic researchers is fully equivalent to the raw source data sent to CRU by various NMS’s, but is simply being converted to another format before transmission? If this is so, then what specifically is the nature of the “value” CRU is adding?

    • Antonio San
      Posted Aug 13, 2009 at 7:59 PM | Permalink | Reply

      Re: Ross McKitrick (#7),

      It’s a sketch right out of “Little Britain
      “http://www.bbc.co.uk/comedy/littlebritain/
      … “I want that one…”

    • Trowser Trumpet
      Posted Aug 13, 2009 at 8:15 PM | Permalink | Reply

      Re: Ross McKitrick (#7),

      and the Climatic Research Unit already makes requested information available on it’s website in a gridded format. We believe, following DEFRA guidance, that it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available. …
      [My underlining. TT]

      I thought the data sent to Peter Webster contained the raw data and that that was therefore at least part of what you requested. And since the gridded data is not the same thing nor even equivalent to the raw data (at least without the code that converts the raw data to the “added-value” gridded data, and probably even with it too) why are they implying the two things are the same and that the difference now is merely one of format? WTF?

      Have I got that right?

      Also:

      and the Climatic Research Unit already makes requested information available on it’s website
      [My underlining. TT]

      That doesn’t say “already makes all requested information available” or even “already makes the information you requested available“. Sure, “all” is logically implicit. But still, they might be playing word games here too.

      I always put my shoes on when I go out.
      What, all 5000 pairs, Imelda?

    • Adam Soereg
      Posted Aug 14, 2009 at 8:38 AM | Permalink | Reply

      Re: Ross McKitrick (#7),

      This reply is plain nonsense.

      They say that the requested information is publicly available (which is partially true, because the CRUTEM dataset almost entirely rely on station data archived in GHCN) Despite this fact, they try to avoid releasing their data at all costs…

      • Steve McIntyre
        Posted Aug 14, 2009 at 8:45 AM | Permalink | Reply

        Re: Adam Soereg (#62),

        I’ve examined enough station data to say that the CRUTEM data is not a subset of GHCN. Even for stations that are GHCN stations, CRUTEM versions can differ materially.

        CRU’s statement that their data set is a “subset” of GHCN is simply untrue.

        In my opinion, they do themselves no good by making untrue statements, as their untruthfulness will be demonstrated. And, Ford, if someone makes untrue statements, they have only themselves to blame for the consequences even if you prefer to blame the person who merely reports that the statement is untrue.

  8. curious
    Posted Aug 13, 2009 at 6:37 PM | Permalink | Reply

    Are you sure this hasn’t come from Laurie and Fry already?? Please check the postmark! :)

  9. Richard Henry Lee
    Posted Aug 13, 2009 at 6:49 PM | Permalink | Reply

    David Palmer placed an apostrophe in the possessive pronoun “its” at “on it’s website”. Perhaps a [sic] should be placed after its inappropriate use.

  10. Posted Aug 13, 2009 at 6:54 PM | Permalink | Reply

    Did they just bump months before release to at least 6 months, 2010? –

    I’m starting to suspect that these guy’s aren’t being honest.

  11. JS
    Posted Aug 13, 2009 at 7:08 PM | Permalink | Reply

    It would seem like the file is just about ripe to send in full to the information commissioner. But, I suspect that a formal internal appeal is necessary before reaching that stage. The farce is amusing, but at some point an external oversight body needs to be brought in to formally comment on the ridiculousness of their behaviour.

  12. Calvin Ball
    Posted Aug 13, 2009 at 7:14 PM | Permalink | Reply

    If I can paraphrase:

    1) Its a secret.
    2) It’s publicly available.

    Is that about right?

  13. curious
    Posted Aug 13, 2009 at 7:17 PM | Permalink | Reply

    JS – see page 4 and 5:

    http://www.ico.gov.uk/upload/documents/library/freedom_of_information/practical_application/ico_foi_eir_how_to_comp_final.pdf

  14. Calvin Ball
    Posted Aug 13, 2009 at 7:19 PM | Permalink | Reply

    Let me revise that.

    1) Its a secret.
    2) It’s publicly available.
    3) It’s a secret that we spilled the beans to Georgia Tech.
    4) We don’t have a bean-spilling agreement with Georgia Tech.
    5) If we spill the beans to you, some of the other people to whom we spilled the beans might not spill their beans to us in the future.
    6) Your beans don’t count.

    Anything else?

  15. James S
    Posted Aug 13, 2009 at 7:46 PM | Permalink | Reply

    I received something similar to my request:
    http://www.whatdotheyknow.com/request/raw_climate_data#incoming-39657

    I’m going to use the weekend to decide how to respond to them!

  16. Paul Penrose
    Posted Aug 13, 2009 at 7:49 PM | Permalink | Reply

    They have to know you are going to publish these responses, and knowing this how can they make such ridiculous claims? Don’t they know how stupid it makes them look?

    • PaddikJ
      Posted Aug 16, 2009 at 2:51 PM | Permalink | Reply

      Re: Paul Penrose (#19),

      Which kinda puts puts the quietus on Steve & Lucia’s bend-over-backwards-to-be-reasonable alternative explanations (“Maybe they’re just prima-donnas”, etc), ie: possible but vanishingly unlikely, ie: the laughably transparent stonewalling may make them look stupid, but not nearly as stupid as the state of the data would were it to become public. Least of all evils.

      BTW, I’m still undecided – I don’t think Full Disclosure would prove catastrophic AGW is a hoax; more likely it would expose laughably non-existent data management practices.

  17. Trowesr Trumpet
    Posted Aug 13, 2009 at 8:17 PM | Permalink | Reply

    Huh! My underlining disappeared.

    I’ll have to rejig things. I’ll do Mannian transformation.

    Where’s my magic wand?

  18. Brooks Hurd
    Posted Aug 13, 2009 at 8:22 PM | Permalink | Reply

    I would like to hear Ali G read these CRU responses.

  19. Trowser Trumpet
    Posted Aug 13, 2009 at 8:25 PM | Permalink | Reply

    Rejigged attempt of my #21 (now with my bold in the block quotes).

    Re: Ross McKitrick (#7),

    …and the Climatic Research Unit already makes requested information available on it’s website in a gridded format. We believe, following DEFRA guidance, that it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available. …
    [My bold. TT]

    I thought the data sent to Peter Webster contained the raw data and that that was therefore at least part of what you requested. And since the gridded data is not the same thing nor even equivalent to the raw data (at least without the code that converts the raw data to the “added-value” gridded data, and probably even with it too) why are they implying the two things are the same and that the difference now is merely one of format? WTF?

    Have I got that right?

    Also:

    …and the Climatic Research Unit already makes requested information available on it’s website …
    [My bold. TT]

    That doesn’t say “already makes all requested information available” or even “already makes the information you requested available“. Sure, “all” is logically implicit. But still, they might be playing word games here too.

    I always put my shoes on when I go out.
    What, all 5000 pairs, Imelda?

  20. jae
    Posted Aug 13, 2009 at 8:53 PM | Permalink | Reply

    Really, I feel bad to see anyone discredited. But, when folks are at the bottom of a hole and just keep digging to get out of it, I simply cannot be sympathetic. I must laugh at them!

  21. pete m
    Posted Aug 13, 2009 at 9:02 PM | Permalink | Reply

    I apologise that your request will be met

    You guys have missed the best typo of the lot.

    lols.

  22. BradH
    Posted Aug 13, 2009 at 9:07 PM | Permalink | Reply

    Does anyone else consider this to be a very, very strange issue one which to make a final stand and utterly destroy the reputation of your institution?

    • Trowser Trumpet
      Posted Aug 13, 2009 at 9:32 PM | Permalink | Reply

      Re: BradH (#29),
      It was a very, very strange issue right from the start. The pace has picked up somewhat recently but otherwise it seems to me to be business as usual with the screamers.

  23. AnonyMoose
    Posted Aug 13, 2009 at 9:35 PM | Permalink | Reply

    (They’re going to have to re-do their Roger Pielke rejection, since they replied to the wrong request in his case.)

    The climate record accountants’ FOI record keepers messed up their record keeping?

  24. steven mosher
    Posted Aug 13, 2009 at 9:40 PM | Permalink | Reply

  25. a jones
    Posted Aug 13, 2009 at 9:53 PM | Permalink | Reply

    I posted on this in WUWT when it first appeared and that I would make some inquiries but that I expected it would take some time, not least because it was coming into August, holiday season etc.

    So the impossible we do immediately miracles take a little longer.

    I note that Prof. Jones, NO RELATION, has got himself in a tizz over this perhaps for a number of reasons I can guess at.

    You must understand that for this purpose there are three tiers in the UK system HMG departments themselves which answer to a Permanent Secretary, a civil servant, and thence to the Minister, UK government agencies, which are quasi autonomous and carry out specific functions for HMG departments, and finally many universities whose research is funded by HMG: as is the case with CRU.

    Now apart from knowing which departments you need to apply with an FOI request, you should also know that whilst one set of regulations may apply to one department of HMG, they do not necessarily apply to another. Wrong department wrong FOI request, wrong regulations.

    Now as I suspected the CRU data and the Met Office data is also lodged, in whole or in part, with other HMG Departments, and I think an agency as well.

    At the moment I cannot confirm whether this is raw data or adjusted data or how complete the records are.

    Moreover as I thought research funded by HMG, whether it is a university and thus CRU or indeed the IPCC itself, probably cannot be covered by formal confidentiality aggreements. These can only apply in the UK if either the research is for commercial purposes, such as the association of widget makers asking for help from universities to improve their widget making process, or to the anonymity of unpaid individuals responding to surveys, such as what do you think about a blue moon?

    What is more if any restrictive agreement outside of the above exists it has to be notified to the appropriate HMG department.We don’t yet know what that department might be but we can ask the law departments, who have to answer, FOI or no FOI, to look it up and tell us. Don’t hold your breath, these boys can take weeks to answer a letter.

    Finally of course if the information is not covered by the Official Secrets Act, and I cannot see why it should be, then the governments of certain Crown Dependencies can ask HMG for it, and what might be confidential in the UK is not necessarily so in their jurisdiction. Can we persuade them to ask, I think so.

    So you see there is more than one way to skin a cat, but it is slow patchy work, and I suspect, nothing to do with me at this stage, Dr. Jones may decide, or possibly has been advised, that making a bare breast might be a politically desirable move at this point.

    snip – p[olicy

    I shall continue to press for some answers and advise, but again don’t hold your breath, it may be that given the shifting political sands you will get the answers quicker than I can. I do hope so, but if not I will get them eventually.

    Kindest Regards.

  26. AnonyMoose
    Posted Aug 13, 2009 at 9:57 PM | Permalink | Reply

    In regards your request for any stipulations accompanying the transmission of the data to academics at Georgia Tech, no such instructions or stipulations are held by the University.

    The phrasing in the “non-academics” reply was:

    Regulation 12(5)(f) applies because the information requested was received by the University on terms that prevent further transmission to non-academics

    It appears that they sent the data to Georgia Tech without telling Georgia Tech of any restrictions. And perhaps with good reason; if 12(5)(f) does not apply to Georgia Tech then Georgia Tech would not be bound by such a restriction. The “Environmental Information Regulations 2004″ look like they are UK regulations, thus Georgia Tech doesn’t have to follow 12(5)(f).

  27. Walter Manny
    Posted Aug 13, 2009 at 10:27 PM | Permalink | Reply

    I sent in a request and received a much more reasonable reply than the above, one which I hope will clear things up for one and all:

    “We believe that Regulation 13(a)(/z) applies to your request and/or a subset of your request insofar as said item was originally transmitted and subsequently directed to our department or departments, pursuant to the Official Responses to Academic Entities Act (1856). Further, the data set requested is subject to pre-existing non-disclosure and/or disclosure agreements directly or indirectly associated with our institutional suppliers, which suppliers are either formally aligned with or subsidiary to select southeastern academies and sister institutions in the colonies, the association with which your particular institution is not an ex-nonmember.

    Pursuant to the filing of your request, our attorneys have advised us to note, not that it’s any of your damn business but because a we’re a friendly lot, the considerable risk posed by shards of raw data sets (some of which are uncalibrated) flying out of unguarded windows and damaging relationships with allied countries or other entities latitudinally congruent with or variously tangent to the antipodes. Public resources accruing to our institutional sector for FOI compliance have previously vectored horizontally to prioritized allocation zones such as this week’s office party, at which time there will be conducted an informal assessment of the degree to which the reluctance of our best buddies to share information and participate in scientific projects in the future correlates with or indeed causes damage to the ability of the University and other UK institutions to co-operate or otherwise party with friendly meteorological organisations and governments of other countries.”

    Expecting data to be FedExed shortly.

  28. Rathtyen
    Posted Aug 13, 2009 at 11:03 PM | Permalink | Reply

    I’ve always loved the British sense of humour. I think you are all being taken for a ride here. The refusals are obviously penned by someone who desperatley wanted to be one of the Monty Python crowd. He wears Eric Morecombe glasses, drives a Mr Bean mini, and does that disturbing Benny Hill funny walk. He proably even has a Steptoe fashion sense (I don’t think these guys get out much) and his betamax VCR is in meltdown watching “Carry On Up the Weather”.

    The real problem is, I’m sure he fits in at CRU just fine.

    Lets face it, its the only explanation that really makes sense!

  29. Geoff Sherrington
    Posted Aug 13, 2009 at 11:18 PM | Permalink | Reply

    Elsewhere I mentioned a Royal Commission on espionage in Australia as preserving minute detail in great quantity from the mid 1970s, in the context that the technology existed and that data can be retrieved today.

    One statement that the above discussion brings to mind is -

    “In fact, the study of security and intelligence over these years leaves a continuing, unrelenting impression of a complete vacuity of objectives and astonishing ineptitude in execution.”

    Can’t you image His Honour saying words like this from the Bench eventually in this matter?

  30. Ian
    Posted Aug 14, 2009 at 12:51 AM | Permalink | Reply

    My request pre-dated the recent set by a week or two and seemed to get lost by the influx for which David has apologised. My request for the agreements has apparently been satisfied by that new web page, but I add the following couple of nuggets, one of which was rejected with the to much effort defence (s12), so I’ll be appealing that one on the grounds that the request does not require them to go back over 15 years to satisfy the question given that he’ll have an inbox full of such information over the last month or so.

    3,. and where not available what steps have been taken by UEA, Dr Jones or the Met Office to confirm these terms and determine if they still stand.

    See answer to Question 1. A search has been conducted for all extant agreements and all have been placed on the above-noted website

    4. I ask also for any email records referring to these agreements either within or without UEA, either to or from Dr Jones, or any third party who has tried to confirm the existence of such agreements

    [Information not released pursuant to s.12, Freedom of Information Act]

  31. Jon-Anders Grannes
    Posted Aug 14, 2009 at 1:02 AM | Permalink | Reply

    GISS/Hansen use Oslo up to the 1980s and get the full UHI from increase in population from 10.000 in 1815 to 500.000 in the 1980s.
    Then they change to Gardermoen 30 miles North that is very rural at that time, but later in the late 1990s become the main airport for Oslo and is now beeing urbanized much and fast. Probably give a lot of renewed UHI effect today and in the future?

    I wonder where they will jump next, when Gardermoen has been fully urbanized and there is little further increase in UHI effect?

    UHI jumping?

  32. Posted Aug 14, 2009 at 1:02 AM | Permalink | Reply

    Steve and Ross

    You have done things by the book to date.

    Don’t you think it is time to eascalate this to the British Press?

    Both Christopher Booker-Daily Telegraph- and Alexander of the Daily Mail might get some traction and they also know various MP’s who might take it up. Also our local Euro MP is from UKIP- a party who have an official Sceptical policy towards climate change. They might also be willing to ask questions in the right places.

    Tony Brown

  33. Jon-Anders Grannes
    Posted Aug 14, 2009 at 1:12 AM | Permalink | Reply

    “We can’t send you the data since it’s already available in so many places, including the CRU web site. Furthermore, we can’t give it to you since we’re forbidden from making it available.

    Maybe Hugh Laurie and Stephen Fry could work up a sketch around this.”

    This is more Monty Phyton Flying Circus?

  34. Posted Aug 14, 2009 at 2:13 AM | Permalink | Reply

    Here is the response I got.

    Pursuant to your rights under section 1(1)(a) of Freedom of Information Act 2000 to be informed whether information is held, this letter is to formally advise you that we do not hold the requested information for the requested countries save that relating to Cyprus.
    In regards Cyprus, it is not possible to provide the information directly. In accordance with section.17 of the Freedom of Information Act 2000 this letter acts as a Refusal Notice, and I am not obliged to supply this information. The reasons for exemption are as stated below:
    Exemption Reason

    s.21(1)(a), Information reasonably accessible to applicant by other means All of the requested information can be obtained from the Climate Research Unit website

    All written agreements that we possess in relation to any data received from any country or geographic area are now all available via the Climate Research Unit website at: http://www.cru.uea.ac.uk/cru/data/availability/. This page also has information regarding the compilations, processing and handling of any data received. The manner in which the station data was collected, the changes in national boundaries and entities, and the nature of the processing of the data by the CRU have all evolved & changed over the time of collection of the data. This means that there is not always a direct correlation between any agreement and the geographic location of the data or stations that the agreement covers

  35. Pat
    Posted Aug 14, 2009 at 2:22 AM | Permalink | Reply

    Oh be reasonable-its obvious that the dog ate their homework and they’re too embarrassed to say.

  36. thefordprefect
    Posted Aug 14, 2009 at 3:18 AM | Permalink | Reply

    Mr McIntyre
    I repeat
    If Georgia Tech has a copy then get it from there. They will surely honour FOI requests, even if Peter Webster verbally agreed not to distribute.
    You do not reply.

    It is quite plausible that Georgia T and UEA have no documents concerning the restrictions on the data as I though the data was passed as part of a collaboration between webster and jones.

    Why not believe that the data is hamstrung by confidentiality agreements?
    Why not believe that they are now working to remove those agreements?
    A couple of months has been suggested before this can happen. Can you not wait? Or is it more important to destroy the reputation of CRU?

    Mike

    • EdBhoy
      Posted Aug 14, 2009 at 6:03 AM | Permalink | Reply

      Re: thefordprefect (#45),
      It’s not Steve that is destroying CRU’s reputation it is their own lack of competence. There is never any good scientific reason for hiding your data and there doesn’t appear to be a significant number of confidentiality agreements to prevent them from doing so. If you can see the consequences upon their credibility of their lack of cooperation on this issue why cant they? The continued stonewalling will produce nothing but negative press.

      • thefordprefect
        Posted Aug 14, 2009 at 6:40 AM | Permalink | Reply

        Re: EdBhoy (#52), Scientifically they should not hide their data – LEGALLY they have to (would you copy music/videos and pass them on?).

        They say they are trying to free the data – is this not acceptable. If not then try FOI on Peter Webster at GT. He has the data!

        • Steve McIntyre
          Posted Aug 14, 2009 at 7:06 AM | Permalink

          Re: thefordprefect (#53),
          Ford, I don’t think that you’ve fully followed the chain of thought here.

          CRU’s collection of station data was funded by U.S. nuclear labs. As we know, CRU delivered its station data collection, as it then stood, to CDIAC, a unit of the Oak Ridge nuclear lab, in the early 1990s, who then placed it online, where it remains to this day.

          Previously, no one would have thought that there was anything untoward about that.

          however, CRU has now informed us that its data from that period was collected under various confidential agreements that prohibit the present data set from being transmitted to me, Roger Pielke Jr, Ross McKitrick, Roman Mureika and others, but not to Peter Webster.

          unfortunately, they are unable to produce any agreements that contain language that permit them to make this distinction.

          if the language of the few agreements presently online is representative of the “lost” agreements, as CRU allege, then it appears to me that CRU never paid the slightest attention to these agreements. For example, one of their agreements requests 1961-1990 normals for a specific academic project and states that the data will only be used for the purposes of that project and will not be passed on to third parties. While they interpret than language against me, the language is much broader than that – it prohibits the transmission of the data to Peter Webster, to CDIAC and to the Met Office and its use in the construction of the CRUTEM index.

          Their request letter gives no indication that their collection is funded by the Oak Ridge nuclear lab in the U.S. and that they planned to pass the information on to the Oak Ridge nuclear lab.

          In my opinion, CRU would be better off making the decision that the alleged early confidential “agreements” were ineffective or superceded by WMO Resolution 40 or something like that, rather than playing out the hand. However, I often make sensible suggestions to people, that they disregard.

        • thefordprefect
          Posted Aug 14, 2009 at 7:42 AM | Permalink

          Re: Steve McIntyre (#54),

          In my opinion, CRU would be better off making the decision that the alleged early confidential “agreements” were ineffective or superceded by WMO Resolution 40 or something like that, rather than playing out the hand. However, I often make sensible suggestions to people, that they disregard.

          I believe you said that Peter Webster at GT was to use the data in a joint project with CRU. Legally how does this pan out? [Steve: I didn't say this; Peter was reported as saying this.]

          You seem to be suggesting that CRU forget about legal issues and pass the data? Would this be wise for CRU? [Steve- I am absolutely not saying that CRU forget about legal issues. However, I am completely unconvinced by their legal reasoning. Thus far, they haven't provided a single shred of evidence to support their position that they can provide data to CDIAC, the Met Office, Advance 10K dendrochronologists, Peter Webster, but not to Warwick Hughes, Ross McKitrick, Hu McCulloch, Roger Pielke, Roman Mureika or me. I do not believe that there is a valid reason.]

          To me it seems that your one aim is to ridicule the CRU and destroy its reputation. If this were not the case then why do you carry out these actions in full view on a blog? It can only be to provide maximum embarassment to CRU. [Steve - my objective is to make the data public.]

          May I suggest that you wipe all threads pertaining to this issue and wait for CRU to free up the data? [ Steve- Again, I do not believe that there is any valid legal reason why CRU can't provide the data right now and intend to pursue my own appeals and report on them.]

          These FOI must be costing a small UK university a fortune to process yet you enthusiastically encourage your followers to continue. The funding For the CRU research is running at about UKP40,000 per month Your actions must be costing them a sizeable proportion of this. By continuing this action there is a possibility that such funding goes elswhere in future. [Steve - they have a FOI officer on staff.]

          This FOI attack is obviously being organised on this site. Be aware of the things I privately emailed you a few months ago!!!!!

          Why can you not accept that the data is commercial?
          Why can you not accept that CRU are trying to free up the data?
          Why continue FOIing attacks?
          [Steve - I have seen no evidence that the data is commercial. They've turned data over to CDIAC and others without compunction. I have seen no evidence that CRU has made the slightest attempt in the past 5 years to "free up the data". On the contrary, over the past few weeks, they've deleted data that has been available for up to 13 years.]

          This blog was once a source of honesty. It has decended into hatred. I am saddened by this!
          Mike [ Steve: I strongly disagree. I for one do not "hate" anyone. I dislike bureaucratic obfuscation and don't mind taking the extra effort to oppose it. contrary to your characterization of the blog, I think that one of the great strengths of blogs is the ability to expose bureaucratic obfuscation. Rather than get frustrated with stonewalling, one can place sunshine on it.]

        • steven mosher
          Posted Aug 14, 2009 at 10:18 AM | Permalink

          Re: thefordprefect (#56),

          Ford. As you can see from my FOI CRU have denied my request for their PROCEDURES ( read employee handbook)
          that govern entering into confidential agreements. They determined it would take them more than 18 hours to
          find this. So, I don’t see how you support your claim that this is costing a small fortune.

        • Ross McKitrick
          Posted Aug 14, 2009 at 12:51 PM | Permalink

          Re: thefordprefect (#56), I hope you will at least acknowledge that these recent efforts by CRU to liberate the data only began in response to the pressure on this site. If you think the application of so much public pressure makes the various parties awkward and uncomfortable, well so do I. But that’s apparently what it took to get the ball rolling.
          .
          For the record, I don’t expect there are any clerical errors, data shenanigans or computational problems in the CRU compilation, and if there are glitches I doubt they would affect the global average very much. I don’t expect there to be any major problems in my bank’s annual financial statements either. But I feel much more confident about the latter knowing that there are independent auditors going over the books. My confidence in their accounting systems does not mean that I would support a suspension of the laws governing audits of public corporations.
          .
          As an academic matter I would like to check the extent to which spatial correlations between surface grid cell trends and indicators of nonclimatic contamination differ between the CRU input data and their gridded output data. I can only do that if I have the exact input data. If they refuse to release it then their claims to have removed the nonclimatic biases cannot be verified.
          .
          The other issue concerns what status should be given to CRU data. If they are prepared to issue an “as-is” warning, saying that the rules under which they operate make it impossible for them to permit independent replication of their work, and they cannot guarantee that even they would be able to replicate it themselves since they may not even have their original data, then they could be excused for this secrecy, since users would have fair warning. But they put their data forward as a global record of sufficient reliability on which to base public policy, international treaties, downstream scientific research, etc. If they want that prestige, they need to accept the quality control commensurate with it. It’s a question of not having things both ways.

        • steven mosher
          Posted Aug 14, 2009 at 2:33 PM | Permalink

          Re: Ross McKitrick (#90),

          I for one want to find the country that says the following.

          1. Our data is too valuable to make public.
          2. We support public policies based on this secret data that will cost YOU trillions.

          In short the world is considering global policies that have rather large costs. Those policies in part derive their
          justification from this data. On principal that data should be available to those it would impact. Colin Powell in front of the UN comes to mind.

    • Gerald Machnee
      Posted Aug 14, 2009 at 7:57 AM | Permalink | Reply

      Re: thefordprefect (#45),

      I repeat
      If Georgia Tech has a copy then get it from there. They will surely honour FOI requests, even if Peter Webster verbally agreed not to distribute.

      Well, you send an FOI to Webster or Tech and let us know.
      You say Georgia Tech has a copy. I say CRU has a copy. Why can’t Steve get it there???
      Have you asked Jones to explain his statement of “Why should I give you the data so you can just find something wrong with it?”

      Steve: I am not going to ask Peter Webster for the data. Nor do I want any CA readers to do so. It’s CRU’s obligation to provide the data or provide an explanation of why they won’t that is valid. Thus far, they haven’t done either.

      • thefordprefect
        Posted Aug 14, 2009 at 8:25 AM | Permalink | Reply

        Re: Gerald Machnee (#57), Comment by McIntyre

        I am not going to ask Peter Webster for the data. Nor do I want any CA readers to do so. It’s CRU’s obligation to provide the data or provide an explanation of why they won’t that is valid. Thus far, they haven’t done either

        And it would not fit in with your Vendetta against CRU

        The data is not CRUs to give in the same way it is not Webster’s.
        The data is not given because it is commercial.

        Steve: You say: “The data is not given because it is commercial. What do you not understand?” They’ve given a variety of reasons for refusing the data, commencing with the reason given to Warwick Hughes. However, the most recent reason given to me is that the y have confidentiality agreements that specifically prohibit transmission of the data to a “non-academic”. Perhaps this is not the “true” reason and the “true” reason is the different one that you propose here. However, I am guided by CRU’s actual response.

        .

        • EdBhoy
          Posted Aug 14, 2009 at 8:59 AM | Permalink

          Re: thefordprefect (#60),
          You mention that CRU costs the tax payer £40000 per month. Public money comes with public responsibilities that include transparency. Why do they not want to publish the raw data? Steve asked politely for the data and they refused to give it to him for reasons that cannot be justifed academically or legally. The few confidentiality agreements they have published this week are hardly a serious obstacle to passing the data to Steve.

          The UK government and the rest of the developed world are committing themselves to trillions of dollars of expense in carbon trading, clean coal technology etc. etc. based partly on the CRU data. We have to be able to audit it independently and Steve is ideally placed to do so. Phil Jones’ exclamation that Steve will just try to find fault with it is ridiculous. Every climate scientist or statistician should be trying to find faults in the data as that is how scientific method works. If they succeed they may save the world a huge fortune, if they fail then the world will thank them for helping to raise the Global Warming alarm.

          This is not an attack on CRU. The issues are much bigger than that.

        • thefordprefect
          Posted Aug 14, 2009 at 9:12 AM | Permalink

          Re: EdBhoy (#68),

          You mention that CRU costs the tax payer £40000 per month. Public money comes with public responsibilities that include transparency. Why do they not want to publish the raw data? Steve asked politely for the data and they refused to give it to him for reasons that cannot be justifed academically or legally. The few confidentiality agreements they have published this week are hardly a serious obstacle to passing the data to Steve.

          I said they had funding I did not say it was public.
          They have said they will work toward publishing the data.
          It is commercial data!!!!!!!
          You too suggest that CRU acts illegally and distributes other peoples property!

          Steve: Ford, understand what you’re accusing CRU of. You’re accusing CRU of acting illegally for the past 15 years. That’s not us making the allegation, it’s you. They’ve sent the “commercial” data to CDIAC, to Advance 10K dendrochronologists, posted two separate versions on their website, sent it to Met Office, apparently used data requested for a project on 1961-90 normals for inclusion in CRUTEM and HadCRU and in 2002 even sent the data to me. According to you, all these actions are illegal. I hope that you have some evidence to back this up. My own position is different: I assume that CRU has been acting legally all these years. If they’ve been acting legally in the past, then they should send me the data. If you insist on maintaining your accusations that CRU has acted illegally in the past, then I guess the chips will fall where they may, but understand that you’re making the accusation, not me.

        • EdBhoy
          Posted Aug 14, 2009 at 9:42 AM | Permalink

          Re: thefordprefect (#70),
          CRU have not been able to demonstrate any significant legal impediment to being able to distribute the data to Steve or Ross McKittrick etc.. If they could do so the FOIs would fail. I don’t know why they continue to stonewall, it just doesn’t do CRUs reputation any good whatsoever. Since they did give the data to Steven Webster, the Met Office and others it seems that they discriminate when deciding who should or should not get the data. If it were simply a legal issue they would not have passed it to anyone.

          Mike, there is a bigger issue here. Every scientist with an interest in climate change should be demanding that the raw unadulterated data be freely available and that any method used to process the raw data be published. The costs of getting this wrong on either side of the argument are far too high for us to blindly accept unreproducible results.

        • henry
          Posted Aug 14, 2009 at 9:16 AM | Permalink

          Re: thefordprefect (#60),

          The data is not CRUs to give in the same way it is not Webster’s.

          Yet CRU gave it to Webster.

  37. curious
    Posted Aug 14, 2009 at 3:36 AM | Permalink | Reply

    Mike – CRU are destroying their own reputation. Just read the words.

  38. Pierre Gosselin
    Posted Aug 14, 2009 at 3:55 AM | Permalink | Reply

  39. Pierre Gosselin
    Posted Aug 14, 2009 at 4:01 AM | Permalink | Reply

    curious,
    That’s the question I raised not long ago. Why could not releasing the data be worth destroying their entire reputation?
    Steve said he doesn’t expect to find anything surprising in the data. Who knows, it could be that Steve would discover that global temps have been even understated. It’s kinda like risking a major car accident to swat a fly in your car.

  40. Jonathan
    Posted Aug 14, 2009 at 5:04 AM | Permalink | Reply

    Just got my refusal; the same laughable excuses. I shall of course be appealing.

    • Jonathan
      Posted Aug 27, 2009 at 5:13 AM | Permalink | Reply

      Re: Jonathan (#49), I have now lodged my appeal as follows.

      Dear Mr Palmer,

      Thank you for your letter dated 14 August, reference ENVIRONMENTAL INFORMATION REGULATIONS 2004 – INFORMATION REQUEST (FOI_09-117; EIR_09-14) in response to my request for “a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009″.

      I regret that I do not consider your response satisfactory, and am therefore appealing your decision. As I understand you are currently on holiday I am copying this to Bob Heath (r.heath@uea.ac.uk) and Jane Baker (jane.baker@uea.ac.uk) as you requested in your vacation message.

      You have refused my request on three grounds, all of which are incorrect.

      1. Reg. 12(4)(b) – Request is manifestly unreasonable: Information is available elsewhere.

      You claim that “the requested data is a subset of data already available from other sources” namely the gridded data made available by the GHCN and the CRU. It is factually incorrect to claim that “the requested data is a subset of data already available from other sources” and your argument cannot stand. A “subset of data already available” would mean that the data I requested could be obtained from “the gridded data made available by the GHCN and the CRU” by downloading some or all of this data and deleting selected parts. The data I have requested cannot be obtained in this manner. I refer you to the discussion of the gridding process at http://www.cru.uea.ac.uk/cru/data/landstations/ .

      You further claim that “it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available”. However I asked for “a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009″. I have only requested a copy of a data set which has already been prepared by the university, and so is already available. Once again your statement is factually incorrect and your argument cannot stand.

      2. Reg. 12(5)(a) – Adverse effect on international relations: Release would damage relations with scientists & institutions from other nations
      3. Reg. 12(5)(f) – Adverse effect on the person providing information: Information is covered by a confidentiality agreement

      I will take these two points together as they are in essence the same. I begin by noting that it is wholly perverse to claim simultaneously that the data is “already available” and that the data is “confidential”. Clearly these two statements cannot simultaneously be true.

      With regard to Reg. 12(5)(a) you state that releasing this information “would damage the trust that other national scientists and institutions have in UK-based public sector organisations” and consequently “would damage the ability of the University and other UK institutions to co-operate with meteorological organisations and governments of other countries”. I draw your attention to resolution 40 of the World Meteorological Organization which states that “WMO commits itself to broadening and enhancing the free and unrestricted international exchange of meteorological and related data and products”. It is perverse to claim that acting in accordance with this resolution could endanger cooperation with meteorological organizations.

      With regard to Reg. 12(5)(f), the data I requested has already been provided to at least one other individual, namely Peter Webster at Georgia Tech. Clearly this data cannot be covered by a strict confidentiality agreement.

      It is, of course, true that this data could be covered by limited confidentiality agreements. The FOI and EIR are quite clear on the responsibilities of organizations claiming exemption on grounds of confidentiality. The exemption “only applies if a breach of confidence would be ‘actionable’”. Courts will only recognise that a person holds information subject to a duty of confidence in two types of situations:

      a) where that person expressly agrees or undertakes to keep information confidential: there is an express duty of confidence

      b) where the nature of the information of the circumstances in which the information is obtained imply that the person should keep the information confidential: there is an implied duty of confidence

      From your letter it appears that UEA is claiming an exemption of the first kind, as you cite a number of supposed confidentiality agreements that you do hold, which are available at http://www.cru.uea.ac.uk/cru/data/availability/ . In fact the great majority of these are not clearly confidentiality agreements:

      a) The 1994 FAX to the Met Office is simply a statement from Dr Hulme about the planned use of the data; there is no reply as to the conditions under which the data is provided.

      b) The 1993 letter from DNMI is a limited request for confidentiality not a formal agreement, and is almost certainly superseded by WMO Resolution 40. If UEA wishes to claim exemption under this clause it must first establish with DNMI that an express duty of confidentiality still applies.

      c) The form in Spanish simply states that the data should only be used for the specified purpose, and as no purpose was specified this cannot establish a duty of confidentiality.

      d) The web page is simply a statement by the Met Office of its own policies; this provides no evidence whatsoever of any duties under which UEA might hold data. It further notes that NERC data centres may make the data available under certain circumstances, so there is no absolute duty of confidence.

      e) The 1994 letter from Bahrain International Airport is a limited request for confidentiality not a formal agreement, and is almost certainly superseded by WMO Resolution 40. If UEA wishes to claim exemption under this clause it must first establish with Bahrain International Airport that an express duty of confidentiality still applies.

      I understand that in the past UEA has refused to release the data I have requested and related data because the request came from a person who was not an academic. I remind you that “No regard may be had to the identity of the person who is requesting the information nor to the purpose to which they will put the information.” I also remind you that “When considering the balance of interests, public authorities must have regard to the interests of the person to whom the duty of confidence is owed; the public authority’s own interests in non-disclosure are not relevant to the application of this exemption.” I further remind you that “If you receive a request for information which, although it was confidential when it was obtained, was obtained a long time ago, you should consider carefully whether the disclosure of that information would still constitute an actionable breach of confidence within the meaning of section 41.”

      At best UEA has limited evidence for the existence of limited confidentiality agreements covering part of the data I have requested. It is not clear to me that these documents in any way establish an express duty of confidence. However, even if they do, the responsibilities of UEA under Reg. 12(11) of the EIR are clear.

      Regulation 12 (11) says: (11) Nothing in these Regulations shall authorise a refusal to make available any environmental information contained in or otherwise held with other information which is withheld by virtue of these Regulations unless it is not reasonably capable of being separated from the other information for the purpose of making available that information.

      Thus UEA is certainly required to provide me with all the data I have requested with the possible exception of data held under an express duty of confidence (for data withheld it is required to establish that such an express duty of confidence does in fact exist). Please note that if it is not possible to identify which data is covered by supposed confidence agreements, then it is difficult to maintain that the release of this data will breach such agreements.

      I therefore appeal your decision, and reiterate my request for “a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009″.

      • Jonathan
        Posted Aug 30, 2009 at 3:36 AM | Permalink | Reply

        Re: Jonathan (#121), I am promised a reply to my appeal by 24th September at the latest.

        • Jonathan
          Posted Sep 10, 2009 at 12:50 PM | Permalink

          Re: Jonathan (#122), RealClimate are busy congratulating themselves on the commitment to open data sharing in the climate science community. My comment about obfuscation at the CRU was not thought worthy.

      • Jonathan
        Posted Sep 22, 2009 at 2:20 PM | Permalink | Reply

        Re: Jonathan (#121), the saga continues!

        ENVIRONMENTAL INFORMATION REGULATIONS 2004 – INFORMATION REQUEST (Our ref: FOI_09-117; EIR_09-14)

        Further to my letter of 28 August 2009 in response to your appeal of 27 August, I am writing to update you on the appeals process as it applies to this request.

        I have had discussions with relevant staff regarding this matter and considered the grounds of your appeal closely. Upon reflection, I feel that the distance that exists between your position and that of the University as stated in my letter of 14 August 2009 is too great to be bridged by an initial informal attempt at resolution by myself.

        Given the above, and in attempt to expedite the process, I am hereby initiating the second stage of our internal complaint process as laid out in our Code of Practice for Responding to Requests, and am referring this matter to Mr. Jonathan Colam-French, Director of Information Services. As per our Code of Practice, Mr. Colam-French will respond to you within 28 calendar days of referral of this matter, effectively, by 20 October 2008.

        All information in my file on this matter has been passed to Mr. Colam-French for his review and he will advise you of the outcome of this review within the timeframe noted above.

        For your information, the complaint process is within our Code of Practice and can be found at: http://www1.uea.ac.uk/polopoly_fs/1.2750!uea_manual_draft_04b.pdf

        If you are dissatisfied with the final adjudication of your complaint by our internal complaint process, you have the right of appeal to the Information Commissioner at:
        Information Commissioner’s Office
        Wycliffe House
        Water Lane
        Wilmslow
        Cheshire
        SK9 5AF
        Telephone: 01625 545 700
        Website: http://www.ico.gov.uk

        Yours sincerely

        David Palmer
        Information Policy & Compliance Manager
        University of East Anglia

        We’ll see how this turns out.

  41. benpal
    Posted Aug 14, 2009 at 5:59 AM | Permalink | Reply

    The Register, 13 August 2009
    http://www.theregister.co.uk/2009/08/13/cru_missing/

    “The world’s source for global temperature record admits it’s lost or
    destroyed all the original data that would allow a third party to
    construct a global temperature record. The destruction (or loss) of the
    data comes at a convenient time for the Climatic Research Unit (CRU) in
    East Anglia – permitting it to snub FoIA requests to see the data.
    [...]“

  42. Posted Aug 14, 2009 at 7:23 AM | Permalink | Reply

    For the time being, can we get insight on at least simple average of all stations used in HadCRUT from the source-which-must-not-be-named? We already know it is not “influential”, but yet.. :D

  43. Merrick
    Posted Aug 14, 2009 at 7:31 AM | Permalink | Reply

    Lord Monckton is “our friend.” Do we know anyone in the House of Commons who might be able to raise this issue of flagrant misuse of FOI during questions to the PM? This whole issue really needs a “light of day” moment.

  44. Tom C
    Posted Aug 14, 2009 at 8:01 AM | Permalink | Reply

    #26

    Re: Jeff Id (#25), is it Orwellian or Kafkaesque? I think Kafka is more relevant than Orwell here. But doublethink is undoubtedly an advantage when writing such letters.

    I wouldn’t dignify these pathetic responses with such lofty expressions. They are more akin to the incoherent excuses that would pour forth from a boy with his hand in the cookie jar.

  45. Posted Aug 14, 2009 at 8:23 AM | Permalink | Reply

    How many studies are dependent on this secret data, directly or indirectly? Is this the data that is used to reconstruct these graphs?

  46. Severian
    Posted Aug 14, 2009 at 8:30 AM | Permalink | Reply

    This is been an amazing thing to follow. It’s getting eerily like a scene out of Brazil. Maybe you need a 27B stroke 6 form to get the data? Of course, they can’t give you a 27B stroke 6 form, because in order to get said form you need a signed, stamped 27B stroke 6 form in triplicate, and they can’t tell you how to proceed because that’s classified.

  47. Craig Loehle
    Posted Aug 14, 2009 at 8:48 AM | Permalink | Reply

    It seems possible to me that the official sending these denial letters has no clue that the data requested differs from the gridded data available at the cru website. That someone like Ross might want to compare, let’s say individual citys or countries weather trends to their economic status, as he did in a publication or two, may escape them. To this official, perhaps data is data and he thinks that requests are just asking for special favors in format or something. Assume ignorance before bad motives.

    • Scott Brim
      Posted Aug 14, 2009 at 9:18 AM | Permalink | Reply

      Re: Craig Loehle (#64),

      It seems possible to me that the official sending these denial letters has no clue that the data requested differs from the gridded data available at the cru website.

      Having been in a records management type of career position myself as an official custodian of scientific data, engineering data, and also of various kinds of sensitive business data — I can say with some authority that one simply can’t accomplish one’s job without having a basic understanding of the data’s fundamental characteristics, its potential end uses, and its historical provenance. Something else is going on here. But what?

  48. Posted Aug 14, 2009 at 8:54 AM | Permalink | Reply

    This gets better and better. They are not releasing the data because it’s confidential and top secret and available elsewhere.
    The more they refuse and come up with ridiculous self-contradictory excuses, the more publicity this will get and the more stupid they will look. It’s even funnier that they keep referring to ‘public interest’. Sooner or later this will escalate to a higher level when it will be ruled that it is overwhelmingly in the public interest that the data should be released.
    .
    Ford, I feel like digging out the old joke about what planet you are from again. What is the commercial value of the temperature in Timbucktu in April 1967? Lots of temperature data is available. The request is for the exact data on which CRUTEM (and hence the IPCC report) is based.
    Regarding your naive suggestion to just wait for CRU to free up the data, you should read the “A 2002 Request to CRU” thread – they said they would release the data back in 2002!
    What do you not understand?

    • thefordprefect
      Posted Aug 14, 2009 at 9:08 AM | Permalink | Reply

      Re: PaulM (#65), It is commercial data. I.e. someone else OWNS it not CRU. It is not secret.
      If you by a dvd the price you paid does not give you right to redistribute that data to others – simples

      Re: PaulM (#65), I thought the data was “accidentally” placed on an open server in 2003.

      OK then, continue harassing CRU. I give up. You seem totally incapable of understanding what commercial data means.

      • QBeamus
        Posted Aug 17, 2009 at 9:19 AM | Permalink | Reply

        Re: thefordprefect (#70),

        “You keep saying that word… I don’t think it means what you think it means.”

        Seriously, I hear you keep repeating that the data is “commercial,” but I don’t know what you mean by it. DVDs cannot be redistributed because they are copyrighted. Are you suggesting the raw data is copyrighted, or are you simply arguing by bad analogy? (The inside joke among attorneys is that “Legal reasoning is reasoning by bad analogy.”)

        Perhaps more importantly, I have no idea why you think it. Simply repeating it over and over again doesn’t make it true.

        • Mark T
          Posted Aug 17, 2009 at 9:54 AM | Permalink

          Re: QBeamus (#118),

          Are you suggesting the raw data is copyrighted

          Inconceivable.

          Mark

  49. Mark Fawcett
    Posted Aug 14, 2009 at 8:55 AM | Permalink | Reply

    Knowing a lot of government types (I used to be one), or closely associated people – I can safely say that Jones’ behaviour is pretty typical in my experience, and is most likely to be as a result of the following items:

    1. Ivory tower building.
    2. Not-invented-here mindset.
    3. Worrying realisation that a ‘house-inspection’ of said ivory-tower would reveal a somewhat disorganised house; not in keeping with written government standards.
    4. A slight, nagging, doubt that possibly, just possibly, an effective lifetime’s work may have some fundamental flaws or errors.

    Cheers

    Mark.

  50. Posted Aug 14, 2009 at 8:58 AM | Permalink | Reply

    theforcperfect–
    If you think asking Peter Webster for the data and making it available to the world would solve some problem, why don’t you ask Peter Webster for the data and then post the data and maintain it so we can all have the sort of access you feel should be sufficient to our needs.

    Mind you, I don’t think making this request of Peter would solve the problem that CRU refuses data to people who deserve to have access because I think the problem is CRU won’t make the data public. Clearly, Steve doesn’t think getting the data from Peter solves the problem.

    But if you think this is the solution to “the problem” or “some problem”, or if you think your providing this data would make some important point you think needs to be made, I think rather than asking others to get the data from Peter, you should do get the data and make it available to those member of the public who might want it now or in the future. After posting it, you can explain how you have solved the important substantive issue related to data availability.

    To help you, I’ve gone to the trouble of finding Peter’s homepage which includes his email address. You will find it here.

    • thefordprefect
      Posted Aug 14, 2009 at 9:27 AM | Permalink | Reply

      Re: lucia (#67),
      I think rather than asking others to get the data from Peter, you should do get the data and make it available to those member of the public who might want it now or in the future. After posting it, you can explain how you have solved the important substantive issue related to data availability.

      I do not want the data.
      If I got the data, my integrity would prevent me passing it on unless the data sources gave approval.
      I would hope Peter Webster would also refuse the FOI as it is not his data to give.

      I don’t understand why his data is wothless and only the data from CRU is required. Take the FOI and place it at the feet of Webster, I say.

      Re: thefordprefect (#60), McIntres comment
      Perhaps commercial is not valid for all the agreements. But if it is confidential the same inabilty to transfer is implied. It is still not “secret” data.

      • Posted Aug 14, 2009 at 9:36 AM | Permalink | Reply

        Re: thefordprefect (#73),

        I would hope Peter Webster would also refuse the FOI as it is not his data to give.

        I don’t understand why his data is wothless and only the data from CRU is required. Take the FOI and place it at the feet of Webster, I say.

        Why in the world would you suggest SteveM file a request at Georgia Tech if you hope that Georgia Tech would not grant it?

        I’m mystified that you don’t understand why data underlying the CRU product is best obtained from CRU. I am particularly mystified if you think that third parties (like you or Webster) would violate their integrity by passing it on.

        One of the reasons the data for SteveM’s project must come from CRU is that third parties Peter Webster could be accused of lack of integrity for passing the data on.

        • thefordprefect
          Posted Aug 14, 2009 at 10:27 AM | Permalink

          Re: lucia (#74), Exactly – you’ve got it at last:

          One of the reasons the data for SteveM’s project must come from CRU is that third parties Peter Webster could be accused of lack of integrity for passing the data on

          .

          CRU do not own the data. Their integrity would be questioned if they passed it on to third parties. It is not CRUs poblem it is the restrictions of the first parties that is the problem.

          CRU have more than integrity to loose, however. If the data sources did not like what was being done with their data then they could stop providing it.

          To have given megabytes on this on this and all the other leech blogs is unbeleivable. The whole way this has been handled seem to many to be a way of discrediting CRU.
          Mike

        • Steve McIntyre
          Posted Aug 14, 2009 at 10:47 AM | Permalink

          Re: thefordprefect (#78),

          CRU do not own the data. Their integrity would be questioned if they passed it on to third parties.

          Ford, you consistently refuse to answer my responses to your post. It is a matter of record that CRU has passed the data on to third parties, including a US nuclear lab. It is you that believes that transmission to a third party impugns their integrity, not me. I think that they were entitled to do so, precisely because there were no relevant confidential agreements. Is it your position that CRU acted illegally in transmitting data to the Oak Ridge nuclear lab?

          Please answer this before making any other posts.

        • thefordprefect
          Posted Aug 14, 2009 at 2:58 PM | Permalink

          Re: Steve McIntyre (#79),

          Ford, you consistently refuse to answer my responses to your post. It is a matter of record that CRU has passed the data on to third parties, including a US nuclear lab. It is you that believes that transmission to a third party impugns their integrity, not me. I think that they were entitled to do so, precisely because there were no relevant confidential agreements. Is it your position that CRU acted illegally in transmitting data to the Oak Ridge nuclear lab?
          Please answer this before making any other posts.

          The question was so ridiculous I did not realise it was a question.
          I cannot answer for what CRU have or have not done. I do not know what agreements were in place between the parties who received the data – do you?

          Witout full knowledge I cannot answer you question, and you know that.
          I do know that should data be released to 3rd parties Oak ridge, Webster – a co-author, other professional/academic institution then it is unlikely to be passed on. If it is given to a blogger who professes that the data should be released then I would be less sure of this.

        • steven mosher
          Posted Aug 14, 2009 at 3:28 PM | Permalink

          Re: thefordprefect (#99),

          Ford, now YOU are stepping in to set the conditions of all these agreements ( some of them verbal) which either..

          1. Don’t exist
          2. exist but where lost.
          3. were made but not memorialized in writing.
          4. were never made to begin with.

          But just to show your utter lack of understanding I give you this.
          You put this condition on: Co authorship.

          Do you seriously believe that Norway or Spain or WHOEVER gave the data to
          Jones and stipulated that he could only release it to people he was
          “co authoring with” The facts of the case point to this. Jones and CRU are
          using the agreements to control data in a capricious self interested way.
          They are not protecting commercial interests ( hence the positing of this data by mistake) they are protecting their interests. not the public interest, not the interest of other parties.

        • Jonathan
          Posted Oct 23, 2009 at 12:38 PM | Permalink

          Re: Steve McIntyre (#80), UEA has now stated that the release of data to Georgia Tech was an “error” and that they were not entitled to do this. No statement about other releases of course.

        • TAG
          Posted Oct 23, 2009 at 1:07 PM | Permalink

          Re: Jonathan (#136),

          Could somebody do an FOI to determine if this error has occurred before or since the Georgia Tech incident?

        • Cold Lynx
          Posted Aug 14, 2009 at 10:49 AM | Permalink

          Re: thefordprefect (#78),
          Exactly – you’ve got it at last:
          - Their integrity would be questioned if they passed it on to third parties.
          - CRU have more than integrity to loose, however.

          BUT:
          They have passed it to third parties. For Years. But only to “belivers”
          They themselves reduce the value of their own work by not provide the data for their now “scientific”(?) work.

          The only reason why they refuse must be that if they let the data be properly evaluated will they be in a even bigger mess than today.

        • Gerald Machnee
          Posted Aug 14, 2009 at 11:01 AM | Permalink

          Re: thefordprefect (#78),

          Re: lucia (#74), Exactly – you’ve got it at last:

          One of the reasons the data for SteveM’s project must come from CRU is that third parties Peter Webster could be accused of lack of integrity for passing the data on

          First you suggest to Steve that he get the data from Webster and now you agree with Lucia that Webster should not release it. You are doing enough 180′s to spiral underground. Steve has been correct all along. It must come from CRU. In addition he and others have been asking for documentation to show why the data cannot be released. There has not been one legitimate reason for CRU acting the way they did.
          In addition you accuse Steve of having a vendetta. You should really check who has the vendetta. Steve is seeking information.

  51. Calvin Ball
    Posted Aug 14, 2009 at 11:15 AM | Permalink | Reply

    I think this has gotten to the point of mud wrestling with a pig. Steve, Lucia, and everyone else, you can’t win mud wrestling a pig, because the pig enjoys it.

  52. Posted Aug 14, 2009 at 11:43 AM | Permalink | Reply

    thefordprefect–
    Re: thefordprefect (#78),

    Re: lucia (#74), Exactly – you’ve got it at last:

    One of the reasons the data for SteveM’s project must come from CRU is that third parties Peter Webster could be accused of lack of integrity for passing the data on

    But you don’t get it. it’s you and CRU would would accuse Peter of lack of integrity for passing it on. CRU personell might, in consequence add him to the list of people to whom they would not release data. This would have negative consequences for Peter. If Peter wasn’t asked to sign any confidentiality agreements, I wouldn’t think badly of him at all.

    But why should SteveM ask Peter to do something that might cause CRU to think badly of him and possibly retaliate?!
    In anycase, SteveM wants CRU data. He wants to be able to say that he is sure it is CRU data. Logic dictates he should get it from CRU.

    Over time, we will discover which confidentiality agreements actually exist. The inquiries to date seem to be the factor that has motivated CRU/Jones to make the effort to locate the many three bits of paper and to prod Jones to refresh his memory about any verbal ones by contacting various countries. This seems all to the good to me.

    • thefordprefect
      Posted Aug 14, 2009 at 2:46 PM | Permalink | Reply

      Re: lucia (#83), Yet again you confirm what I have been saying all along:

      But why should SteveM ask Peter to do something that might cause CRU to think badly of him and possibly retaliate?!
      In anycase, SteveM wants CRU data. He wants to be able to say that he is sure it is CRU data. Logic dictates he should get it from CRU.

      Why should McIntyre ask CRU to do something that might cause the data sources to withdraw their data from fututre releases in retaliation for breaking verbal/written agreements.

      McIntyre should only get data from the sources if he really wants to check things out Otherwise how does he no that paper records have been correctly digitised? As this could be costly and difficult then the second best is to get the data from the compiler – CRU.

      • jeez
        Posted Aug 14, 2009 at 2:52 PM | Permalink | Reply

        Re: thefordprefect (#97),

        And I’ll say again. Who cares if CRU loses access to data? Their incoherent stonewalling, making up reasons as they go along, demonstrates that they don’t deserve access to the data, let alone any respect for any product which may slide off their sloppy desk onto the web or into policy makers hands.

  53. Corey S.
    Posted Aug 14, 2009 at 11:44 AM | Permalink | Reply

    Regulation 12(5)(f) applies to the data requested because the data was received by the University on terms that limits further transmission. We believe that there would be an adverse effect on the institutions that supplied data under those agreements as it would undermine the conditions under which they supplied the data to the Climate Research Unit.

    In his response to James S. (#18), Mr. Palmer cites the Environmental Information Regulations 2004, which have produced a Code of Practice that he should follow. I find it odd that he mentions agreements that go against dissemination of information, when that is what the Regulations says one who enters into ‘agreements’ should avoid:

    VIII ENVIRONMENTAL INFORMATION REGULATIONS AND PUBLIC SECTOR CONTRACTS

    46. When entering into contracts public authorities should refuse to include contractual terms that purport to restrict the disclosure of environmental information held by the authority and relating to the contract beyond the restrictions permitted by the EIR. Public authorities cannot “contract out” of their obligations under the Regulations. This means that they cannot sign a contract that gives an undertaking to a private firm (or anyone else) that they will not comply with their obligations under the Regulations. Unless an exception provided for under the EIR is applicable in relation to any particular information and the balancing of public interest favours refusal, a public authority will be obliged to disclose that information in response to a request, regardless of the terms of any contract. Where personal data is concerned this will be done in accordance with the requirements of Regulation 13 and the Data Protection Act 1998.

    47. When entering into contracts with non-public authority contractors, public authorities may be under pressure to accept confidentiality clauses so that information relating to the terms of the contract, its value and performance will be exempt from disclosure. Public authorities should reject such clauses wherever possible and explain the relevance of the public interest test. Where, exceptionally, it is necessary to include non-disclosure provisions in a contract, an option could be to agree with the contractor a schedule of the contract that clearly identifies information that should not be disclosed. But authorities will need to take care when drawing up any such schedule, and be aware that any restrictions on disclosure provided for could potentially be overridden by their obligations under the EIR, as described above.

    48. In any event, public authorities should not agree to hold information ‘in confidence’ which is not in fact confidential in nature. Authorities should be aware that certain exceptions including those for commercial confidentiality, and voluntarily supplied data, are not available when the information requested is about emissions into the environment.

    Mr. Palmer was also suppose to include this, from the 2004 Regulations:

    (5) The refusal shall inform the applicant -

    (a) that he may make representations to the public authority under regulation 11; and

    (b) of the enforcement and appeal provisions of the Act applied by regulation 18.

    in his response. Here is the section on exceptions. All links from here.

  54. AnonyMoose
    Posted Aug 14, 2009 at 12:14 PM | Permalink | Reply

    No, getting a copy of the data does not solve the problem. One copy of the data is only useful for one audit. The data and its description should be available from the keeper of the data, to avoid misinterpretation and so the ongoing data is available for research. Should the new data be available in 22 years for reexamination?

  55. Terry
    Posted Aug 14, 2009 at 12:28 PM | Permalink | Reply

    Has anyone yet asked (or submitted an FOI request I suppose) for the policies and procedures governing how they enter into confidentially agreements? (ie who can approve/sign, agreement retention policies, etc).

    Also, (Steve, feel free to snip, editorializing) thefordprefect’s logic reminds me of the rubber band on an old balsa wood wind up airplane after 100 turns of the prop – twisted to the point that it’s ready violently fail just a few turns from now…

    Steve: Steve Mosher made such a request. It was refused, though there is an opening for a re-specification of the request.

  56. jeez
    Posted Aug 14, 2009 at 12:38 PM | Permalink | Reply

    This was my letter:

    Pursuant to the Environmental Information Regulations, I hereby make an EIR/FOI request for any confidentiality agreements covering CRUTEM station data involving station data in Mauritius, Mexico, Morocco, Mozambique, and Myanmar.

    1. the date of such agreement;

    2. the parties to the agreement;

    3. a copy of that part of the agreement that prevents further distribution of the data to non-academics,

    4. a copy of the entire agreement,

    5. a copy of policies and procedures covering employee responsibilities entering into confidentiality agreements,

    6. a copy of policies and procedures covering employee responsibilities regarding the preservation of written agreements,

    7. a copy of policies and procedures regarding employees entering into verbal agreements, and

    8. a copy of instructions to staff regarding compliance with FOI requests.

    I am requesting this information for academic research purposes.

    Thank you for your attention,

    Here is the response to which I have yet to respond:

    FREEDOM OF INFORMATION ACT 2000 – INFORMATION REQUEST
    (Our Ref: FOI_09-72)
    Further to your request for information received on 27 July 2009, I have consulted relevant units within the University and, pursuant to my obligations under section 16 of the Act to provide advice and guidance, I am writing to request clarification of several aspects of your request. Apologies for the delay in responding to you on this matter, but as you may know, we have received a large number of requests for information under the Act recently and it is taking some time to deal with each request.
    In your request, you have asked for a copy of policies and procedures regarding employee responsibilities regarding entering into confidentiality agreements and verbal agreements, and for a copy of policies and procedures regarding employee responsibilities regarding the preservation of written agreements.
    The University does not have one, overarching policy or procedure regarding entering into confidentiality or verbal agreements. Each division within the University has policies and procedures specific to their area of work . This also applies to the preservation of written agreements. In order to answer your question for all of the University, it is highly likely that we would exceed the statutory appropriate limit of 18 person hours to locate, retrieve & review the requested information.
    In order to avoid this situation, I would therefore ask you to clarify what aspect of the University’s work would be the focus of your request for such policies and procedures. For example, given the nature of the other components of your request, are you simply interested in policies in relation to research activities?
    Please note that the statutory timescale of 20 working days as defined by the Freedom of Information Act 2000 will be ‘suspended’ until such time as we receive clarification of your request. Once that is received, the ‘clock’ will recommence, your request considered, and you will receive the information requested within the statutory timescale, subject to the information not being exempt or containing a reference to a third party. You will be informed of any exemptions or references to third parties.
    I trust this is to your satisfaction and look forward to your reply.
    Yours sincerely
    David Palmer
    Information Policy & Compliance Manager
    University of East Anglia

    • Terry
      Posted Aug 14, 2009 at 1:38 PM | Permalink | Reply

      Re: jeez (#88),

      Cool. Maybe you can narrow that down to Dr. Jones’ department, as it applies to CRU data? Just a thought.

      • steven mosher
        Posted Aug 14, 2009 at 3:21 PM | Permalink | Reply

        Re: Terry (#91),

        Jeez and I got the same letter. We will make sure that we don’t tax them beyond there 18 hour limit. I find it odd that some bras have the same limit

  57. Paul Penrose
    Posted Aug 14, 2009 at 12:44 PM | Permalink | Reply

    Mike,
    Despite your hyperventilating about the data being “commercial” products, or your strawman comparison to copyrighted works like DVDs, these are almost the only claims that CRU is not making for refusing to release the data. All the claims they are making are absolute rubbish, so it’s beyond me how you can continue to support them. If some of the data is confidential they have so far failed to show that, either from a moral or legal perspective.

  58. Posted Aug 14, 2009 at 1:42 PM | Permalink | Reply

    As an interested non scientist,I find such behavior suspicious and stupid.

    I thought science research was supposed to be about learning and sharing what we learn with others,that means publishing the research and providing the data along with it.Going to science seminars,exchanging letters,e-mails and phone calls about ongoing research and so on.

    Here we have this absurd resistance (OVER YEARS TIME) to provide data for long published papers,some as far back at the late 1980′s.I consider that silly and unproductive.

    What is the point of publishing science research and then resist producing the relevant data that was part of that paper? It makes no sense do that.

  59. Jonathan
    Posted Aug 14, 2009 at 2:36 PM | Permalink | Reply

    People may be interested to know that Dave Palmer is now on holiday.

    I am on leave from Friday 14 August 2009 until Monday 7 September inclusive. If you have any queries regarding Freedom of Information Act or Data Protection Act matters, please contact either Bob Heath (r.heath@uea.ac.uk; tel. x3483) or Jane Baker (jane.baker@uea.ac.uk; tel. x3483). I will deal with any other correspondence upon my return to the office on Tuesday 8 September 2009.

  60. thefordprefect
    Posted Aug 14, 2009 at 3:05 PM | Permalink | Reply

    As I cannot stop your hounds attacking CRU (e.g.jeez (#98)) I am retiring from this ridiculous situation. Enjoy you denigration.

    • jeez
      Posted Aug 14, 2009 at 3:09 PM | Permalink | Reply

      Re: thefordprefect (#100),

      Before you leave, any chance you can post a DVD Rip of HGTTG somewhere?

      Or even better, the original radio program?

  61. Posted Aug 14, 2009 at 3:22 PM | Permalink | Reply

    Thank you Steve and everyone continuing to press for freedom of basic climate information, from the prime source which should be making it publicly available and easily accessible, should never have entered into any agreements that might hamper such accessibility, and should be legally obliged to nullify any such agreements. Re: Corey S. (#84), this code of good practice seems exactly relevant, and would seem to have legal teeth.

    The foolish wording makes it look as if Mr Palmer is a greenhorn the CRU are using to buy time and flush out the full range of CA energies, while they rethink what to do in an “unprecedented” climate of “robust” audit. Mark Fawcett #66 feels about correct.

  62. Posted Aug 14, 2009 at 3:26 PM | Permalink | Reply

    Computer says no…

  63. BrianMcL
    Posted Aug 14, 2009 at 3:28 PM | Permalink | Reply

    Ref. Jeez’s point abut CRU loosing access to climate data, now that the various WMO members know the degree of care that the CRU has taken with their data over the years who can blame them for not entrusting it to the CRU any more?

    And fordprefect, that’s not Steve’s fault either. They’ll only have themselves to blame

  64. BrianMcL
    Posted Aug 14, 2009 at 3:40 PM | Permalink | Reply

    Also, it might be a small but the the Norwegian agreement appears to specifically preclude commercial use of the data.

    Presumably there’s no risk that anyone might have tried to charge anyone else for anything directly associated with or derived from their data. That would appear to be most improper.

    Further, that presumably means that we can rule out any kind of commercial reason to withhold the data.

  65. steven mosher
    Posted Aug 14, 2009 at 4:07 PM | Permalink | Reply

    Anybody care to lay odd when fordperfect returns?

    • Michael Smith
      Posted Aug 14, 2009 at 4:21 PM | Permalink | Reply

      Re: steven mosher (#109),

      Anybody care to lay odd when fordperfect returns?

      Based on what he wrote in comment 101….

      As I cannot stop your hounds attacking CRU (e.g.jeez (#98)) I am retiring from this ridiculous situation.

      … he won’t return until Steve rescinds the “Havoc!” order and withdraws the dogs of war.

  66. BrianMcL
    Posted Aug 14, 2009 at 4:59 PM | Permalink | Reply

    Lastly for now, but if the information is freely available elsewhere it’s hard to imagine what use the confidentiality agreement would be.

    I’ve yet to see one that doesn’t exclude information that’s freely available elsewhere as the CRU claim.

    Even if they had these agreements they’re unlikely to be worth the paper they’re not written on anyway.

  67. John Nicol
    Posted Aug 14, 2009 at 11:11 PM | Permalink | Reply

    Lucia,
    Lucia, while I admire your tenacity and potential ability to get hold of the information everyone wants, I think you would agree that your suggestion that someone should obtain the data, if possible, from Peter Webster and then publish it, would be highly unethical and would place Webster in an untenable position himself. If Peter has had the privelige of being supplied with the data, even though we all believe/know that UEA are acting dishonourably and should make efforts to obtain release from agreements and make the data public, Webster is in no moral or possibly legal, position to make it available to a third party. He has just been lucky and one should recognise that, until more information becomes available as to why he was provided with the information, so jealously guarded by Phil Jones, Webster should be left to get on and use it. Perhaps he will publish an article declaring that it is all utter rubbush. In the meantime, anyone needing the data should continue to harangue Phil Jones and associated institutions by every means possible.
    John Nicol

    Steve: Lucia did not make that suggestion, she opposed it. It was fordprefect who said that.

  68. Shallow Climate
    Posted Aug 15, 2009 at 11:35 AM | Permalink | Reply

    Re Sunsettommy, #94:
    Even though you are, in your words, a “nonscientist”, you have it exactly right, and that is really the crux of SM’s post here. In order to qualify as “science”, the data have to be PUBLICLY available (and it does make a bit of difference whether it’s a PhD climate scientist or an unbathed, scatterbrained amateur blog-reader who examines it and uses it). It’s curious and appalling to me that the climate-science Community is not itself the one MOST up-in-arms about data/methods nonposting–don’t they want to uphold the legitimacy of their profession? No one could ever get away with this in physics or chemistry.

  69. antonius
    Posted Aug 16, 2009 at 10:56 AM | Permalink | Reply

    The data is available for “commercial purposes” though “different licensing arrangements and charges will apply”. It doesn’t seem likely that the “scientist rate” will be available to anyone whose intended purpose is to audit the adjusted data available to the public.

    It might be an interesting exercise for someone to apply for a commercial licensing deal to see what kind of value they place on keeping a tight hold on the data.

    I seem to recall a mention that the licensing costs are intended to cover administrative costs. Given that the data obviously already exists in the format that would be requested, the price quoted for the licenses should be minimal. Anything above that cost would seem to be very difficult to justify (though I’m sure they will provide one that is sure to entertain).

  70. Craig Loehle
    Posted Aug 17, 2009 at 2:27 PM | Permalink | Reply

    In academia it is common for a scientist to not want to ever release his data. He may mine it for publications for decades (or not). Some scientists are hoarders. That is why journals & funding agencies need to enforce archiving. Here we have a case where there are only 3 real global historical datasets (NOAA, GISS, and CRU) and none of them will release the data, as if it were the “private” data of an individual academic. Furthermore, since new data come in every month, it never gets old enough that it can be released (like an academic might his old data when he is tired of it).

  71. mondo
    Posted Aug 30, 2009 at 9:22 PM | Permalink | Reply

    Jonathan,

    Since you haven’t exactly been overwhelmed with responses to your post(s), I thought that I should tell you that in my view you have done a truly excellent job of framing your appeal. I look forward to hearing the outcome.

  72. Ivbo
    Posted Sep 1, 2009 at 3:57 PM | Permalink | Reply

    Dear Sirs,

    I am neither a scientist nor an ‘academic’, however I am a tax paying British citizen who has an interest in science and appreciates that critical & open analysis of the data behind a scientists’ results/claims is key to the scientific process (especially where that science has been given such crucial global importance).

    Your collective efforts to obtain this base data for independent analysis and review are to be applauded and, personally, I am very disappointed (although not surprised) at the infuriatingly obstructive attitude of those persons (who’s wages I pay by the way) to release data that I have paid for (either directly or indirectly – no-one does anything for nothing).

    Sadly I’m afraid this is generally indicative of a current UK Government that is more concerned with control/PR/’spin’ than absolutely anything else. So, on behalf of the (silent) majority of the UK populace, I would like to apologise to the rest of the world for our (or at least the CRU’s) failure to release the data that is, ultimately, being used world-wide to support an agenda of control, taxation and restriction – not to mention moral panic and the mis-allocation of resources & redirection of priorities.

    Unfortunately I have no practical suggestions on this particular case beyond keep trying & keep building the pressure to release this raw data so that we (i.e. the non-scientist, non-academic, tax paying lay-people of the world) can be reassured (or otherwise) that the fundamental data behind the science behind AGW behind the policies is really sound (or not).

    Good look in your quest.

    Regards

    Ivbo

  73. curious
    Posted Sep 23, 2009 at 6:32 PM | Permalink | Reply

    Hi Jonathan -

    Can I suggest you read this?:

    http://www.ico.gov.uk/upload/documents/library/freedom_of_information/practical_application/ico_foi_eir_how_to_comp_final.pdf

    I’m not sure the UEA are doing this correctly. Your excellent request and appeal was made under the EIR regs and the link they have provided is to a 2004 Code of Practice relevant to FoIA. These are different things. I do not see where the EIR allow for “an initial informal attempt at resolution” as an internal process followed by another 28 days to have a “formal” attempt. My understanding is your appeal was a formal request under EIR and there has been no “informal” element. Is this correct or have you had discussions with them seeking solutions? If not I think you may be in a position to complain now as effectively this is an unauthorised extension to the process with no good reason.

    In my opinion to simply “feel that the distance that exists between your position and that of the University” prevents supplying the information is not enough – explicit reasons need to be supplied, otherwise this is simply prevarication. I think it is worth seeking the opinion of the ICO.

    • Jonathan
      Posted Sep 24, 2009 at 2:41 AM | Permalink | Reply

      Re: curious (#127), thanks for the helpful link. However I don’t see anything there which clearly defines what format the Complaints Procedure should take, simply that one should exist and that it should be followed; there is an implicit (but not explicit) requirement that the procedure should be reasonable. Appendix H of the UEA code of practice at http://www.uea.ac.uk/polopoly_fs/1.2750!uea_manual_draft_04b.pdf sets out their procedure fairly clearly and they seem to be following it. Currently I am in stage 2 of their 3 stage process, and I don’t propose to take any action beyond letting it run. If anyone has experience of these stages at UEA it would be good to hear from them!

      Of course if you think the approach should be run differently you are free to make your own request and pursue it as you think best.

  74. curious
    Posted Oct 19, 2009 at 4:22 PM | Permalink | Reply

    Hi Jonathan – any news? I hope you aren’t going to have to admit you’ve had the data for a year now?! :)

    As per our Code of Practice, Mr. Colam-French will respond to you within 28 calendar days of referral of this matter, effectively, by 20 October 2008.

  75. Jonathan
    Posted Oct 21, 2009 at 3:26 PM | Permalink | Reply

    Well spotted. I took the 2008 date as a typo, similar to the signoff “I apologise that your request will be met”. But yes, the 28 calendar days have passed, and no I have had no response.

    I will enquire again fairly soon, but haven’t decided exactly when to do so yet.

    • Jonathan
      Posted Oct 22, 2009 at 3:46 AM | Permalink | Reply

      Re: Jonathan (#130), I have now emailed as follows

      Dear Mr Palmer,

      Further to your email dated 22nd September, included below, and the letter attached to it, 2nd_level_referral_090922.doc, I had been expecting a response from Mr Colam-French by last Tuesday, 20th October.

      I have yet to hear from him, and would be grateful for your suggestions as to how I should proceed at this stage.

      We shall see what transpires. Incidentally I have just discovered that Dave Palmer may well be Canadian! His first two degrees (BA MLS) are from British Columbia.

      • Jonathan
        Posted Oct 22, 2009 at 11:31 AM | Permalink | Reply

        Re: Jonathan (#131), Dave has replied

        Just a quick note to inform you that I understand that Mr. Colam-French’s response should be out to you later today, tomorrow at the latest Apologies for the delay in responding.

        I have made the obvious reply

        Thanks for the update. I note, however, that UEA is now in formal breach of its own code of practice, appendix H paragraph 10.

        • Jonathan
          Posted Oct 23, 2009 at 5:29 AM | Permalink

          Re: Jonathan (#132), they have refused my appeal, but have made some interesting admissions.

          ENVIRONMENTAL INFORMATION REGULATIONS 2004 – INFORMATION
          REQUEST (Our ref: FOI_09-117; EIR_09-14)

          Pursuant to Mr. Palmer’s letter of 22 August 2009 to you regarding the handling of
          your appeal of 27 August to our response of 14 August, to your FOI request of 24
          July 2009, I have undertaken a review of the contents of our file and have spoken
          with Mr. Palmer and other relevant staff involved in this matter.

          As a result of this investigation, I am satisfied that our overall decision to not disclose
          the requested information is correct.

          In response to your first point regarding the existing availability of the requested data,
          whilst it is true that the requested data is station data and not gridded data, we still
          maintain our position that this information is already available from the Global
          Historical Climatology Network (GHCN )1.

          1 http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php

          In regards our exceptions under Reg. 12(5)(a) & (f), we would maintain the position
          taken to date. Whilst you make a number of arguments relevant to an exemption
          under s.41 of the Freedom of Information Act, the EIR Regulation sections cited in
          this matter are different and the tests for exception are different. The EIR does not
          mention confidentiality nor is there any equivalent test under EIR to that imposed by
          section 41 of the FOIA. It is the terms under which we secured the data that restrict
          our further transmission of the data which are key, not necessarily the inherent
          confidentiality of the data itself.

          Regardless of any guidelines from WMO or any other organisation, given that there
          are restrictions on at least some of the data cited that are not in any way effected or
          superceded by any such guidelines, our opinion is that releasing the data would be
          contrary to the agreements we have made with other organisations and the effect on
          international relations and on those organisations would be adversely effected.

          In considering the exception under Reg. 12(5)(a), it is still our position that disclosure
          might compromise future co-operation with the UK or undermine the relationship
          between UK and other countries or international organisations. Release of
          information contrary to the wishes of the organisation that give us that information
          would likely result in a reluctance of any such organisation to provide further
          information to, or cooperate with, not only CRU, but any UK-based institution subject
          to the EIR where such information could be subject to release.

          In regards the Reg. 12(5)(f) exception, DEFRA guidance notes that the Aarhus
          Convention, which contains the origins of the Directive on which the EIRs are based,
          protects information volunteered by a third party and requires their consent to
          disclose it. The purpose of the exception is to encourage the free flow of information
          from private persons or institutions in order to protect the environment where making
          it available to the public could inhibit that process. To provide information that has a
          restriction on further transmission on it would not only damage CRU’s ability to
          secure such information in future, but would also harm the interests of the
          organisations providing the information, who clearly have an interest in restricting
          transmission of the information due to the very existence of the restrictions.

          We do concede that information was provided to Georgia Tech without securing
          consent of the institutions that provided it, and, upon reflection, this is an action we
          would not choose to take again. However, having made one error does not, in our
          eyes, justify making the same error again.

          In regards your point regarding your status as an academic and any perceived effect
          on our decision in this matter, I can assure you that at no time have we based any of
          our decisions on your status as an academic or not.

          I note your reference to Regulation 12(11) requesting such data as can be provided
          that is not subject to any restriction on further transmission. The data is organised in
          such a way as to make it extremely difficult and time-consuming to segregate the
          data in the manner that you suggest and would indeed, in our view, amount to an
          unreasonable diversion of resources from the provision of services for which we, as
          an institution, are mandated. Further, we would maintain that where no such
          segregation has, or will occur, we should not release any of the data for fear of
          breaching such restrictions as do exist.

          I would note that we are, however, proceeding with efforts with the international
          community to secure consent from national meteorological institutions for the release
          of the information that they provide us with, and it is fully our intention to publish such
          data where, and when, we have secured such consent. This is in line with guidance
          from DEFRA that suppliers of volunteered information should be encouraged to
          consent to release where appropriate, and where it is lacking, such consent can be
          sought later in response to a particular request or in order to proactively disseminate
          the information.

          In regards our obligation to assess the public interest in applying these exceptions, I
          am of the opinion that the public interest balance is in favour of non-disclosure of the
          requested information. As noted above, the public interest in maintaining the flow of
          information from institutions to CRU, and maintaining good working relations with
          international organisations, outweighs, in this case, the interest in the release of the
          data.

          We have contacted the Information Commissioners Office in regard to this matter
          and their advice is that if you are still dissatisfied with this response, you should, at
          this time, exercise your right of appeal to the Information Commissioner.

          Yours sincerely

          Jonathan Colam-French

          Director of Information Services

          Any thoughts appreciated, but I will have to mull on this for a while.

        • Punch My Ticket
          Posted Oct 23, 2009 at 1:31 PM | Permalink

          Re: Jonathan (#133),

          IOW, we have dumped everything into one vat, some of it is poison (but we cannot or will not demonstrate that), so the whole vat is poisoned and letting anyone have a taste of any kind would endanger public health.

          You’re snookered.

          Maybe shaming them again in the papers will work but I doubt it. In the end someone will have to spend some money and ask a court to order an end to the runaround. If the only “terms under which we secured the data” they can come up with are what’s been published already, a judge would see through the farce pretty fast.

          Of course, there is another alternative, not cheap nor easy nor quick. Get equivalent data from open sources, QC it to hell and gone, and publish the result. That would put CRU out of business which would be an appropriate penalty.

  76. bender
    Posted Oct 23, 2009 at 6:03 AM | Permalink | Reply

    I am of the opinion that the public interest balance is in favour of non-disclosure of the requested information.

    well you’re fired

  77. MrPete
    Posted Oct 23, 2009 at 6:50 AM | Permalink | Reply

    Essentially, CRU is a data sink that believes its work requires no V&V.

  78. Tony Hansen
    Posted Oct 23, 2009 at 4:41 PM | Permalink | Reply

    The public interest shall always be our first priority.
    Except in those situations where the public interest is not in perfect alignment with our own.

  79. curious
    Posted Oct 24, 2009 at 10:14 AM | Permalink | Reply

    Jonathan – thanks for the update. IMO shameful response that (further) destroys the credibility of the UEA and shows a disregard for the intended purpose of FOI and EIR legislation.

    • Jonathan
      Posted Oct 24, 2009 at 3:01 PM | Permalink | Reply

      Re: curious (#140), I will of course be appealing this all the way. However Mr Colam-French is distinctly cleverer than the somewhat hapless Dave Palmer, and I’m going to need some help with this. For newcomers please note that winning an FOI case is essentially a technical matter, which must rely on identifying technical breaches by UEA and not on appeals to science or common decency or things of that kind.

      Readers may have noted that although UEA has conceded some minor points the essence of their claim remains the same: their claim that the data is simultaneously publicly available and confidential. My understanding is that although a tribunal is likely to be unsympathetic to this approach I will nonetheless have to defeat both claims.

      Claim 1: the information is available elsewhere
      UEA has accepted that the gridded data that CRU publish is not equivalent to the data I requested, but are still claiming that it is available from GHCN. Previously Steve McIntyre (#64) has claimed that the two data sets are materially different, and any details of this would be extremely helpful. I will of course be making the various obvious points about this claim being in contradiction to their other statements, but clearly identifying an actual difference would help.

      Claim 2: the data is held subject to non-disclosure agreements
      I will of course be making the obvious points about the applicability of non-disclosure agreements to data which is allegedly publicly available, but I will mostly be using this to attack claim 1, and my principal attack on claim 2 will be based on their inability to produce any such agreements.

      Their decision to declare the earlier release of the data to Peter Webster as improper is clever, as I think it would be very difficult for them to justify releasing data to him but not to me. However it is not without its own weaknesses. Steve McIntyre (#80) has claimed that the data has also been released to Oak Ridge, and more details of this would be helpful, as would details of any other transfers.

  80. See - owe to Rich
    Posted Oct 24, 2009 at 2:54 PM | Permalink | Reply

    I think the CRU refusal was very well worded and polite. The weakest part of their argument is the effort (which they are apparently not paid to do) to sort out which sources are protected and which are not. It cannot really be that difficult.

    Anyway, Steve, have you had enough fun yet with trying to winkle out the data from CRU? You appear to have had access to most of the data back in late July – did you really not file away a copy for a rainy day such as this? Or is there some long game that I don’t understand?

    Rich.

  81. curious
    Posted Oct 24, 2009 at 4:51 PM | Permalink | Reply

    CO2 Rich – “well worded and polite” is often the format used as a facade for an inadequate response constructed to frustrate due process – just to clarify your position: do you think that the CRU raw data set should be publicly available? I agree it is weak to claim an unreasonable effort is required to determine the “confidential” elements (in a properly organised set up this would simply be a case of opening the “file” and sorting by tag “confidential”) but, IMO, the fundamental weakness is their argument that it is not in the public interest that this raw data should be available in case it should prejudice the flow of future information. If scrutiny at this stage could improve the quality of the future flow of information, as well as the current information, I would think that would be very much in the public interest whereas a flow of unverified and potentially incorrect information is not.

    Jonathan – IMO you have grounds for appeal as they failed to meet their response deadlines. However, though noteworthy, an appeal upheld on this basis is unlikely to progress release of the data. I am not sure what to make of this statement:

    We have contacted the Information Commissioners Office in regard to this matter
    and their advice is that if you are still dissatisfied with this response, you should, at
    this time, exercise your right of appeal to the Information Commissioner.

    Quite why the UEA are contacting the IC on your behalf escapes me but perhaps I’ve misunderstood. They may have included this with the intent to advise that you have a right to appeal to the IC; yet that was already known. I think it is more likely they are saying there is no point in any further correspondence with them.

    As noted above to CO2 Rich I think the fundamental issue is that it is in the public interest the data should be available and, on my reading, this is the primary consideration in the EIRs. I also think the claim that the resource required to identify the confidential elements is “unreasonable”:

    I note your reference to Regulation 12(11) requesting such data as can be provided
    that is not subject to any restriction on further transmission. The data is organised in
    such a way as to make it extremely difficult and time-consuming to segregate the
    data in the manner that you suggest and would indeed, in our view, amount to an
    unreasonable diversion of resources from the provision of services for which we, as
    an institution, are mandated. Further, we would maintain that where no such
    segregation has, or will occur, we should not release any of the data for fear of
    breaching such restrictions as do exist.

    should be challenged. If they are mandated to provide “information services” then having proper data management and quality control procedures is entirely reasonable and IMO, in fact, required. As a supporting argument to quantify the likely resource required, I think the work Steven Mosher and others did on FOI requests for the Confidentiality Agreements applicable to the dataset is relevant. I didn’t follow it blow by blow but I formed the impression the “confidentiality” requirement was only applicable to a handful of data providers. If this is the case then identifying and separating the relevant stations should be straightforwards and I’d estimate a reasonably competent person starting from scratch with the relevant station IDs could do this in a half to a full day’s effort? (I may be off beam here – other views appreciated).

    I agree with your analysis of the other two issues and I’ll refresh my memory on the regs. and let you have any additional thoughts. I think the Georgia tech transmission is of little relevance other than to illustrate the (admitted) laxity of their data management. If you have the resource I think it would be well worth getting expert opinion on this – my reading of the EIR regs. is only as a lay person. Possible starting point for relevant expertise here:

    http://www.pdpjournals.com/freedom_of_information/editorial_board/

    • Jonathan
      Posted Oct 25, 2009 at 2:55 AM | Permalink | Reply

      Re: curious (#143),

      Quite why the UEA are contacting the IC on your behalf escapes me but perhaps I’ve misunderstood. They may have included this with the intent to advise that you have a right to appeal to the IC; yet that was already known. I think it is more likely they are saying there is no point in any further correspondence with them.

      I think this is fairly straightforward: their complaints procedure is a three-step one, but they are aborting it at the end of stage two, and I presume they have obtained an opinion that this not in itself a breach of their procedure.

      • Jonathan
        Posted Nov 24, 2009 at 4:14 PM | Permalink | Reply

        Re: Jonathan (#144), I have finally sent my appeal to the Information Commissioner. Their web form is moderately helpful, but doesn’t like URLs or ampersands. Anyway my appeal is as follows:

        I wish to appeal the refusal of my FOI/EIR request to the University of East Anglia for “a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009″, their reference FOI_09-117; EIR_09-14. My appeal through the UEA internal process has been refused (final refusal on 23rd October 2009). Copies of all major items of correspondence are attached. I apologise for the delay in this appeal, which was due to illness.

        SUMMARY OF PROCESS

        My request (JAJFOI-1-request) on 24th July 2009 was initially rejected on 14th August 2009 (JAJFOI-3-request-refusal). The grounds for rejection were (my numbering and summary):

        1a) Information is available elsewhere: “the requested data is a subset of data already available from other sources; namely the Global Historical Climatology Network (GHCN) and the Climatic Research Unit already makes requested information available on it’s website in a gridded format”.

        1b) Excessive effort required: “We believe, following DEFRA guidance, that it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available.”

        2a) Release would damage relations with scientists and institutions from other nations: “If this information were to be released contrary to the conditions under which this institution received it, it would damage the trust that other national scientists and institutions have in UK-based public sector organisations”

        2b) Information is covered by a confidentiality agreement: “the data was received by the University on terms that limits further transmission”

        I rebutted these four claims in my appeal (JAJFOI-4-appeal). The response from UEA was delayed (JAJFOI-5-appeal-ack, JAJFOI-6-appeal-referral, JAJFOI-7-appeal-reminder, JAJFOI-8-appeal-reminder-ack) and the eventual reply (JAJFOI-9-appeal-refusal) rejected my appeal, citing modified versions of claims 1a, 2a and 2b (claim 1b was not formally conceded by UEA, but they did not contest my rebuttal). UEA also refused my request under regulation 12(11) for access to parts of the data.

        I am therefore appealing to the Information Comissioner.

        REBUTTAL OF CLAIMS BY UEA

        1a) Information is available elsewhere

        In their original reply UEA claimed that “the requested data is a subset of data already available from other sources; namely the Global Historical Climatology Network (GHCN) and the Climatic Research Unit already makes requested information available on it’s website in a gridded format”.

        UEA has accepted my argument that the gridded data made available on the CRU website. However they continue to claim that the data I requested is available at the GHCN. This is not correct: while the GHCN database is similar to the data I requested it is not identical. This is stated by UEA themselves in a recent statement on their website at http://www.uea.ac.uk/mac/comm/media/press/2009/nov/homepagenews/CRUupdate “Climatic Research Unit update – November 24, 3.30pm”, attached as JAJFOI-A-CRU-statement, which says “Much of these data are already available from the websites of the Global Historical Climate Data Network and the Goddard Institute for Space Science”: the word “much” rather than “all” is critical in this context.

        1b) Excessive effort required

        UEA appears to have accepted my point that I requested “a copy of a data set which has already been prepared by the university, and so is already available” rather than “information in a different format to that which is already available”. I believe this point is no longer at issue between us.

        2a) Release would damage relations with scientists and institutions from other nations
        2b) Information is covered by a confidentiality agreement

        As in my appeal I will take these two points together. Fundamentally UEA’s original claim was that the data I requested was covered by confidentiality agreements, and they provided a link to a supposed list of such agreements at http://www.cru.uea.ac.uk/cru/data/availability/. As described in my appeal document, none of the documents available at this website are confidentiality agreements, and none of them create an actionable breach of confidence within the meaning of section 41. I further noted the perversity of UEA’s claim that the data I requested was simultaneously “publicly available” and “confidential”.

        In his response to my appeal Mr Colam-French accepted that the data was not confidential, but claimed that it was nevertheless acquired under terms which prevented its further transmission. No additional evidence of such terms was provided and I can only assume that he is referring to supposed agreements available at http://www.cru.uea.ac.uk/cru/data/availability/. However none of the documents available at this website assert clear restrictions on further distribution.

        My appeal noted that the data I requested had already been transmitted to a third party, namely Peter Webster of Georgia Tech, which is inconsistent with their claims on restrictions on further distribution. Mr Colam-French now claims that this transmission was in error. This is not, however, the only such “error” on their part: in UEA’s reply to a previous FOI request to Steve McIntyre, available online at camirror.wordpress.com/2009/11/21/test/ , UEA accepts that earlier versions of this data set were transferred to the US Department of Energy and to Steve McIntyre himself, and were made available on a public ftp site. It is extremely difficult to square UEA’s sudden concern about distribution rights with its clearly established pattern of behaviour.

        GENERAL COMMENTS ON APPROACH OF UEA

        As noted above I do not believe that UEA has countered my rebuttals of their claims in their original refusal notice, and so I wish to appeal their refusal of my appeal. Beyond this, however, I have three particular concerns about the manner in which UEA has handled this case.

        1) UEA did not respond to my appeal within their self-imposed time scale. I belive this shows a lack of seriousness in their approach.

        2) UEA’s approach has been characterised throughout by a reliance on two mutually contradictory claims about the data (minor variations on the claims that it is publicly available and that it is simultaneously confidential or restricted). I believe this shows a lack of good faith in their approach.

        3) Recent emails and documents obtained from within the CRU appear to show a plan to subvert the Freedom of Information Act (while UEA has not formally confirmed the accuracy of these emails, they have not denied this either). I draw particular attention to the email apparently sent by Phil Jones of the CRU which is attached (JAJFOI-B-CRU-email) and which includes the paragraph:

        “When the FOI requests began here, the FOI person said we had to abide by the requests. It took a couple of half hour sessions – one at a screen, to convince them otherwise showing them what CA was all about. Once they became aware of the types of people we were dealing with, everyone at UEA (in the registry and in the Environmental Sciences school – the head of school and a few others) became very supportive. I’ve got to know the FOI person quite well and the Chief Librarian – who deals with appeals. The VC is also aware of what is going on – at least for one of the requests, but probably doesn’t know the number we’re dealing with. We are in double figures.”

        It is hard to interpret this as indicating a willingness by the CRU and UEA to conform with the principles of the Freedom of Information act.

        SUMMARY OF DOCUMENTS ATTACHED

        JAJFOI-0-summary: This document
        JAJFOI-1-request: My original request
        JAJFOI-2-request-ack: Acknowledgment sent by Dave Palmer, UEA
        JAJFOI-3-request-refusal: Refusal sent by Dave Palmer
        JAJFOI-4-appeal: My appeal against this refusal
        JAJFOI-5-apeal-ack: Acknowledgment sent by Dave Palmer
        JAJFOI-6-appeal-referral: Referral from Dave Palmer to Jonathan Colam-French
        JAJFOI-7-appeal-reminder: My request for a reply to my appeal, which was now late
        JAJFOI-8-appeal-reminder-ack: Acknowledgment sent by Dave Palmer
        JAJFOI-9-appeal-refusal: Refusal sent by Jonathan Colam-French
        JAJFOI-A-CRU-statement: webpage statement bu CRU/UEA
        JAJFOI-B-CRU-email: email ascribed to Phil Jones, CRU/UEA

        We’ll see how it goes!

        • Jonathan
          Posted Jan 28, 2010 at 3:20 PM | Permalink

          Re: Jonathan (Nov 24 16:14), in the light of recent developments at UEA I thought it might be interesting to reopen discussions with Mr Colam-French:

          Dear Mr Colam-French,

          (CC: Dave Palmer)

          Thank you for your email of 23 October 2009. As you may already be aware I have appealed this refusal to the Information Commissioner.

          I understand that UEA has now accepted the Information Commissioner’s decision that it has in the past mishandled some FOI requests referring to the CRU, and in these circumstances it seems only right to give you an opportunity to reconsider your decisions with regard to my request in the light of this new guidance.

          I look forward to hearing from you.

          Regards,

          Jonathan

          We’ll see what happens!

    • See - owe to Rich
      Posted Oct 25, 2009 at 2:14 PM | Permalink | Reply

      Re: curious (#143), you asked me to clarify. Well, I’m pretty much in agreement with you. CRU must sort out its confidentiality agreements, and since they clearly don’t think the data should in general be confidential ;-) they should ask each supplier to waive such. If any one supplier is a defence establishment with whom there would be great surprise/concern that there was a liaison, then they might be able to claim “national security” for that one. (What, we receive “hot” climate data from the Taleban?!)

      Regarding their approach to the information commissioner, we have seen one explanation. Another might be that they want to get their word in first, and warn him what might be coming his way…

      Rich.

      • Jonathan
        Posted Oct 25, 2009 at 3:14 PM | Permalink | Reply

        Re: See – owe to Rich (#146), we already know about CRU’s supoosed confidentiality agreements, as a full list can be found at http://www.cru.uea.ac.uk/cru/data/availability/ and none of them are particularly impressive.

        The switch from “confidentiality” to non-disclosure agreements is superficially clever, as it lets them move from a completely indefensible position, but since they don’t actually have any NDAs it doesn’t gain them much.

  82. henry
    Posted Oct 25, 2009 at 12:03 PM | Permalink | Reply

    It is the terms under which we secured the data that restrict our further transmission of the data which are key, not necessarily the inherent confidentiality of the data itself.

    1. Has anyone gotten a copy of their “terms”?

    2. If the earlier release of the data to Peter Webster was improper, what have they done to get the information back? IIRC, CRU asked Steve to delete his copy of the information, so have they insisted that Webster delete his?

    The fact that they allow Webster to hold on to this “confidential” data says that they’re not TOO concerned about whether this data should be released. They’ve proven thay can’t be trusted to keep the data secret.

    The fact that Webster may actually produce a paper using this secret data makes the future paper un-reviewable (any peer review couldn’t ask to see the data, any journal couldn’t force data to be archived, etc.)

  83. curious
    Posted Nov 24, 2009 at 6:24 PM | Permalink | Reply

    Comprehensive and convincing. Good luck – we’ll see now if FOI and EIR really are there to serve the “public interest”. C

    • Jonathan
      Posted Dec 10, 2009 at 9:04 AM | Permalink | Reply

      As I never got an acknowledgement of my appeal I gave them a call, and they said they had no record of it. (I suspect incompetence rather than foul play). I have, therefore, sent them a paper copy.

      • Jonathan
        Posted Dec 18, 2009 at 6:45 AM | Permalink | Reply

        I have now received acknowledgments of both copies of my appeal (paper and web) and an apology that their web based system had problems.

  84. Jonathan
    Posted Dec 18, 2009 at 6:51 AM | Permalink | Reply

    I sent in a second FOI request last month

    Pursuant to the Freedom of Information Act, I hereby request a copy of (1) any emails sent by, sent to, or copied to Phil Jones, David Palmer, Jonathan Colam-French, or any other member of the CRU or any other member of the UEA Information Services Directorate during the last ten years concerning FOI requests involving the CRU or any of its memebers, and (2) any documents prepared by or held by Phil Jones, David Palmer, Jonathan Colam-French, or any other member of the CRU or any other member of the UEA Information Services Directorate during the last ten years concerning FOI requests involving the CRU or any of its members.

    My request includes, but is not limited to, any emails or documents containing any of the following phrases:

    (a) As for FOIA Sarah isn’t technically employed by UEA
    (b) Think I’ve managed to persuade UEA to ignore all further FOIA
    (c) I do now wish I’d never sent them the data after their FOIA
    (d) Have agreed to under the FOIA here in the UK
    (e) number of Freedom of Information (FOI) requests for CRU data
    (f) CA are now to send out FOIA requests for the Review Editor comments
    (g) our FOI about the letters Review Editors sent when signing off
    (h) this is the person who is putting in FOI requests for all emails Keith and Tim
    (i) These follow-up questions appear directed more towards Keith than to me
    (j) Can you delete any emails you may have had with Keith
    (k) I have been of the opinion right from the start of these FOI requests
    (l) you to submit FOI requests to UK public institutions
    (m) The FOI line we’re all using is this. IPCC is exempt from any countries FOI
    (n) Not too much they can request via FOI, but they will think of something
    (o) When the FOI requests began here, the FOI person said we had to abide
    (p) CRU has had numerous FOI requests since the
    (q) the 50 or so FOI/EIR requests we’ve had over the weekend
    (r) Even though I’ve had loads of FOIs and nasty emails
    (s) is a Freedom of Information Act now in the UK, I think I’ll delete the file
    (t) Don’t any of you three tell anybody that the UK has a Freedom of Information Act
    (u) Our university has received a request, under the UK Freedom of Information
    (v) Send them the raw data as is, by reconstructing it from GHCN

    I didn’t expect much joy from this, but was interested to see what their grounds for refusal would be. As expected I have just been refused, with an interesting range of grounds:

    FREEDOM OF INFORMATION ACT 2000 – INFORMATION REQUEST
    (Our Ref: FOI_09-160)

    Your request for information received on 23 November 2009 for any emails sent by, to, or copied to Phil Jones, David Palmer, Jonathan Colam-French, or any other member of the CRU or any other member of the UEA Information Services Directorate during the last ten years concerning FOI requests involving the CRU or any of its members, and any documents prepared by or held by the same persons for the same time period in regards the same subject has now been considered and unfortunately, it is not possible to provide the information.

    In accordance with section 17 of the Freedom of Information Act 2000 this letter acts as a Refusal Notice, and I am not obliged to supply this information. The exemptions are clearly indicated within the attached document and the reasons for exemption are as stated below:

    Exemption Reason

    s.12(1), Appropriate limit exceeded The time it would take to extract the information would exceed the appropriate limit

    s.31(1)(a) & (b), Disclosure would prejudice detection of crime & the apprehension of criminals Information is held by the police in connection with an ongoing criminal investigation

    s.36(2)(b)(i) & (ii) – Prejudice to the conduct of public affairs Release of the requested information, would inhibit provision of advice, and the free & frank exchange of view for deliberation

    s.40(1), Personal information Release of personal information would contravene the data protection principles

    Section 12 applies because, given the extent of information requested and the number of persons potentially involved, extraction of data from all the locations where it might be held would exceed the statutory limit as provided for by section 12 and the Fees Regulations.

    Even if we could assemble the requested information within the appropriate limit, it is our belief that s.31(a) and (b) applies because the requested information is part of an ongoing investigation into criminal activity being carried out by Norfolk Constabulary. Disclosure of any information relevant to that investigation at this point could or would prejudice the ongoing investigation of this matter.

    There is a clear public interest in preserving the integrity of, and public confidence in, criminal investigations. As this matter is currently under investigation, the public interest in non-disclosure is even higher than it would be otherwise.

    Regardless of the fact that some of the information is likely to be already in the public domain due to the illegal penetration and use of University computing facilities, this fact does not relieve us of our obligations to address any request on its merits under the Act.

    As to section 36(2), in the opinion of our ‘qualified person’ as defined by the Act, the Vice-Chancellor of this University, disclosure of this information would be likely to inhibit ‘the free and frank exchange of views for the purpose of deliberation’. The ability to speak freely and frankly about the administration of a request under the Act, is essential to ensure that we meet our obligations under the Act and address the legitimate interests of the University. All options need to be discussed openly and advice offered to staff involved in the process, and the disclosure of such information would certainly inhibit our ability or willingness to engage in such discussions in future.

    As required by the Act, we state in every response the reason for our decisions, and the thinking behind those decisions. These decisions, and the reasoning behind them are open to challenge by the requester and can ultimately be adjudicated by the Information Commissioner and Information Tribunal.

    There is a very strong public interest in preserving the ability of an institution to discuss FOIA requests openly and freely in order to ensure that all options are addressed, assessed and either chosen or rejected. As noted above, the outcome of this process is openly to challenge and adjudication which provides the transparency and accountability required under the Act. Therefore, that the public interest in the non-disclosure of this internal correspondence outweighs the public interest in disclosure.

    We invoke section 40 because it is our belief based on internal consultation, that personal information is within the requested information, and we do not have consent for the release of that information. Therefore, disclosure without that consent would contravene the first data protection principle and is therefore barred by section 40.

    I apologise that your request will not be met but if you have any further information needs in the future, then please contact me.

    • curious
      Posted Dec 18, 2009 at 7:13 AM | Permalink | Reply

      Thanks for the update. Interesting to see that the ICO already know the outcome of the “ongoing investigation” they are so keen not to prejudice:

      …”due to the illegal penetration and use of University computing facilities”…

      I’ve not been following that closely recently but I don’t recall seeing this being confirmed beyond supposition.

      • Jonathan
        Posted Dec 18, 2009 at 7:41 AM | Permalink | Reply

        Note that this is from UEA not the ICO; I don’t think it is unreasonable of them to treat the “position” as “true” in this case.

        • curious
          Posted Dec 18, 2009 at 7:49 AM | Permalink

          Noted – my mistake. I guess this would count as an official position from UEA. It will be interesting to see if the investigation confirms the “illegal penetration and use of University computing facilities”.

  85. Posted Dec 22, 2009 at 5:19 AM | Permalink | Reply

    I’ve been looking at these, but waiting for a review.

  86. Jonathan
    Posted Jan 10, 2010 at 7:37 AM | Permalink | Reply

    Following this up I asked Mr Palmer

    Thanks again for this. I would be grateful if you could send me a copy of the opinion of the Vice-Chancellor which you cite in your section 36(2) claim.

    Somewhat to my surprise UEA has decided to treat this as a formal FOI request.

    FREEDOM OF INFORMATION ACT 2000 – INFORMATION REQUEST
    (Our Ref: FOI_09-204)
    I acknowledge your request for information received on 21 December 2009 for a copy of the opinion of the Vice-Chancellor cited in the s.36(2)(b) exemption claimed in my letter of 18 December regarding a request for information under the Freedom of Information Act 2000. Your request is being considered and you will receive the information requested within the statutory timescale of 20 working days as defined by the Freedom of Information Act 2000, subject to the information not being exempt or containing a reference to a third party.
    I do apologise for the extensive delay in acknowledging your request; it was simply overlooked as it arrived when we were extremely busy responding to a large number of requests. I take full responsibility for this and do apologise sincerely.
    If appropriate, the information may be provided in paper copy, normal font size. If you require alternative formats, e.g. language, audio, large print, etc. then please let me know.
    For your information, the Act defines a number of exemptions which may prevent release of the information you have requested. There will be an assessment and if any of the exemption categories apply then the information will not be released. You will be informed if this is the case, including your rights of appeal.
    If the information you request contains reference to a third party then they may be consulted prior to a decision being taken on whether or not to release the information to you. You will be informed if this is the case.
    There may a fee payable for this information. This will be considered and you will be informed if a fee is payable. In this event the fee must be paid before the information is processed and released. The 20 working day time limit for responses is suspended until receipt of the payment.

    I await the result with bated breath!

    • Jonathan
      Posted Jan 23, 2010 at 8:47 AM | Permalink | Reply

      Re: Jonathan (Jan 10 07:37), I did eventually get a reply on 18th January. UEA very helpfully remind me that they hold copyright on the letter in question and I should not reproduce it without their permission, so if you want to see the whole thing I suggest you email foi@uea.ac.uk asking for “a copy of the decision notice issued by Edward Acton on 18th December 2009 concerning the application of section 36(2) of the FOI act to correspondence between Prof. Phil Jones and others”. In summary it pretty much claims a blanket exemption on all correspondence, including specifically correspondence concerning the handling of FOI requests.

      Steve: Could you please post up the exact language of their copyright notice.

      • Jonathan
        Posted Jan 23, 2010 at 1:31 PM | Permalink | Reply

        Re: Jonathan (Jan 23 08:47), Steve the relevant paragraph is

        I would also add that any material released over which UEA has copyright is released subject to the understanding that you will comply with all relevant copyright rules regarding reproduction and/or transmission of the information released.

        This would appear to be boilerplate rather than hand crafted for the occasion. From my own reading of the relevant acts they are entitled to make this claim, although I consider it very foolish of them to do so as any individual can obtain their own copy as indicated above. One might ask for a waiver of copyright, or request them to post a copy at their own website, to avoid a flood of such requests, and I am happy to do so if you wish.

        Steve: Just because material is copyrighted doesn’t prevent you from providing relevant excerpts from the letter. You are not permitted to use the letter for commercial purposes. If you want to post up the salient paragraphs of the refusal here, you are welcome to do so. I’ve posted up letters containing similar language.

        • Jonathan
          Posted Jan 23, 2010 at 1:55 PM | Permalink

          Re: Jonathan (Jan 23 13:31), Steve UK copyright law is considerably more restrictive than you might think, and as the letter is very brief even short extracts would make up a substantial portion of the whole. Since I have an appeal in with the ICO I am also keen to play this by the letter of the rules, even (especially?) when they are silly. Obviously if you obtain your own copy and post it that’s your business.

        • Jonathan
          Posted Jan 25, 2010 at 5:39 AM | Permalink

          Re: Jonathan (Jan 23 13:31), I have now emailed back

          Thank you for your email, copy included below.

          There has been some interest in the blogosphere in the contents of the letter from Prof. Acton given the extremely broad nature of the exemptions he is asserting. Unfortunately discussion is difficult as UEA’a claim of copyright over this material makes it difficult to provide an accurate account of his statements in their full context. Of course individuals can acquire their own copy of the text by making their own FOI request, but to avoid an undue burden (which would fall largely on your office) it would be very helpful if UEA were to permit posting of a verbatim transcript of this letter. Alternatively UEA might choose to place a copy of this letter on its own web pages and provide a link.

          I would be grateful for your thoughts on this matter.

  87. Posted Jan 25, 2010 at 7:21 AM | Permalink | Reply

    I agree with Steve here. UK copyright law also allows fair use which would include making quotes. I think in order to make a breach of copyright claim stick, you have to show commercial loss.

  88. Posted Jan 25, 2010 at 7:23 AM | Permalink | Reply

    Jonathan

    It strikes me that this is a pretty outrageous step by UEA when they are already under investigation by the Information Commissioner. I was speaking to someone from the ICO’s investigations team this morning, who might be interested in hearing about UEA’s position on copyright.

    Would you like me to call him? Drop me a line if you do.

    • curious
      Posted Jan 25, 2010 at 8:27 AM | Permalink | Reply

      Bishop Hill: Agreed – Anyway doesn’t copyright apply to “creative works”? Are they claiming the VC’s opinion is of the status of a “creative work” or is it based on law, precedent or policy?

      I would also think that FOI would qualify as an allowable act under the “statutory enquiries” exemption of para 8 item 6 here:

      http://www.copyrightservice.co.uk/copyright/p01_uk_copyright_law

      If it doesn’t – what value FOI?

      • Jonathan
        Posted Jan 27, 2010 at 2:12 PM | Permalink | Reply

        Re: curious (Jan 25 08:27), thanks for this. While Bishop Hill above is entirely right in practice (there is no way they would ever sue me, or that they would win if they tried), I have been seeking to stay rigorously within the (remarkably restrictive) letter of the law. However I find your argument, in essence that the VC is here acting in a semi-judicial role rather than a creative role, fairly convincing. I am also much encouraged by the recently reported good decisions the ICO has been making. So here is a transcript of the core of the VC’s statement:

        FOI requests for research-related correspondence between Prof Phil Jones and others

        18 Dec 2009

        This is to confirm that in my opinion section 36(2) b i) and ii) should be applied in respect of FOI requests relating to correspondence between Prof Phil Jones and others on the basis that:

        a) disclosure of this information would be likely to inhibit “the free and frank exchange of views for the purpose of deliberation”.

        b) In particular I view the ability of academic staff to converse/correspond freely and frankly about their work with other academic and support staff as essential to the proper assessment and evaluation of research prior to publication, and in my opinion this outweighs the public interest in disclosure.

        c) In my opinion this exemption should also apply to internal communications relating to the administration of FOIA requests.

        Edward Acton

        Vice Chancellor

        If anyone would like to take any of this up with the ICO that’s obviously fine by me.

  89. curious
    Posted Jan 31, 2010 at 1:55 PM | Permalink | Reply

    This looks a good course offered by the Campaign for Freedom of Information in London on February the 4th:

    http://www.cfoi.org.uk/pdf/foicoursefeb2010.pdf

    http://www.cfoi.org.uk/

7 Trackbacks

  1. [...] LINK AKPC_IDS += “2871,”;Popularity: unranked [?] (No Ratings Yet)  Loading … [...]

  2. [...] http://www.climateaudit.org/?p=6825  [...]

  3. [...] makes up the gridded Hadley Climate Research Unit HadCRUT dataset has been fraught with delays, FOI denials, and obvious [...]

  4. [...] The CRU has been less than forthcoming with these processes in the past, citing everything from lost data to political sensitivity to prevent other scientits from getting data and other information. When one scientist was finally [...]

  5. [...] to use that as an excuse to deny FOI requests for data and code, rather than lessons, is ‘misleading’ to put it [...]

  6. By The Final Straw « Watts Up With That? on Mar 2, 2010 at 4:18 AM

    [...] got from Jones, CRU started again with a series of denials again citing confidentiality agreements, inventing the terms of those agreements ex nihilo. Webster could have the data. McIntyre could [...]

  7. [...] document” is online at CRU here. It was reported at CA on August 11, 2009 here, here and here (among [...]

Post a Comment

Required fields are marked *

*
*

Follow

Get every new post delivered to your Inbox.

Join 2,874 other followers

%d bloggers like this: