Today brought in some CRU refusals- their rejections of Ross Mc, Roman M, myself. (They’re going to have to re-do their Roger Pielke rejection, since they replied to the wrong request in his case.) Each one deserves to be savored. So today I’ll post up their obstruction of Ross McKitrick.
FOI officer Palmer denied the request on the grounds that the request is “manifestly unreasonable” as the data is “available elsewhere”, that its disclosure would have an “adverse effect on international relations” and would have an adverse impact on the institutions supplying the data.
CA readers will recall that I requested the same version of CRU station data as was sent to Peter Webster and that they refused on the grounds that they had “confidentiality agreements” (all of which have been destroyed or lost other than stale agreements with Norway and Bahrain and and an agreement with Spain that does not require confidentiality) with parties that they can no longer identify, but the one thing that they were certain of was that these agreements prohibited the delivery of the data to a “non-academic”.
Ross McKitrick is obviously an “academic”. And aside from being an “academic”, he even has relevant publications in the field. Here is Ross’ original request:
Pursuant to the Environmental Information Regulations, I hereby request:
1. A copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009
2. A copy of any instructions or stipulations accompanying the transmission of data to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009 limiting its further dissemination or disclosure.
I write as an academic with publications in peer-reviewed journals and an ongoing research program on the subject of surface climate measurement. With respect to #2, please be aware that restrictions on data disclosure may disqualify any research arising from this data set from being published in many peer-reviewed journals, therefore I require a complete response as to whether any such instructions accompanied the data.
Thank you for your attention,
Here is the CRU response in full:
Your request for information received on 24 July 2009 for a “A copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009” and “a copy of any instructions or stipulations accompanying the transmission of data to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009 limiting its further dissemination or disclosure” has now been considered and it is, unfortunately, not possible to meet your request.
In accordance with Regulation 14 of the Environmental Information Regulations 2004 this letter acts as a Refusal Notice, and I am not obliged to supply this information and the reasons for exemption are as stated below:
Exception Reason
Reg. 12(4)(b) – Request is manifestly unreasonable. Information is available elsewhereReg. 12(5)(a) – Adverse effect on international relations. Release would damage relations with scientists & institutions from other nations
Reg. 12(5)(f) – Adverse effect on the person providing information. Information is covered by a confidentiality agreement
We believe that Regulation 12(4)(b) applies to your request for the data because the requested data is a subset of data already available from other sources; namely the Global Historical Climatology Network (GHCN ) , and the Climatic Research Unit already makes requested information available on it’s website in a gridded format. We believe, following DEFRA guidance, that it is unreasonable for the University to spend public resources on providing information in a different format to that which is already available.
In regards Regulation 12(5)(a), much of the requested data comes from both individual scientists and institutions from countries around the world. If this information were to be released contrary to the conditions under which this institution received it, it would damage the trust that other national scientists and institutions have in UK-based public sector organisations and would likely result in them becoming reluctant to share information and participate in scientific projects in future. This would damage the ability of the University and other UK institutions to co-operate with meteorological organisations and governments of other countries.
Regulation 12(5)(f) applies to the data requested because the data was received by the University on terms that limits further transmission. We believe that there would be an adverse effect on the institutions that supplied data under those agreements as it would undermine the conditions under which they supplied the data to the Climate Research Unit.In regards your request for any stipulations accompanying the transmission of the data to academics at Georgia Tech, no such instructions or stipulations are held by the University.
All the agreements that we do hold in relation to the requested data are available on the Climate Research Unit website at: http://www.cru.uea.ac.uk/cru/data/availability/
Regulation 12(1)(b) mandates that we consider the public interest in any decision to release or refuse information under Regulation 12(4). In this case, we feel that there is a strong public interest in upholding contract terms governing the use of received information. To not do so would be to potentially risk the loss of access to such data in future as noted above. In regards Regulation 12(4)(b), we believe it is not in the public interest to divert public resources away from other work to provide information that is available elsewhere. Finally in regards Regulation 12(5)(a), we feel that there is a clear public interest in neither damaging nor restricting scientific collaboration between UK-based scientists and institutions with international colleagues.I should note, however, that the University is commencing work, in concert with the Met Office Hadley Centre, to seek permission from data suppliers in advance of the next update of the CRUTEM database in 2010 in order to provide public access to this data. This work has been announced on the CRU website and further updates on it’s progress will be available there.
I apologise that your request will be met but if you have any further information needs in the future then please contact me.
Climate Audit 
162 Comments
The coverup is always worse than the crime.
I love the first one – it becomes a catch all for rejection of a request for any data:
1) We refuse to provide you with the reasons for the rejection of your application for political assylum because they are a subset of the reasons that have been used for other rejections.
2) We refuse to provide you with the currencies and exchange rate(s) that have been used in our calculation of your fees because they are a subset of the foreign exchange data which is already publically available from multiple sources.
3) We refuse to provide you with the definition of “academic” that we use in judging requests for data by “academics” because our definition is a subset of the set of all definitions that are already publically available.
The needle is in the haystack – you find it!
I’m wondering how they reconcile the transmission of data to Webster with the apparently devastating results that occur upon transmission of data.
Re: Ryan O (#3),
They know the sound a boomerang makes as it is thrown and are thus able to dodge it successfully when it comes back.
Just slowly read these sentences out loud.
One more time
If your tea has not yet snorted out your nose, read again. It will hit you one of these times.
Re: Ross McKitrick (#4), Right. There are stipulations on the data, with the exception that there are no stipulations on the data.
Obviously, under such nonstipulated stipulations, we cannot supply the requested data.
Re: Ross McKitrick (#4),
It’s good to know someone finally found a use for triplespeak.
Only in climatology.
Re: Jeff Id (#25), is it Orwellian or Kafkaesque? I think Kafka is more relevant than Orwell here. But doublethink is undoubtedly an advantage when writing such letters.
Re: Jeff Id (#25), Don’t you know? This method of communication is insensitive to the sign of the input.
I think this is a stalling tactic whilst they stomach pump the pup. IMO this will need an onside UK politician to expedite it – otherwise it is just going to be climb aboard the merry-go-round of back and forth requests which would ultimately require a formal complaint and all the delay and aggro that would entail.
Claiming:
when all they have to do is resend the data already sent to Webster is pathetic. The EIR states they have to make every effort to assist. Or am I missing something – are they claiming that the website version is what they supplied to Webster?
And I’m not sure about the last line – is that a cut and paste thing or as per the original?:
I await the next instalment with interest!
Ross – kettle on!! 8<~0
If you enjoyed the first installment of this British farce, here’s the second.
We can’t send you the data since it’s already available in so many places, including the CRU web site. Furthermore, we can’t give it to you since we’re forbidden from making it available.
Maybe Hugh Laurie and Stephen Fry could work up a sketch around this.
Re: Ross McKitrick (#7),
Is there an implication here, deliberate or otherwise, that the processed, value added data transmitted by CRU to qualifying academic researchers is fully equivalent to the raw source data sent to CRU by various NMS’s, but is simply being converted to another format before transmission? If this is so, then what specifically is the nature of the “value” CRU is adding?
Re: Ross McKitrick (#7),
It’s a sketch right out of “Little Britain
“http://www.bbc.co.uk/comedy/littlebritain/
… “I want that one…”
Re: Ross McKitrick (#7),
I thought the data sent to Peter Webster contained the raw data and that that was therefore at least part of what you requested. And since the gridded data is not the same thing nor even equivalent to the raw data (at least without the code that converts the raw data to the “added-value” gridded data, and probably even with it too) why are they implying the two things are the same and that the difference now is merely one of format? WTF?
Have I got that right?
Also:
That doesn’t say “already makes all requested information available” or even “already makes the information you requested available“. Sure, “all” is logically implicit. But still, they might be playing word games here too.
“I always put my shoes on when I go out.”
“What, all 5000 pairs, Imelda?“
Re: Ross McKitrick (#7),
This reply is plain nonsense.
They say that the requested information is publicly available (which is partially true, because the CRUTEM dataset almost entirely rely on station data archived in GHCN) Despite this fact, they try to avoid releasing their data at all costs…
Re: Adam Soereg (#62),
I’ve examined enough station data to say that the CRUTEM data is not a subset of GHCN. Even for stations that are GHCN stations, CRUTEM versions can differ materially.
CRU’s statement that their data set is a “subset” of GHCN is simply untrue.
In my opinion, they do themselves no good by making untrue statements, as their untruthfulness will be demonstrated. And, Ford, if someone makes untrue statements, they have only themselves to blame for the consequences even if you prefer to blame the person who merely reports that the statement is untrue.
Are you sure this hasn’t come from Laurie and Fry already?? Please check the postmark! 🙂
David Palmer placed an apostrophe in the possessive pronoun “its” at “on it’s website”. Perhaps a [sic] should be placed after its inappropriate use.
Re: Richard Henry Lee (#9),
“IT’S…“
Did they just bump months before release to at least 6 months, 2010? —
I’m starting to suspect that these guy’s aren’t being honest.
It would seem like the file is just about ripe to send in full to the information commissioner. But, I suspect that a formal internal appeal is necessary before reaching that stage. The farce is amusing, but at some point an external oversight body needs to be brought in to formally comment on the ridiculousness of their behaviour.
If I can paraphrase:
1) Its a secret.
2) It’s publicly available.
Is that about right?
JS – see page 4 and 5:
Click to access ico_foi_eir_how_to_comp_final.pdf
Let me revise that.
1) Its a secret.
2) It’s publicly available.
3) It’s a secret that we spilled the beans to Georgia Tech.
4) We don’t have a bean-spilling agreement with Georgia Tech.
5) If we spill the beans to you, some of the other people to whom we spilled the beans might not spill their beans to us in the future.
6) Your beans don’t count.
Anything else?
I received something similar to my request:
http://www.whatdotheyknow.com/request/raw_climate_data#incoming-39657
I’m going to use the weekend to decide how to respond to them!
They have to know you are going to publish these responses, and knowing this how can they make such ridiculous claims? Don’t they know how stupid it makes them look?
Re: Paul Penrose (#19),
Which kinda puts puts the quietus on Steve & Lucia’s bend-over-backwards-to-be-reasonable alternative explanations (“Maybe they’re just prima-donnas”, etc), ie: possible but vanishingly unlikely, ie: the laughably transparent stonewalling may make them look stupid, but not nearly as stupid as the state of the data would were it to become public. Least of all evils.
BTW, I’m still undecided – I don’t think Full Disclosure would prove catastrophic AGW is a hoax; more likely it would expose laughably non-existent data management practices.
Huh! My underlining disappeared.
I’ll have to rejig things. I’ll do Mannian transformation.
Where’s my magic wand?
I would like to hear Ali G read these CRU responses.
Rejigged attempt of my #21 (now with my bold in the block quotes).
Re: Ross McKitrick (#7),
I thought the data sent to Peter Webster contained the raw data and that that was therefore at least part of what you requested. And since the gridded data is not the same thing nor even equivalent to the raw data (at least without the code that converts the raw data to the “added-value” gridded data, and probably even with it too) why are they implying the two things are the same and that the difference now is merely one of format? WTF?
Have I got that right?
Also:
That doesn’t say “already makes all requested information available” or even “already makes the information you requested available“. Sure, “all” is logically implicit. But still, they might be playing word games here too.
“I always put my shoes on when I go out.”
“What, all 5000 pairs, Imelda?“
Really, I feel bad to see anyone discredited. But, when folks are at the bottom of a hole and just keep digging to get out of it, I simply cannot be sympathetic. I must laugh at them!
You guys have missed the best typo of the lot.
lols.
Does anyone else consider this to be a very, very strange issue one which to make a final stand and utterly destroy the reputation of your institution?
Re: BradH (#29),
It was a very, very strange issue right from the start. The pace has picked up somewhat recently but otherwise it seems to me to be business as usual with the screamers.
The climate record accountants’ FOI record keepers messed up their record keeping?
I posted on this in WUWT when it first appeared and that I would make some inquiries but that I expected it would take some time, not least because it was coming into August, holiday season etc.
So the impossible we do immediately miracles take a little longer.
I note that Prof. Jones, NO RELATION, has got himself in a tizz over this perhaps for a number of reasons I can guess at.
You must understand that for this purpose there are three tiers in the UK system HMG departments themselves which answer to a Permanent Secretary, a civil servant, and thence to the Minister, UK government agencies, which are quasi autonomous and carry out specific functions for HMG departments, and finally many universities whose research is funded by HMG: as is the case with CRU.
Now apart from knowing which departments you need to apply with an FOI request, you should also know that whilst one set of regulations may apply to one department of HMG, they do not necessarily apply to another. Wrong department wrong FOI request, wrong regulations.
Now as I suspected the CRU data and the Met Office data is also lodged, in whole or in part, with other HMG Departments, and I think an agency as well.
At the moment I cannot confirm whether this is raw data or adjusted data or how complete the records are.
Moreover as I thought research funded by HMG, whether it is a university and thus CRU or indeed the IPCC itself, probably cannot be covered by formal confidentiality aggreements. These can only apply in the UK if either the research is for commercial purposes, such as the association of widget makers asking for help from universities to improve their widget making process, or to the anonymity of unpaid individuals responding to surveys, such as what do you think about a blue moon?
What is more if any restrictive agreement outside of the above exists it has to be notified to the appropriate HMG department.We don’t yet know what that department might be but we can ask the law departments, who have to answer, FOI or no FOI, to look it up and tell us. Don’t hold your breath, these boys can take weeks to answer a letter.
Finally of course if the information is not covered by the Official Secrets Act, and I cannot see why it should be, then the governments of certain Crown Dependencies can ask HMG for it, and what might be confidential in the UK is not necessarily so in their jurisdiction. Can we persuade them to ask, I think so.
So you see there is more than one way to skin a cat, but it is slow patchy work, and I suspect, nothing to do with me at this stage, Dr. Jones may decide, or possibly has been advised, that making a bare breast might be a politically desirable move at this point.
snip – p[olicy
I shall continue to press for some answers and advise, but again don’t hold your breath, it may be that given the shifting political sands you will get the answers quicker than I can. I do hope so, but if not I will get them eventually.
Kindest Regards.
The phrasing in the “non-academics” reply was:
It appears that they sent the data to Georgia Tech without telling Georgia Tech of any restrictions. And perhaps with good reason; if 12(5)(f) does not apply to Georgia Tech then Georgia Tech would not be bound by such a restriction. The “Environmental Information Regulations 2004” look like they are UK regulations, thus Georgia Tech doesn’t have to follow 12(5)(f).
I sent in a request and received a much more reasonable reply than the above, one which I hope will clear things up for one and all:
Expecting data to be FedExed shortly.
I’ve always loved the British sense of humour. I think you are all being taken for a ride here. The refusals are obviously penned by someone who desperatley wanted to be one of the Monty Python crowd. He wears Eric Morecombe glasses, drives a Mr Bean mini, and does that disturbing Benny Hill funny walk. He proably even has a Steptoe fashion sense (I don’t think these guys get out much) and his betamax VCR is in meltdown watching “Carry On Up the Weather”.
The real problem is, I’m sure he fits in at CRU just fine.
Lets face it, its the only explanation that really makes sense!
Elsewhere I mentioned a Royal Commission on espionage in Australia as preserving minute detail in great quantity from the mid 1970s, in the context that the technology existed and that data can be retrieved today.
One statement that the above discussion brings to mind is –
Can’t you image His Honour saying words like this from the Bench eventually in this matter?
My request pre-dated the recent set by a week or two and seemed to get lost by the influx for which David has apologised. My request for the agreements has apparently been satisfied by that new web page, but I add the following couple of nuggets, one of which was rejected with the to much effort defence (s12), so I’ll be appealing that one on the grounds that the request does not require them to go back over 15 years to satisfy the question given that he’ll have an inbox full of such information over the last month or so.
GISS/Hansen use Oslo up to the 1980s and get the full UHI from increase in population from 10.000 in 1815 to 500.000 in the 1980s.
Then they change to Gardermoen 30 miles North that is very rural at that time, but later in the late 1990s become the main airport for Oslo and is now beeing urbanized much and fast. Probably give a lot of renewed UHI effect today and in the future?
I wonder where they will jump next, when Gardermoen has been fully urbanized and there is little further increase in UHI effect?
UHI jumping?
Steve and Ross
You have done things by the book to date.
Don’t you think it is time to eascalate this to the British Press?
Both Christopher Booker-Daily Telegraph- and Alexander of the Daily Mail might get some traction and they also know various MP’s who might take it up. Also our local Euro MP is from UKIP- a party who have an official Sceptical policy towards climate change. They might also be willing to ask questions in the right places.
Tony Brown
“We can’t send you the data since it’s already available in so many places, including the CRU web site. Furthermore, we can’t give it to you since we’re forbidden from making it available.
Maybe Hugh Laurie and Stephen Fry could work up a sketch around this.”
This is more Monty Phyton Flying Circus?
Here is the response I got.
Oh be reasonable-its obvious that the dog ate their homework and they’re too embarrassed to say.
Mr McIntyre
I repeat
If Georgia Tech has a copy then get it from there. They will surely honour FOI requests, even if Peter Webster verbally agreed not to distribute.
You do not reply.
It is quite plausible that Georgia T and UEA have no documents concerning the restrictions on the data as I though the data was passed as part of a collaboration between webster and jones.
Why not believe that the data is hamstrung by confidentiality agreements?
Why not believe that they are now working to remove those agreements?
A couple of months has been suggested before this can happen. Can you not wait? Or is it more important to destroy the reputation of CRU?
Mike
Re: thefordprefect (#45),
It’s not Steve that is destroying CRU’s reputation it is their own lack of competence. There is never any good scientific reason for hiding your data and there doesn’t appear to be a significant number of confidentiality agreements to prevent them from doing so. If you can see the consequences upon their credibility of their lack of cooperation on this issue why cant they? The continued stonewalling will produce nothing but negative press.
Re: EdBhoy (#52), Scientifically they should not hide their data – LEGALLY they have to (would you copy music/videos and pass them on?).
They say they are trying to free the data – is this not acceptable. If not then try FOI on Peter Webster at GT. He has the data!
Re: thefordprefect (#53),
Ford, I don’t think that you’ve fully followed the chain of thought here.
CRU’s collection of station data was funded by U.S. nuclear labs. As we know, CRU delivered its station data collection, as it then stood, to CDIAC, a unit of the Oak Ridge nuclear lab, in the early 1990s, who then placed it online, where it remains to this day.
Previously, no one would have thought that there was anything untoward about that.
however, CRU has now informed us that its data from that period was collected under various confidential agreements that prohibit the present data set from being transmitted to me, Roger Pielke Jr, Ross McKitrick, Roman Mureika and others, but not to Peter Webster.
unfortunately, they are unable to produce any agreements that contain language that permit them to make this distinction.
if the language of the few agreements presently online is representative of the “lost” agreements, as CRU allege, then it appears to me that CRU never paid the slightest attention to these agreements. For example, one of their agreements requests 1961-1990 normals for a specific academic project and states that the data will only be used for the purposes of that project and will not be passed on to third parties. While they interpret than language against me, the language is much broader than that – it prohibits the transmission of the data to Peter Webster, to CDIAC and to the Met Office and its use in the construction of the CRUTEM index.
Their request letter gives no indication that their collection is funded by the Oak Ridge nuclear lab in the U.S. and that they planned to pass the information on to the Oak Ridge nuclear lab.
In my opinion, CRU would be better off making the decision that the alleged early confidential “agreements” were ineffective or superceded by WMO Resolution 40 or something like that, rather than playing out the hand. However, I often make sensible suggestions to people, that they disregard.
Re: Steve McIntyre (#54),
I believe you said that Peter Webster at GT was to use the data in a joint project with CRU. Legally how does this pan out? [Steve: I didn’t say this; Peter was reported as saying this.]
You seem to be suggesting that CRU forget about legal issues and pass the data? Would this be wise for CRU? [Steve- I am absolutely not saying that CRU forget about legal issues. However, I am completely unconvinced by their legal reasoning. Thus far, they haven’t provided a single shred of evidence to support their position that they can provide data to CDIAC, the Met Office, Advance 10K dendrochronologists, Peter Webster, but not to Warwick Hughes, Ross McKitrick, Hu McCulloch, Roger Pielke, Roman Mureika or me. I do not believe that there is a valid reason.]
To me it seems that your one aim is to ridicule the CRU and destroy its reputation. If this were not the case then why do you carry out these actions in full view on a blog? It can only be to provide maximum embarassment to CRU. [Steve – my objective is to make the data public.]
May I suggest that you wipe all threads pertaining to this issue and wait for CRU to free up the data? [ Steve- Again, I do not believe that there is any valid legal reason why CRU can’t provide the data right now and intend to pursue my own appeals and report on them.]
These FOI must be costing a small UK university a fortune to process yet you enthusiastically encourage your followers to continue. The funding For the CRU research is running at about UKP40,000 per month Your actions must be costing them a sizeable proportion of this. By continuing this action there is a possibility that such funding goes elswhere in future. [Steve – they have a FOI officer on staff.]
This FOI attack is obviously being organised on this site. Be aware of the things I privately emailed you a few months ago!!!!!
Why can you not accept that the data is commercial?
Why can you not accept that CRU are trying to free up the data?
Why continue FOIing attacks?
[Steve – I have seen no evidence that the data is commercial. They’ve turned data over to CDIAC and others without compunction. I have seen no evidence that CRU has made the slightest attempt in the past 5 years to “free up the data”. On the contrary, over the past few weeks, they’ve deleted data that has been available for up to 13 years.]
This blog was once a source of honesty. It has decended into hatred. I am saddened by this!
Mike [ Steve: I strongly disagree. I for one do not “hate” anyone. I dislike bureaucratic obfuscation and don’t mind taking the extra effort to oppose it. contrary to your characterization of the blog, I think that one of the great strengths of blogs is the ability to expose bureaucratic obfuscation. Rather than get frustrated with stonewalling, one can place sunshine on it.]
Re: thefordprefect (#56),
Ford. As you can see from my FOI CRU have denied my request for their PROCEDURES ( read employee handbook)
that govern entering into confidential agreements. They determined it would take them more than 18 hours to
find this. So, I don’t see how you support your claim that this is costing a small fortune.
Re: thefordprefect (#56), I hope you will at least acknowledge that these recent efforts by CRU to liberate the data only began in response to the pressure on this site. If you think the application of so much public pressure makes the various parties awkward and uncomfortable, well so do I. But that’s apparently what it took to get the ball rolling.
.
For the record, I don’t expect there are any clerical errors, data shenanigans or computational problems in the CRU compilation, and if there are glitches I doubt they would affect the global average very much. I don’t expect there to be any major problems in my bank’s annual financial statements either. But I feel much more confident about the latter knowing that there are independent auditors going over the books. My confidence in their accounting systems does not mean that I would support a suspension of the laws governing audits of public corporations.
.
As an academic matter I would like to check the extent to which spatial correlations between surface grid cell trends and indicators of nonclimatic contamination differ between the CRU input data and their gridded output data. I can only do that if I have the exact input data. If they refuse to release it then their claims to have removed the nonclimatic biases cannot be verified.
.
The other issue concerns what status should be given to CRU data. If they are prepared to issue an “as-is” warning, saying that the rules under which they operate make it impossible for them to permit independent replication of their work, and they cannot guarantee that even they would be able to replicate it themselves since they may not even have their original data, then they could be excused for this secrecy, since users would have fair warning. But they put their data forward as a global record of sufficient reliability on which to base public policy, international treaties, downstream scientific research, etc. If they want that prestige, they need to accept the quality control commensurate with it. It’s a question of not having things both ways.
Re: Ross McKitrick (#90),
I for one want to find the country that says the following.
1. Our data is too valuable to make public.
2. We support public policies based on this secret data that will cost YOU trillions.
In short the world is considering global policies that have rather large costs. Those policies in part derive their
justification from this data. On principal that data should be available to those it would impact. Colin Powell in front of the UN comes to mind.
Re: thefordprefect (#45),
Well, you send an FOI to Webster or Tech and let us know.
You say Georgia Tech has a copy. I say CRU has a copy. Why can’t Steve get it there???
Have you asked Jones to explain his statement of “Why should I give you the data so you can just find something wrong with it?”
Steve: I am not going to ask Peter Webster for the data. Nor do I want any CA readers to do so. It’s CRU’s obligation to provide the data or provide an explanation of why they won’t that is valid. Thus far, they haven’t done either.
Re: Gerald Machnee (#57), Comment by McIntyre
And it would not fit in with your Vendetta against CRU
The data is not CRUs to give in the same way it is not Webster’s.
The data is not given because it is commercial.
Steve: You say: “The data is not given because it is commercial. What do you not understand?” They’ve given a variety of reasons for refusing the data, commencing with the reason given to Warwick Hughes. However, the most recent reason given to me is that the y have confidentiality agreements that specifically prohibit transmission of the data to a “non-academic”. Perhaps this is not the “true” reason and the “true” reason is the different one that you propose here. However, I am guided by CRU’s actual response.
.
Re: thefordprefect (#60),
You mention that CRU costs the tax payer £40000 per month. Public money comes with public responsibilities that include transparency. Why do they not want to publish the raw data? Steve asked politely for the data and they refused to give it to him for reasons that cannot be justifed academically or legally. The few confidentiality agreements they have published this week are hardly a serious obstacle to passing the data to Steve.
The UK government and the rest of the developed world are committing themselves to trillions of dollars of expense in carbon trading, clean coal technology etc. etc. based partly on the CRU data. We have to be able to audit it independently and Steve is ideally placed to do so. Phil Jones’ exclamation that Steve will just try to find fault with it is ridiculous. Every climate scientist or statistician should be trying to find faults in the data as that is how scientific method works. If they succeed they may save the world a huge fortune, if they fail then the world will thank them for helping to raise the Global Warming alarm.
This is not an attack on CRU. The issues are much bigger than that.
Re: EdBhoy (#68),
I said they had funding I did not say it was public.
They have said they will work toward publishing the data.
It is commercial data!!!!!!!
You too suggest that CRU acts illegally and distributes other peoples property!
Steve: Ford, understand what you’re accusing CRU of. You’re accusing CRU of acting illegally for the past 15 years. That’s not us making the allegation, it’s you. They’ve sent the “commercial” data to CDIAC, to Advance 10K dendrochronologists, posted two separate versions on their website, sent it to Met Office, apparently used data requested for a project on 1961-90 normals for inclusion in CRUTEM and HadCRU and in 2002 even sent the data to me. According to you, all these actions are illegal. I hope that you have some evidence to back this up. My own position is different: I assume that CRU has been acting legally all these years. If they’ve been acting legally in the past, then they should send me the data. If you insist on maintaining your accusations that CRU has acted illegally in the past, then I guess the chips will fall where they may, but understand that you’re making the accusation, not me.
Re: thefordprefect (#70),
CRU have not been able to demonstrate any significant legal impediment to being able to distribute the data to Steve or Ross McKittrick etc.. If they could do so the FOIs would fail. I don’t know why they continue to stonewall, it just doesn’t do CRUs reputation any good whatsoever. Since they did give the data to Steven Webster, the Met Office and others it seems that they discriminate when deciding who should or should not get the data. If it were simply a legal issue they would not have passed it to anyone.
Mike, there is a bigger issue here. Every scientist with an interest in climate change should be demanding that the raw unadulterated data be freely available and that any method used to process the raw data be published. The costs of getting this wrong on either side of the argument are far too high for us to blindly accept unreproducible results.
Re: thefordprefect (#60),
Yet CRU gave it to Webster.
Mike – CRU are destroying their own reputation. Just read the words.
curious,
That’s the question I raised not long ago. Why could not releasing the data be worth destroying their entire reputation?
Steve said he doesn’t expect to find anything surprising in the data. Who knows, it could be that Steve would discover that global temps have been even understated. It’s kinda like risking a major car accident to swat a fly in your car.
Just got my refusal; the same laughable excuses. I shall of course be appealing.
Re: Jonathan (#49), I have now lodged my appeal as follows.
Re: Jonathan (#121), I am promised a reply to my appeal by 24th September at the latest.
Re: Jonathan (#122), RealClimate are busy congratulating themselves on the commitment to open data sharing in the climate science community. My comment about obfuscation at the CRU was not thought worthy.
Re: Jonathan (#121), the saga continues!
We’ll see how this turns out.
The Register, 13 August 2009
http://www.theregister.co.uk/2009/08/13/cru_missing/
“The world’s source for global temperature record admits it’s lost or
destroyed all the original data that would allow a third party to
construct a global temperature record. The destruction (or loss) of the
data comes at a convenient time for the Climatic Research Unit (CRU) in
East Anglia – permitting it to snub FoIA requests to see the data.
[…]”
For the time being, can we get insight on at least simple average of all stations used in HadCRUT from the source-which-must-not-be-named? We already know it is not “influential”, but yet.. 😀
Lord Monckton is “our friend.” Do we know anyone in the House of Commons who might be able to raise this issue of flagrant misuse of FOI during questions to the PM? This whole issue really needs a “light of day” moment.
#26
I wouldn’t dignify these pathetic responses with such lofty expressions. They are more akin to the incoherent excuses that would pour forth from a boy with his hand in the cookie jar.
How many studies are dependent on this secret data, directly or indirectly? Is this the data that is used to reconstruct these graphs?
This is been an amazing thing to follow. It’s getting eerily like a scene out of Brazil. Maybe you need a 27B stroke 6 form to get the data? Of course, they can’t give you a 27B stroke 6 form, because in order to get said form you need a signed, stamped 27B stroke 6 form in triplicate, and they can’t tell you how to proceed because that’s classified.
It seems possible to me that the official sending these denial letters has no clue that the data requested differs from the gridded data available at the cru website. That someone like Ross might want to compare, let’s say individual citys or countries weather trends to their economic status, as he did in a publication or two, may escape them. To this official, perhaps data is data and he thinks that requests are just asking for special favors in format or something. Assume ignorance before bad motives.
Re: Craig Loehle (#64),
Having been in a records management type of career position myself as an official custodian of scientific data, engineering data, and also of various kinds of sensitive business data — I can say with some authority that one simply can’t accomplish one’s job without having a basic understanding of the data’s fundamental characteristics, its potential end uses, and its historical provenance. Something else is going on here. But what?
This gets better and better. They are not releasing the data because it’s confidential and top secret and available elsewhere.
The more they refuse and come up with ridiculous self-contradictory excuses, the more publicity this will get and the more stupid they will look. It’s even funnier that they keep referring to ‘public interest’. Sooner or later this will escalate to a higher level when it will be ruled that it is overwhelmingly in the public interest that the data should be released.
.
Ford, I feel like digging out the old joke about what planet you are from again. What is the commercial value of the temperature in Timbucktu in April 1967? Lots of temperature data is available. The request is for the exact data on which CRUTEM (and hence the IPCC report) is based.
Regarding your naive suggestion to just wait for CRU to free up the data, you should read the “A 2002 Request to CRU” thread – they said they would release the data back in 2002!
What do you not understand?
Re: PaulM (#65), It is commercial data. I.e. someone else OWNS it not CRU. It is not secret.
If you by a dvd the price you paid does not give you right to redistribute that data to others – simples
Re: PaulM (#65), I thought the data was “accidentally” placed on an open server in 2003.
OK then, continue harassing CRU. I give up. You seem totally incapable of understanding what commercial data means.
Re: thefordprefect (#70),
“You keep saying that word… I don’t think it means what you think it means.”
Seriously, I hear you keep repeating that the data is “commercial,” but I don’t know what you mean by it. DVDs cannot be redistributed because they are copyrighted. Are you suggesting the raw data is copyrighted, or are you simply arguing by bad analogy? (The inside joke among attorneys is that “Legal reasoning is reasoning by bad analogy.”)
Perhaps more importantly, I have no idea why you think it. Simply repeating it over and over again doesn’t make it true.
Re: QBeamus (#118),
Inconceivable.
Mark
Knowing a lot of government types (I used to be one), or closely associated people – I can safely say that Jones’ behaviour is pretty typical in my experience, and is most likely to be as a result of the following items:
1. Ivory tower building.
2. Not-invented-here mindset.
3. Worrying realisation that a ‘house-inspection’ of said ivory-tower would reveal a somewhat disorganised house; not in keeping with written government standards.
4. A slight, nagging, doubt that possibly, just possibly, an effective lifetime’s work may have some fundamental flaws or errors.
Cheers
Mark.
theforcperfect–
If you think asking Peter Webster for the data and making it available to the world would solve some problem, why don’t you ask Peter Webster for the data and then post the data and maintain it so we can all have the sort of access you feel should be sufficient to our needs.
Mind you, I don’t think making this request of Peter would solve the problem that CRU refuses data to people who deserve to have access because I think the problem is CRU won’t make the data public. Clearly, Steve doesn’t think getting the data from Peter solves the problem.
But if you think this is the solution to “the problem” or “some problem”, or if you think your providing this data would make some important point you think needs to be made, I think rather than asking others to get the data from Peter, you should do get the data and make it available to those member of the public who might want it now or in the future. After posting it, you can explain how you have solved the important substantive issue related to data availability.
To help you, I’ve gone to the trouble of finding Peter’s homepage which includes his email address. You will find it here.
Re: lucia (#67),
I think rather than asking others to get the data from Peter, you should do get the data and make it available to those member of the public who might want it now or in the future. After posting it, you can explain how you have solved the important substantive issue related to data availability.
I do not want the data.
If I got the data, my integrity would prevent me passing it on unless the data sources gave approval.
I would hope Peter Webster would also refuse the FOI as it is not his data to give.
I don’t understand why his data is wothless and only the data from CRU is required. Take the FOI and place it at the feet of Webster, I say.
Re: thefordprefect (#60), McIntres comment
Perhaps commercial is not valid for all the agreements. But if it is confidential the same inabilty to transfer is implied. It is still not “secret” data.
Re: thefordprefect (#73),
Why in the world would you suggest SteveM file a request at Georgia Tech if you hope that Georgia Tech would not grant it?
I’m mystified that you don’t understand why data underlying the CRU product is best obtained from CRU. I am particularly mystified if you think that third parties (like you or Webster) would violate their integrity by passing it on.
One of the reasons the data for SteveM’s project must come from CRU is that third parties Peter Webster could be accused of lack of integrity for passing the data on.
Re: lucia (#74), Exactly – you’ve got it at last:
.
CRU do not own the data. Their integrity would be questioned if they passed it on to third parties. It is not CRUs poblem it is the restrictions of the first parties that is the problem.
CRU have more than integrity to loose, however. If the data sources did not like what was being done with their data then they could stop providing it.
To have given megabytes on this on this and all the other leech blogs is unbeleivable. The whole way this has been handled seem to many to be a way of discrediting CRU.
Mike
Re: thefordprefect (#78),
Ford, you consistently refuse to answer my responses to your post. It is a matter of record that CRU has passed the data on to third parties, including a US nuclear lab. It is you that believes that transmission to a third party impugns their integrity, not me. I think that they were entitled to do so, precisely because there were no relevant confidential agreements. Is it your position that CRU acted illegally in transmitting data to the Oak Ridge nuclear lab?
Please answer this before making any other posts.
Re: Steve McIntyre (#79),
The question was so ridiculous I did not realise it was a question.
I cannot answer for what CRU have or have not done. I do not know what agreements were in place between the parties who received the data – do you?
Witout full knowledge I cannot answer you question, and you know that.
I do know that should data be released to 3rd parties Oak ridge, Webster – a co-author, other professional/academic institution then it is unlikely to be passed on. If it is given to a blogger who professes that the data should be released then I would be less sure of this.
Re: thefordprefect (#99),
Ford, now YOU are stepping in to set the conditions of all these agreements ( some of them verbal) which either..
1. Don’t exist
2. exist but where lost.
3. were made but not memorialized in writing.
4. were never made to begin with.
But just to show your utter lack of understanding I give you this.
You put this condition on: Co authorship.
Do you seriously believe that Norway or Spain or WHOEVER gave the data to
Jones and stipulated that he could only release it to people he was
“co authoring with” The facts of the case point to this. Jones and CRU are
using the agreements to control data in a capricious self interested way.
They are not protecting commercial interests ( hence the positing of this data by mistake) they are protecting their interests. not the public interest, not the interest of other parties.
Re: Steve McIntyre (#80), UEA has now stated that the release of data to Georgia Tech was an “error” and that they were not entitled to do this. No statement about other releases of course.
Re: Jonathan (#136),
Could somebody do an FOI to determine if this error has occurred before or since the Georgia Tech incident?
Re: thefordprefect (#78),
Exactly – you’ve got it at last:
– Their integrity would be questioned if they passed it on to third parties.
– CRU have more than integrity to loose, however.
BUT:
They have passed it to third parties. For Years. But only to “belivers”
They themselves reduce the value of their own work by not provide the data for their now “scientific”(?) work.
The only reason why they refuse must be that if they let the data be properly evaluated will they be in a even bigger mess than today.
Re: thefordprefect (#78),
First you suggest to Steve that he get the data from Webster and now you agree with Lucia that Webster should not release it. You are doing enough 180’s to spiral underground. Steve has been correct all along. It must come from CRU. In addition he and others have been asking for documentation to show why the data cannot be released. There has not been one legitimate reason for CRU acting the way they did.
In addition you accuse Steve of having a vendetta. You should really check who has the vendetta. Steve is seeking information.
I think this has gotten to the point of mud wrestling with a pig. Steve, Lucia, and everyone else, you can’t win mud wrestling a pig, because the pig enjoys it.
Re: Calvin Ball (#82), Who said Steve doesn’t? 🙂
Mark
thefordprefect–
Re: thefordprefect (#78),
But you don’t get it. it’s you and CRU would would accuse Peter of lack of integrity for passing it on. CRU personell might, in consequence add him to the list of people to whom they would not release data. This would have negative consequences for Peter. If Peter wasn’t asked to sign any confidentiality agreements, I wouldn’t think badly of him at all.
But why should SteveM ask Peter to do something that might cause CRU to think badly of him and possibly retaliate?!
In anycase, SteveM wants CRU data. He wants to be able to say that he is sure it is CRU data. Logic dictates he should get it from CRU.
Over time, we will discover which confidentiality agreements actually exist. The inquiries to date seem to be the factor that has motivated CRU/Jones to make the effort to locate the
manythree bits of paper and to prod Jones to refresh his memory about any verbal ones by contacting various countries. This seems all to the good to me.Re: lucia (#83), Yet again you confirm what I have been saying all along:
Why should McIntyre ask CRU to do something that might cause the data sources to withdraw their data from fututre releases in retaliation for breaking verbal/written agreements.
McIntyre should only get data from the sources if he really wants to check things out Otherwise how does he no that paper records have been correctly digitised? As this could be costly and difficult then the second best is to get the data from the compiler – CRU.
Re: thefordprefect (#97),
And I’ll say again. Who cares if CRU loses access to data? Their incoherent stonewalling, making up reasons as they go along, demonstrates that they don’t deserve access to the data, let alone any respect for any product which may slide off their sloppy desk onto the web or into policy makers hands.
In his response to James S. (#18), Mr. Palmer cites the Environmental Information Regulations 2004, which have produced a Code of Practice that he should follow. I find it odd that he mentions agreements that go against dissemination of information, when that is what the Regulations says one who enters into ‘agreements’ should avoid:
Mr. Palmer was also suppose to include this, from the 2004 Regulations:
in his response. Here is the section on exceptions. All links from here.
No, getting a copy of the data does not solve the problem. One copy of the data is only useful for one audit. The data and its description should be available from the keeper of the data, to avoid misinterpretation and so the ongoing data is available for research. Should the new data be available in 22 years for reexamination?
Has anyone yet asked (or submitted an FOI request I suppose) for the policies and procedures governing how they enter into confidentially agreements? (ie who can approve/sign, agreement retention policies, etc).
Also, (Steve, feel free to snip, editorializing) thefordprefect’s logic reminds me of the rubber band on an old balsa wood wind up airplane after 100 turns of the prop – twisted to the point that it’s ready violently fail just a few turns from now…
Steve: Steve Mosher made such a request. It was refused, though there is an opening for a re-specification of the request.
This was my letter:
Here is the response to which I have yet to respond:
Re: jeez (#88),
Cool. Maybe you can narrow that down to Dr. Jones’ department, as it applies to CRU data? Just a thought.
Re: Terry (#91),
Jeez and I got the same letter. We will make sure that we don’t tax them beyond there 18 hour limit. I find it odd that some bras have the same limit
Mike,
Despite your hyperventilating about the data being “commercial” products, or your strawman comparison to copyrighted works like DVDs, these are almost the only claims that CRU is not making for refusing to release the data. All the claims they are making are absolute rubbish, so it’s beyond me how you can continue to support them. If some of the data is confidential they have so far failed to show that, either from a moral or legal perspective.
As an interested non scientist,I find such behavior suspicious and stupid.
I thought science research was supposed to be about learning and sharing what we learn with others,that means publishing the research and providing the data along with it.Going to science seminars,exchanging letters,e-mails and phone calls about ongoing research and so on.
Here we have this absurd resistance (OVER YEARS TIME) to provide data for long published papers,some as far back at the late 1980’s.I consider that silly and unproductive.
What is the point of publishing science research and then resist producing the relevant data that was part of that paper? It makes no sense do that.
People may be interested to know that Dave Palmer is now on holiday.
As I cannot stop your hounds attacking CRU (e.g.jeez (#98)) I am retiring from this ridiculous situation. Enjoy you denigration.
Re: thefordprefect (#100),
Before you leave, any chance you can post a DVD Rip of HGTTG somewhere?
Or even better, the original radio program?
Thank you Steve and everyone continuing to press for freedom of basic climate information, from the prime source which should be making it publicly available and easily accessible, should never have entered into any agreements that might hamper such accessibility, and should be legally obliged to nullify any such agreements. Re: Corey S. (#84), this code of good practice seems exactly relevant, and would seem to have legal teeth.
The foolish wording makes it look as if Mr Palmer is a greenhorn the CRU are using to buy time and flush out the full range of CA energies, while they rethink what to do in an “unprecedented” climate of “robust” audit. Mark Fawcett #66 feels about correct.
Computer says no…
Ref. Jeez’s point abut CRU loosing access to climate data, now that the various WMO members know the degree of care that the CRU has taken with their data over the years who can blame them for not entrusting it to the CRU any more?
And fordprefect, that’s not Steve’s fault either. They’ll only have themselves to blame
Also, it might be a small but the the Norwegian agreement appears to specifically preclude commercial use of the data.
Presumably there’s no risk that anyone might have tried to charge anyone else for anything directly associated with or derived from their data. That would appear to be most improper.
Further, that presumably means that we can rule out any kind of commercial reason to withhold the data.
Anybody care to lay odd when fordperfect returns?
Re: steven mosher (#109),
Based on what he wrote in comment 101….
… he won’t return until Steve rescinds the “Havoc!” order and withdraws the dogs of war.
Lastly for now, but if the information is freely available elsewhere it’s hard to imagine what use the confidentiality agreement would be.
I’ve yet to see one that doesn’t exclude information that’s freely available elsewhere as the CRU claim.
Even if they had these agreements they’re unlikely to be worth the paper they’re not written on anyway.
Lucia,
Lucia, while I admire your tenacity and potential ability to get hold of the information everyone wants, I think you would agree that your suggestion that someone should obtain the data, if possible, from Peter Webster and then publish it, would be highly unethical and would place Webster in an untenable position himself. If Peter has had the privelige of being supplied with the data, even though we all believe/know that UEA are acting dishonourably and should make efforts to obtain release from agreements and make the data public, Webster is in no moral or possibly legal, position to make it available to a third party. He has just been lucky and one should recognise that, until more information becomes available as to why he was provided with the information, so jealously guarded by Phil Jones, Webster should be left to get on and use it. Perhaps he will publish an article declaring that it is all utter rubbush. In the meantime, anyone needing the data should continue to harangue Phil Jones and associated institutions by every means possible.
John Nicol
Steve: Lucia did not make that suggestion, she opposed it. It was fordprefect who said that.
Re Sunsettommy, #94:
Even though you are, in your words, a “nonscientist”, you have it exactly right, and that is really the crux of SM’s post here. In order to qualify as “science”, the data have to be PUBLICLY available (and it does make a bit of difference whether it’s a PhD climate scientist or an unbathed, scatterbrained amateur blog-reader who examines it and uses it). It’s curious and appalling to me that the climate-science Community is not itself the one MOST up-in-arms about data/methods nonposting–don’t they want to uphold the legitimacy of their profession? No one could ever get away with this in physics or chemistry.
The data is available for “commercial purposes” though “different licensing arrangements and charges will apply”. It doesn’t seem likely that the “scientist rate” will be available to anyone whose intended purpose is to audit the adjusted data available to the public.
It might be an interesting exercise for someone to apply for a commercial licensing deal to see what kind of value they place on keeping a tight hold on the data.
I seem to recall a mention that the licensing costs are intended to cover administrative costs. Given that the data obviously already exists in the format that would be requested, the price quoted for the licenses should be minimal. Anything above that cost would seem to be very difficult to justify (though I’m sure they will provide one that is sure to entertain).
In academia it is common for a scientist to not want to ever release his data. He may mine it for publications for decades (or not). Some scientists are hoarders. That is why journals & funding agencies need to enforce archiving. Here we have a case where there are only 3 real global historical datasets (NOAA, GISS, and CRU) and none of them will release the data, as if it were the “private” data of an individual academic. Furthermore, since new data come in every month, it never gets old enough that it can be released (like an academic might his old data when he is tired of it).
Jonathan,
Since you haven’t exactly been overwhelmed with responses to your post(s), I thought that I should tell you that in my view you have done a truly excellent job of framing your appeal. I look forward to hearing the outcome.
Dear Sirs,
I am neither a scientist nor an ‘academic’, however I am a tax paying British citizen who has an interest in science and appreciates that critical & open analysis of the data behind a scientists’ results/claims is key to the scientific process (especially where that science has been given such crucial global importance).
Your collective efforts to obtain this base data for independent analysis and review are to be applauded and, personally, I am very disappointed (although not surprised) at the infuriatingly obstructive attitude of those persons (who’s wages I pay by the way) to release data that I have paid for (either directly or indirectly – no-one does anything for nothing).
Sadly I’m afraid this is generally indicative of a current UK Government that is more concerned with control/PR/’spin’ than absolutely anything else. So, on behalf of the (silent) majority of the UK populace, I would like to apologise to the rest of the world for our (or at least the CRU’s) failure to release the data that is, ultimately, being used world-wide to support an agenda of control, taxation and restriction – not to mention moral panic and the mis-allocation of resources & redirection of priorities.
Unfortunately I have no practical suggestions on this particular case beyond keep trying & keep building the pressure to release this raw data so that we (i.e. the non-scientist, non-academic, tax paying lay-people of the world) can be reassured (or otherwise) that the fundamental data behind the science behind AGW behind the policies is really sound (or not).
Good look in your quest.
Regards
Ivbo
Hi Jonathan –
Can I suggest you read this?:
Click to access ico_foi_eir_how_to_comp_final.pdf
I’m not sure the UEA are doing this correctly. Your excellent request and appeal was made under the EIR regs and the link they have provided is to a 2004 Code of Practice relevant to FoIA. These are different things. I do not see where the EIR allow for “an initial informal attempt at resolution” as an internal process followed by another 28 days to have a “formal” attempt. My understanding is your appeal was a formal request under EIR and there has been no “informal” element. Is this correct or have you had discussions with them seeking solutions? If not I think you may be in a position to complain now as effectively this is an unauthorised extension to the process with no good reason.
In my opinion to simply “feel that the distance that exists between your position and that of the University” prevents supplying the information is not enough – explicit reasons need to be supplied, otherwise this is simply prevarication. I think it is worth seeking the opinion of the ICO.
Re: curious (#127), thanks for the helpful link. However I don’t see anything there which clearly defines what format the Complaints Procedure should take, simply that one should exist and that it should be followed; there is an implicit (but not explicit) requirement that the procedure should be reasonable. Appendix H of the UEA code of practice at http://www.uea.ac.uk/polopoly_fs/1.2750!uea_manual_draft_04b.pdf sets out their procedure fairly clearly and they seem to be following it. Currently I am in stage 2 of their 3 stage process, and I don’t propose to take any action beyond letting it run. If anyone has experience of these stages at UEA it would be good to hear from them!
Of course if you think the approach should be run differently you are free to make your own request and pursue it as you think best.
Hi Jonathan – any news? I hope you aren’t going to have to admit you’ve had the data for a year now?! 🙂
Well spotted. I took the 2008 date as a typo, similar to the signoff “I apologise that your request will be met”. But yes, the 28 calendar days have passed, and no I have had no response.
I will enquire again fairly soon, but haven’t decided exactly when to do so yet.
Re: Jonathan (#130), I have now emailed as follows
We shall see what transpires. Incidentally I have just discovered that Dave Palmer may well be Canadian! His first two degrees (BA MLS) are from British Columbia.
Re: Jonathan (#131), Dave has replied
I have made the obvious reply
Re: Jonathan (#132), they have refused my appeal, but have made some interesting admissions.
Any thoughts appreciated, but I will have to mull on this for a while.
Re: Jonathan (#133),
IOW, we have dumped everything into one vat, some of it is poison (but we cannot or will not demonstrate that), so the whole vat is poisoned and letting anyone have a taste of any kind would endanger public health.
You’re snookered.
Maybe shaming them again in the papers will work but I doubt it. In the end someone will have to spend some money and ask a court to order an end to the runaround. If the only “terms under which we secured the data” they can come up with are what’s been published already, a judge would see through the farce pretty fast.
Of course, there is another alternative, not cheap nor easy nor quick. Get equivalent data from open sources, QC it to hell and gone, and publish the result. That would put CRU out of business which would be an appropriate penalty.
well you’re fired
Essentially, CRU is a data sink that believes its work requires no V&V.
The public interest shall always be our first priority.
Except in those situations where the public interest is not in perfect alignment with our own.
Jonathan – thanks for the update. IMO shameful response that (further) destroys the credibility of the UEA and shows a disregard for the intended purpose of FOI and EIR legislation.
Re: curious (#140), I will of course be appealing this all the way. However Mr Colam-French is distinctly cleverer than the somewhat hapless Dave Palmer, and I’m going to need some help with this. For newcomers please note that winning an FOI case is essentially a technical matter, which must rely on identifying technical breaches by UEA and not on appeals to science or common decency or things of that kind.
Readers may have noted that although UEA has conceded some minor points the essence of their claim remains the same: their claim that the data is simultaneously publicly available and confidential. My understanding is that although a tribunal is likely to be unsympathetic to this approach I will nonetheless have to defeat both claims.
Claim 1: the information is available elsewhere
UEA has accepted that the gridded data that CRU publish is not equivalent to the data I requested, but are still claiming that it is available from GHCN. Previously Steve McIntyre (#64) has claimed that the two data sets are materially different, and any details of this would be extremely helpful. I will of course be making the various obvious points about this claim being in contradiction to their other statements, but clearly identifying an actual difference would help.
Claim 2: the data is held subject to non-disclosure agreements
I will of course be making the obvious points about the applicability of non-disclosure agreements to data which is allegedly publicly available, but I will mostly be using this to attack claim 1, and my principal attack on claim 2 will be based on their inability to produce any such agreements.
Their decision to declare the earlier release of the data to Peter Webster as improper is clever, as I think it would be very difficult for them to justify releasing data to him but not to me. However it is not without its own weaknesses. Steve McIntyre (#80) has claimed that the data has also been released to Oak Ridge, and more details of this would be helpful, as would details of any other transfers.
I think the CRU refusal was very well worded and polite. The weakest part of their argument is the effort (which they are apparently not paid to do) to sort out which sources are protected and which are not. It cannot really be that difficult.
Anyway, Steve, have you had enough fun yet with trying to winkle out the data from CRU? You appear to have had access to most of the data back in late July – did you really not file away a copy for a rainy day such as this? Or is there some long game that I don’t understand?
Rich.
CO2 Rich – “well worded and polite” is often the format used as a facade for an inadequate response constructed to frustrate due process – just to clarify your position: do you think that the CRU raw data set should be publicly available? I agree it is weak to claim an unreasonable effort is required to determine the “confidential” elements (in a properly organised set up this would simply be a case of opening the “file” and sorting by tag “confidential”) but, IMO, the fundamental weakness is their argument that it is not in the public interest that this raw data should be available in case it should prejudice the flow of future information. If scrutiny at this stage could improve the quality of the future flow of information, as well as the current information, I would think that would be very much in the public interest whereas a flow of unverified and potentially incorrect information is not.
Jonathan – IMO you have grounds for appeal as they failed to meet their response deadlines. However, though noteworthy, an appeal upheld on this basis is unlikely to progress release of the data. I am not sure what to make of this statement:
Quite why the UEA are contacting the IC on your behalf escapes me but perhaps I’ve misunderstood. They may have included this with the intent to advise that you have a right to appeal to the IC; yet that was already known. I think it is more likely they are saying there is no point in any further correspondence with them.
As noted above to CO2 Rich I think the fundamental issue is that it is in the public interest the data should be available and, on my reading, this is the primary consideration in the EIRs. I also think the claim that the resource required to identify the confidential elements is “unreasonable”:
should be challenged. If they are mandated to provide “information services” then having proper data management and quality control procedures is entirely reasonable and IMO, in fact, required. As a supporting argument to quantify the likely resource required, I think the work Steven Mosher and others did on FOI requests for the Confidentiality Agreements applicable to the dataset is relevant. I didn’t follow it blow by blow but I formed the impression the “confidentiality” requirement was only applicable to a handful of data providers. If this is the case then identifying and separating the relevant stations should be straightforwards and I’d estimate a reasonably competent person starting from scratch with the relevant station IDs could do this in a half to a full day’s effort? (I may be off beam here – other views appreciated).
I agree with your analysis of the other two issues and I’ll refresh my memory on the regs. and let you have any additional thoughts. I think the Georgia tech transmission is of little relevance other than to illustrate the (admitted) laxity of their data management. If you have the resource I think it would be well worth getting expert opinion on this – my reading of the EIR regs. is only as a lay person. Possible starting point for relevant expertise here:
http://www.pdpjournals.com/freedom_of_information/editorial_board/
Re: curious (#143),
I think this is fairly straightforward: their complaints procedure is a three-step one, but they are aborting it at the end of stage two, and I presume they have obtained an opinion that this not in itself a breach of their procedure.
Re: Jonathan (#144), I have finally sent my appeal to the Information Commissioner. Their web form is moderately helpful, but doesn’t like URLs or ampersands. Anyway my appeal is as follows:
We’ll see how it goes!
Re: Jonathan (Nov 24 16:14), in the light of recent developments at UEA I thought it might be interesting to reopen discussions with Mr Colam-French:
We’ll see what happens!
Re: curious (#143), you asked me to clarify. Well, I’m pretty much in agreement with you. CRU must sort out its confidentiality agreements, and since they clearly don’t think the data should in general be confidential 😉 they should ask each supplier to waive such. If any one supplier is a defence establishment with whom there would be great surprise/concern that there was a liaison, then they might be able to claim “national security” for that one. (What, we receive “hot” climate data from the Taleban?!)
Regarding their approach to the information commissioner, we have seen one explanation. Another might be that they want to get their word in first, and warn him what might be coming his way…
Rich.
Re: See – owe to Rich (#146), we already know about CRU’s supoosed confidentiality agreements, as a full list can be found at http://www.cru.uea.ac.uk/cru/data/availability/ and none of them are particularly impressive.
The switch from “confidentiality” to non-disclosure agreements is superficially clever, as it lets them move from a completely indefensible position, but since they don’t actually have any NDAs it doesn’t gain them much.
1. Has anyone gotten a copy of their “terms”?
2. If the earlier release of the data to Peter Webster was improper, what have they done to get the information back? IIRC, CRU asked Steve to delete his copy of the information, so have they insisted that Webster delete his?
The fact that they allow Webster to hold on to this “confidential” data says that they’re not TOO concerned about whether this data should be released. They’ve proven thay can’t be trusted to keep the data secret.
The fact that Webster may actually produce a paper using this secret data makes the future paper un-reviewable (any peer review couldn’t ask to see the data, any journal couldn’t force data to be archived, etc.)
Comprehensive and convincing. Good luck – we’ll see now if FOI and EIR really are there to serve the “public interest”. C
As I never got an acknowledgement of my appeal I gave them a call, and they said they had no record of it. (I suspect incompetence rather than foul play). I have, therefore, sent them a paper copy.
I have now received acknowledgments of both copies of my appeal (paper and web) and an apology that their web based system had problems.
I sent in a second FOI request last month
I didn’t expect much joy from this, but was interested to see what their grounds for refusal would be. As expected I have just been refused, with an interesting range of grounds:
Thanks for the update. Interesting to see that the ICO already know the outcome of the “ongoing investigation” they are so keen not to prejudice:
…”due to the illegal penetration and use of University computing facilities”…
I’ve not been following that closely recently but I don’t recall seeing this being confirmed beyond supposition.
Note that this is from UEA not the ICO; I don’t think it is unreasonable of them to treat the “position” as “true” in this case.
Noted – my mistake. I guess this would count as an official position from UEA. It will be interesting to see if the investigation confirms the “illegal penetration and use of University computing facilities”.
I’ve been looking at these, but waiting for a review.
Following this up I asked Mr Palmer
Somewhat to my surprise UEA has decided to treat this as a formal FOI request.
I await the result with bated breath!
Re: Jonathan (Jan 10 07:37), I did eventually get a reply on 18th January. UEA very helpfully remind me that they hold copyright on the letter in question and I should not reproduce it without their permission, so if you want to see the whole thing I suggest you email foi@uea.ac.uk asking for “a copy of the decision notice issued by Edward Acton on 18th December 2009 concerning the application of section 36(2) of the FOI act to correspondence between Prof. Phil Jones and others”. In summary it pretty much claims a blanket exemption on all correspondence, including specifically correspondence concerning the handling of FOI requests.
Steve: Could you please post up the exact language of their copyright notice.
Re: Jonathan (Jan 23 08:47), Steve the relevant paragraph is
This would appear to be boilerplate rather than hand crafted for the occasion. From my own reading of the relevant acts they are entitled to make this claim, although I consider it very foolish of them to do so as any individual can obtain their own copy as indicated above. One might ask for a waiver of copyright, or request them to post a copy at their own website, to avoid a flood of such requests, and I am happy to do so if you wish.
Steve: Just because material is copyrighted doesn’t prevent you from providing relevant excerpts from the letter. You are not permitted to use the letter for commercial purposes. If you want to post up the salient paragraphs of the refusal here, you are welcome to do so. I’ve posted up letters containing similar language.
Re: Jonathan (Jan 23 13:31), Steve UK copyright law is considerably more restrictive than you might think, and as the letter is very brief even short extracts would make up a substantial portion of the whole. Since I have an appeal in with the ICO I am also keen to play this by the letter of the rules, even (especially?) when they are silly. Obviously if you obtain your own copy and post it that’s your business.
Re: Jonathan (Jan 23 13:31), I have now emailed back
I agree with Steve here. UK copyright law also allows fair use which would include making quotes. I think in order to make a breach of copyright claim stick, you have to show commercial loss.
Jonathan
It strikes me that this is a pretty outrageous step by UEA when they are already under investigation by the Information Commissioner. I was speaking to someone from the ICO’s investigations team this morning, who might be interested in hearing about UEA’s position on copyright.
Would you like me to call him? Drop me a line if you do.
Bishop Hill: Agreed – Anyway doesn’t copyright apply to “creative works”? Are they claiming the VC’s opinion is of the status of a “creative work” or is it based on law, precedent or policy?
I would also think that FOI would qualify as an allowable act under the “statutory enquiries” exemption of para 8 item 6 here:
http://www.copyrightservice.co.uk/copyright/p01_uk_copyright_law
If it doesn’t – what value FOI?
Re: curious (Jan 25 08:27), thanks for this. While Bishop Hill above is entirely right in practice (there is no way they would ever sue me, or that they would win if they tried), I have been seeking to stay rigorously within the (remarkably restrictive) letter of the law. However I find your argument, in essence that the VC is here acting in a semi-judicial role rather than a creative role, fairly convincing. I am also much encouraged by the recently reported good decisions the ICO has been making. So here is a transcript of the core of the VC’s statement:
If anyone would like to take any of this up with the ICO that’s obviously fine by me.
This looks a good course offered by the Campaign for Freedom of Information in London on February the 4th:
Click to access foicoursefeb2010.pdf
http://www.cfoi.org.uk/
7 Trackbacks
[…] LINK AKPC_IDS += “2871,”;Popularity: unranked [?] (No Ratings Yet) Loading … […]
[…] http://www.climateaudit.org/?p=6825 […]
[…] makes up the gridded Hadley Climate Research Unit HadCRUT dataset has been fraught with delays, FOI denials, and obvious […]
[…] The CRU has been less than forthcoming with these processes in the past, citing everything from lost data to political sensitivity to prevent other scientits from getting data and other information. When one scientist was finally […]
[…] to use that as an excuse to deny FOI requests for data and code, rather than lessons, is ‘misleading’ to put it […]
[…] got from Jones, CRU started again with a series of denials again citing confidentiality agreements, inventing the terms of those agreements ex nihilo. Webster could have the data. McIntyre could […]
[…] document” is online at CRU here. It was reported at CA on August 11, 2009 here, here and here (among […]