CA readers are well aware that Phil Jones of CRU has jealously refused to provide the surface temperature data set used in the prominent HadCRU temperature index, going so far as to repudiate Freedom of Information requests. Efforts to obtain this data have been chronicled here from time to time. As a result of the previous round of FOI inquiries, we managed to get a (mostly) complete list of station names (but not data) – even the names being refused in the first round of polite requests and subsequent FOI inquiries.
There seems to be a gradual movement away from CRU to the Hadley Center in the ongoing management of this data set, with the Hadley Center seemingly having a much more professional approach to data management, recognizing obligations to the public as well as to their pals. I noticed an interesting statement at a Hadley Center readme today
Q. Where can I get the raw observations?
A. The raw sea-surface temperature observations used to create HadSST2 are taken from ICOADS (International Comprehensive Ocean Atmosphere Data Set). These can be found at http://icoads.noaa.gov/. To obtain the archive of raw land surface temperature observations used to create CRUTEM3, you will need to contact Phil Jones at the Climate Research Unit at the University of East Anglia. Recently archived station reports used to update CRUTEM3 and HadCRUT3 are available from the CRUTEM3 data download page.
Given the above statement is meant to indicate that Jones gives the archive to some people, then his refusal to give the archive to others becomes hard to justify even in “community” terms. If Jones has consistently refused to provide the archive to anyone, this seems to be a circular paper chase.
In any event, I wrote to Dr Kennedy, the maintainer of the webpage, referring to this paragraph and asking him for a copy of the archive in the form that the Hadley Center possessed it. I did not cite the Freedom of Information Act, but will initiate a new round of requests if required.
Climate Audit 
37 Comments
Steve, good luck on this effort. I admire your persistence.
The Hadley quote clearly indicates that they realize that Jones is acting unresponsively (“you will have to contact him”, poor boy). Well, their approach is also a compromise between the service to the public and science on one side – and the service to the “pals” they are going to supersede, such as data dinosaur Jones.
The last sentence in the quote is interesting. It does link to a real page with real data acquired since 2000 on their watch.
John Kennedy responded as follows:
This sort of runaround is so pathetic. Hadley Center are the ones that should be demanding that Jones make the data available.
In any event, I just sent a FOI request to Hadley Center for the archive in the form that they possess it. Once Jones delivered a copy of the data to Hadley Center, I’m not sure that he can thereafter prevent the Hadley Center from complying with a FOI request. We’ll see. There are always interesting legal issues arising from Team stonewalling.
Data Wars IV, A new hope.
Stevie Wan in search of the missing tapes, travels to the ends of the galaxy in the hope of uncovering the data used to power the ultimate political weapon – warm water.
The Imperials in a burst of brilliant strategy, employ the time proven strategy of the shell game cleverly moving the political power source to another encampment.
Unfazed, master Wan follows the trail.
—-
Steve, The NSIDC sent notice this morning that the 2008 gridded sea ice data was updated.
Hey! Steve, leave them kids alone! They don’t need no data-cation! 😉
More seriously, good luck. I hope that you can finally get that data released.
The police have the thin blue line. Apparently Team has the (data) thin warm line.
Why would anybody give any credence to their data if it’s all compiled by one person without any transparency at all? Despite all the problems with GISS over the years at least they have become more open about where the data comes from and what they do with it once they get it. Critics may not agree with some of the data manipulation, but at least we know what they are doing for the most part. GISS is a model of openness compared to HadCRU.
What is the process for filing an official complaint for noncompliance with FOI? What possible legal sanctions can this fellow suffer for his refusals?
I would suggest that this guy needs to be made an example of.
Re: Mike Lorrey (#9),
Under UK Freedom of Information Act they have up to 20 working days to respond to your request. Your request should be as specific as possible to avoid the authority from having to come back to “clarify” your request (in which case the 20 days starts again). You can also make a request for how you would like the data to be formatted which should be complied with provided it is reasonable.
The 20 day time limit can, however, be extended if they need to consider if they believe it is against the public interest. But this data should be classed as environmental information – which means that there is no public interest extension allowed.
If the authority does not respond then you can make a complaint to the information officer at the authority and ask for an internal review.
If they refuse to provide with the information then they must tell you why they have refused – you can then seek an internal review into this decision.
All internal reviews should take about 20 working days but you should be told if they are going to take longer and should not take more than 40 working days in total. You should also be told their findings.
If you still think that they should release the information then your final resport is to complain to the Information Commissioner. Most Information Commissioner conmplaints come down on the side of the requestor; however there is a big backlog of complaints at the moment so it can take years to go down this route!
On the Hadley station data update page it says
So there is partial data for the last 9 years.
It might be useful to summarise what info is available and where?
The above partial data is at
http://hadobs.metoffice.com/crutem3/data/station_updates/
There is also the list of stations “used at some time” at
http://www.cru.uea.ac.uk/cru/data/landstations/
It seems that the station ID’s in these two lists don’t always match up very well.
Perhaps this is because “Some WMO IDs have been updated in the 2000s.”
But never mind. It’s only the data set on which the IPCC report is based.
Re: PaulM (#11),
Maybe I’m ignorant, but shouldn’t ANY journal with standards of peer review automatically reject any paper based on data that is not permitted to be redistributed? This seems to me like basic ethics and defense of basic scientific principles.
Steve,
Perhaps you could make use of Jeff ID’s idea and accuse HadCru of torture since the have failed to post these temperatures on a board somewhere. If Jeff is correct that warm water is the ultimate weapon, they’ve clearly been engaging in waterboarding. Of course the current slow drip, drip, drip of data release is already a form of the old “Chinese water torture”.
I predict that Phil Jones vs. the scientific method is a bout that will go many rounds. I don’t think it will end until there’s a knockout. Even then, if Jones loses, I expect him to make a number of attempts at a comeback. I also expect the hockey team members to leap into the ring to help him battle. If Jones is forced to do science, others may have to follow.
Just imagine if the FDA was removing all antibiotics from the market but got the data from an individual who said they couldn’t release it.
There is a problem with “THE” data because the exact list of stations fed into the computation depends on which ones have been updated and which ones have gaps in reporting (or lags). Every time they compute a global temperature trend, I bet the inputs differ.
When I read and hear stuff like this (refusing data), I get a tad suspicious…
I can’t understand how simply knowing which stations were used can fall into a “Most of the station data was given to us under conditions that don’t allow us to redistribute it” phrase.
If we knew which stations were used we could ask the station managers directly, instead of going through a third-party (one who has his own interests at heart).
Steve,
I also wish you the best. At the moment I have complaints in with the Information Commissioner (ICO) against the MOD/Met Office, UEA/CRU, Reading and Oxford Universities and Ofcom.
Only Defra accepted that information on Climate Change and the IPCC assessment is environmental information covered by the British Environmental Information Regulations 2004, but sadly they had none worth bothering with. The key which I commend to all CA readers is the Aarhus Convention which is European Community Law from which the EIR descend and guarantees everyone in the world “Access to Information, Public Participation in Decision-making and Access to Justice in Environmental Matters”.
Switzerland is a signatory but has not ratified the Convention. The University of Bern is the AR5 WGI TSU. We should all focus on trying to persuade the WGI TSU and the IPCC to adopt the Aarhus principles, first by those of us in Europe leaning on our governments to honour Aarhus Article 3(5) which is;
Each Party shall promote the application of the principles of this Convention in international environmental decision-making processes and within the framework of international organizations in matters relating to the environment.
Aarhus article 5(1) states
Each Party shall ensure that:
(a) Public authorities possess and update environmental information which is relevant to their functions;
(b) Mandatory systems are established so that there is an adequate flow of information to public authorities about proposed and existing activities which may significantly affect the environment;
Aarhus article 5(3) states
Each Party shall ensure that environmental information progressively becomes available in electronic databases which are easily accessible to the public through public telecommunications networks.
But don’t hold your breath.
Mostly I’ve gotten stonewalled, so I’m never optimistic. But there are occasional successes – for example, the IPCC AR4 Review Comments are online because of initiatives taken here e.g. FOI requests from myself and CA readers in multiple jurisdictions.
Given the lengths they have gone to prevent their data from being looked at by others, you would almost think they know there are issues and they are trying to cover up their errors.
Or they just don’t like us uppity colonials 🙂
Mike Lorrie, you make way too much sense! You should also understand that Phil Jones is already an example here, as is Lonnie Thompson, Santer and numerous other members of “the team”.
David Holland makes a good suggestion, but what is needed is an organization, perhaps a group of students of international law, or environmental law to pursue this issue of disclosure. Steve and the odd blogger writing an FOI request is insufficient. We know that from experience.
Any law students or lawyers out there who believe that data relied or intended to be relied upon by a government and that supports (or refutes) the theory that earth is in imminent danger of anthropogenic global warming should be made fully available to all of us?? You just might be able to make a difference!
Those who take and spend public money are responsible to the public. There may be a battle or two ahead but this war can only end one way.
Keep the pressure on, Steve. UK readers should also complain to relevant Ministers and Opposition spokespeople.
CA, May 9, 2007. Extract of email from P Jones to self, March 26, 2006.
What hope FOI?
For interest, from http://www.uea.ac.uk/about/legalstatements
This might tell you the fate of your earlier FOI requests.
Finally, if you do gat to make a FOI request, the name should be right. Is P.D. Jones, commonly shortened to “Phil”, really Phil, or as is sometimes stated “Philip” or “Phillip” or “Phileas” or “Philacre”? The shyness to give a full name is a bit evident on the net. Why, you might even need to fill in the “D.” on official documents before you make progress.
Re: Geoff Sherrington (#23),
The forms are simply to help the Freedom of Information officer. To make a Freedom of Information request in the UK you simply need to ask for the information – you don’t even need to state that you are asking for it under the Freedom of Information Act (although this may help!). In fact, as this data falls under the Environment Information Regulations you don’t even need to put the request into writing – it could be made by telephone for example.
And with respect to how specific you should be it is the organisation’s responsibility to assist anyone making a request. Where a request is unclear the organisation must seek clarification within 20 working days and assist the requestor in clarifying the request or making it more specific.
If the request is to be refused then it may only be done so if it is covered by a specific or general exemption and, importantly, is not in the public interest to disclose the information. The Information Commissioner describes public interest as including, “furthering public understanding and debate on a key policy proposal” – which should cover climate change data.
It is important to note that, under the environment information regulations, you need to appeal any refusal to provide the data within 40 working days of the date of the refusal letter.
Is there not a sympathetic MP who could shake this tree.
Surely this is an issue of public relevance. I assume this data set was compiled by the CRU and Jones in his capacity at a publicly funded organisation.
There is leverage that could be applied.
Maybe Lord Lawson??
My understanding is that if a request for data under the FOI act is not responded to within 20 days it becomes a legal matter, unless its a matter of national security. Phill Jones refusal to provide the data is digusting and as a UK tax payer I feel ashamed that tax payer funded research is being horded away in this manner. When people are relucted to share there data I cant help but feel they are hiding something. Good luck!
Tim Berners-Lee’s recent TED talk was about sharing data. He got the crowd yelling “Raw Data Now!”, and said “you wouldn’t believe the excuses people give” for not releasing data. Indeed.
#24 In the past, with the minimum or bureaucracy, I have walked into Scottish Environmental Protection Agency offices and been given free and unaccompanied access to their environmental correspondence files.
Re: Aylamp (#28),
If you walked into my office with bagpipes, a dirk and a kilt, I’d give you my environmental files meekly too.
Sorry to be dumb, but sort of on this topic, why is it that all the MSU stats I see only go to -70 degrees latitude. What am I missing? They go to 85 degrees in the northern hemisphere but leave out Antartica completely. Maybe I’m looking at the wrong stuff. Given the ice increases there, if this area were included it seems like it would make a difference on the overall totals.
Re: pz (#30),
Most of Antarctica south of 70 degrees is a high ice plateau (c. 10,000 feet) which makes “lower troposphere temperatures” rather meaningless there.
Re: pz (#30), As I understand it, the angle of the satellite precludes good coverage of the poles, and may have some issues with pure ice at ground level. Anyone else?
Thank you so much. That was driving me crazy. The region actually physically inhabits the lower troposphere due to its altitude. Unless I’m wrong, which happens ten times a day, at the least, that still demonstrates a break in the MSU logic by not being able to gather data on an area which, if by quick observational standards, appears to be cooling, and could therefore represent a significan shift in the numbers. Still better, I suppose, than official weather stations gathering data from asphalt parking lots right next to air conditioners.
[snip – I know you’re a new reader. I discourage “piling on” editorializing in comments: Steve]
Perhaps we can work some compensatory data based on other evidence, though I understand that trying to derive connections between land-based measurements and MSU stats is essentially impossible.
By the way, I’m a new reader, so please forgive me if I’m covering ancient history here.
If the source data is not open to review and the method of data analysis is not public, this is by definition not science isn’t it? How does something like this survive any peer review process? How can anyone take results from such a process seriously?
Sorry for the piling on editorializing. I’m not sure if that was something you cut out or something that is still there, like the air conditioning statement, but I’ll try to stick to facts and questions from here.
I would still like to know, or just be directed by link if it is an old topic that doesn’t need to be rehashed, how chopping off 20 degrees of an apparently cooling part of the planet is handled statistically and why it isn’t significant. I have read at NASA’s site that ground-based stations are sparse in Antarctica, too.
Thanks for your patience and sorry for arriving late at the party. I have searched and just can’t seem to find this answer anywhere.
Re: pz (#36), It surely affects the absolute answer, but the satellite data has more complete coverage than any other data, and thus is useful.
6 Trackbacks
[…] attempted to acquire the computational code from Michael Mann and data from Keith Briffa and Phil Jones that were used to build the hockey stick graphs. Like the US, the UK has Freedom of Information […]
[…] Bern in Switzerland, to measures Holland suggests for the next IPCC report. Jones alerts Stocker to specific comments on Climate Audit, arming him for the next […]
[…] prøvet på at erhverve den beregningsmæssige kode fra Michael Mann og data fra Keith Briffa og Phil Jones, som var anvendt i konstruktionen af hockeystavs-graferne. Ligesom i USA, har England en […]
[…] couple of week ago, I noticed that the UK Met Office website contained the following statement Q. Where can I get the raw […]
[…] arose out of the very first post in the current FOI sequence. On May 11, 2009, as mentioned in a CA post here, I had noticed that the UK Met Office webpage on CRUTEM stated: To obtain the archive of raw land […]
[…] May 11, 2009, I reported my request for CRUTEM station data from the Met Office. In a comment, David Holland noted that the […]