Deepening moats has been a lively topic in UK politics recently. One British MP expensed the public for the cost of deepening the moat at his castle and has been forced to resign. We discussed FOI and the MP expense scandal recently.
We have, of course, followed with some considerable amusement the contortions of the UK Met Office to avoid disclosing data. Last year, we reported how John Mitchell, Chief Scientist at the UK Met Office, obstructed compliance with an FOI request for his IPCC review comments by wrongfully claiming that his IPCC correspondence had been destroyed and then that it was his “personal” property, resiling from these absurd claims only when asked whether the Met Office had paid his salary and expenses for trips to IPCC destinations.
Recently, we’ve followed the amusing contortions at the Hadley Center webpage as they are conflicted between their use of Phil Jones’ land station data (CRUTEM3) and Phil Jones’ absurd campaign to keep CRU station data secret.
The Met Office webpage for downloading CRUTEM3 data http://hadobs.metoffice.com/crutem3/data/download.html presently says:
Station data: Most of the station data was given to us under conditions that don’t allow us to redistribute it; but the CLIMAT reports we use to update the data in near-real time have no such restriction. Summaries of these reports are available on this page.
Although many Met Office pages can be retrieved from the Wayback archive, this particular page has a robots.txt block and is not available at the Wayback archive. I wasn’t able to locate a google cache (though perhaps someone else can.)
The statement here raises an interesting question:
1) who is it that it is attaching conditions to the station data and, by what authority are they doing so?
2) If it’s Phil Jones, who is also subject to the UK Environmental Information Regulations, does he have the authority to attach conditions to the Met Office use of this data?
3) If the Met Office can’t show their underlying data, maybe they should discontinue the use of Phil Jones’ data and use data that they can show
4) Maybe whoever is funding Phil Jones (and I believe that the US Department of Energy is one of his funders) should require him to deliver the data back to them.
Update: The following sent to CRU on June 25, 2009:
Dear Mr Palmer,
Pursuant to the Environmental Information Regulations, I hereby request a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009.
Thank you for your attention,
Stephen McIntyre
Climate Audit 
55 Comments
This is most disconcerting. Here we have a public service department using data that was originally collected by other public service agencies (including themselves) but cannot publish said data because a third party, a university official claims intellectual property on said publicly collected data purely because they applied some algorithms to it. You should be at least entitled to the raw unadjusted data, as it is (or should be) in the public domain.
The adjusted data are also not intellectual property. The intellectual property, if any, is in the algorithms themselves.
One can be forgiven for thinking Jones has something to hide.
Steve: This webpage didn’t mention Jones. That’s my speculation. Personally I think that Jones’ main concern is economic – my surmise is that they’ve been charging various agencies a lot of money to make this data set, but, like GISS, have been doing virtually nothing in the way of quality of control. Recall Gavin’s 0.25 man-years per year. I’ll bet the program pretty much operates itself and they don’t want to expose how little work or due diligence that they do. I doubt that the data itself is explosive.
An interesting quote from the UK Intellectual Property Office:
“There is no registration for database right – it is an automatic right like copyright and commences as soon as the material that can be protected exists in a recorded form. However, the term of protection under database right is much shorter than under copyright. Database right lasts for 15 years from making but, if published during this time, then the term is 15 years from publication.”
http://www.ipo.gov.uk/types/copy/c-otherprotect/c-databaseright.htm
I believe that Jones may be trying to claim copyright to the database of the CRUTEM3 records. If so, he does have such right. However, only for the periods stated above. B
But I am no lawyer.
Steve: There’s no need speculating on this sort of thing. To date, he’s not asserted this particular legislation. Let’s stick to what they’ve said on the record.
I thought the MP was expensing cleaning the moat, not deepening it.
Re: Jeff Alberts (#3),
It is cleaning, not deepening. From the linked article:
A minor detail, but deepening would be considerably more expensive than cleaning, one would think. It’s just as easy to report the facts accurately.
snip – I ask readers to avoid making angry posts
I noticed yesterday that both BBC and ITV news channels were broadcasting highly alarming reports on future climate change, citing increased temperatures, reduced rain, more storms (?) etc. etc.
I suspect this a panic push from the UK government who are about to do something REALLY STUPID. If they don’t get their tax gobbling legislation in place before this winter they will not get the support they need. Only two days to solstice and still a cold summer here in the UK.
Any chance Mauna Loa’s implied acceptance of license can be used?
I’ve sent an FoI request to the Met Office to ask for a list of people who have placed restrictions on data redistribution.
Yesterday the Guardian, a left-wing UK paper, ran a question/answer session with Malcolm Fergusson of the UK Environment Agency covering “What the recently-released Climate Predictions mean to you”. I asked why the base data was not available to researchers although it had been initially advertised as available, and was fobbed off with the comment that ‘We are the Environment Agency – you need to ask the Met Office’. Still, every little helps….
I know this thread isn’t really about MP’s expenses but saying the MP resigned implies he did the honorable thing. He did not.
If an MP resigns from office before the end of Parliament they lose part of the overly generous payoffs and pensions they receive. What the MP has done is decide to not stand for re-election to the next Parliament. He is still an MP, claiming expenses, and waiting for his payoff at the tax payers expense.
snip
Frankly I don’t hold out much hope for UK FoI requests that ask for information that the authorities don’t want released. Laughably, the FoI response to a request for MPs expenses has just been released weeks after the information was already leaked. It is heavily censored and contains far less information than is aleady in the public domain. Notably, under a privacy and safety clause, the locations of MPs homes is witheld. This means for instance that it would not have been revealed that the MP for Luton was claiming for a home in Southampton, about 80 miles in the opposite direction from London, nor that a married couple were claiming for separate second homes while living together.
RichardIII – behind what you are referring to is the UK Climate Projections 2009 report (UKCP09) that has just been published. I have not seen it, but it appears to be an excuse for the media to pump out the same scare stories yet again. On ITN news last night they quoted sea level rises up to 36cm and mean temps up 2-6C by 2080 in the UK. On the radio this morning, a Met Office spokesman said “… global temperatures ARE increasing…” Perhaps he needs help interpreting his own graphs?
Just wondering if it is worth doing an FOI for the email from Jones putting these restrictions on the data.
If this email is a latter-day excuse, then presumably there might be years of data which predate the excuse and are not covered…
Re: per (#11),
Theoretically that could go on forever.
1. FOI for data – Refused due to NDA
2. FOI for NDA covering data – Refused due to NDA covering NDA
3. FOI for NDA covering NDA covering data – Refused due to…..
etc
One interesting EIR request of the MOD/Met Office might be for the earlier versions of the web page that the Met Office changed after Steve made a request for data. It is certainly information on climate change and any such information is covered by the EIR. Who changed it and why is probably also environmental information.
As for non disclosure agreements, there is no absolute exception to disclosure in the Regulations, it is subject to the public interest test and the presumption of disclosure. However that almost invariably means they will refuse and make you appeal to the ICO.
For the avoidance of doubt, is Phil Jones’ HADCRUT code public yet? Has anyone requested it?
“You’ve never had it so good” – Harold McMillan.
You’ve never had it so warm – Hilary Benn.
Interview with Environment Secretary Hilary Benn yesterday (Thursday) on BBC’s “Today”, “Radio 4’s flagship news and current affairs programme” here (+2,40 minutes).
Earlier interview with Mark Lynas on the same radio programme here.
“The projected impacts of climate change are occasionally exaggerated” – Mark Lynas. LOL
Re: Phil (#15), As a major exxagerator himself (“Six Degrees”-good tagline but totally implausible IMAO) it really is quite an amusing admission.
The Googlebot obeys robots.txt. You won’t find it in the Google cache either.
Every so often I learn new words by reading CA, and today it is “resiling”. I didn’t even know there was a verb form of “resilience”. So ignorant was I, I at first thought that “resiling” was a typo so grotesque as to be indecipherable. SM, thanks. This site is A-OK, if you ask me.
“resiling” is well known in the USA midwest. That is, if you work around a grain or corn silo.
#20
and I thought it came from the french verb resilier, to terminate, annul etc.
The comment thread on the latest post by Richard Black of the BBC is touching on CRU data too.
Re: Bishop Hill (#22),
The discussion has become a little one-sided:
Re: Bishop Hill (#22),
what had you posted that is now in the censor queue?
Re: Steve McIntyre (#26),
Steve
I merely said that the CRU raw data was not available and quoted Phil Jones’ reply to Warwick Hughes. I linked to your pdf of CRU correspondence. There was also a bit about the public interest in having the data available.
It was rant-free and I didn’t rise to the baiting from the previous poster.
So I have no idea why it didn’t go through.
Re: Moat cleaning/deepening.
Moats notoriously silt up with a variety of debris. Therefore, mucking them out both cleans and deepens them. Mine does. Doesn’t everyone’s?
Re: Fred Harwood (#23),
Only after repelling raiding parties from the neighbouring Barons. And then I usually get the prisoners to take all the dead bodies out and away with them…
An interesting development to the expenses debacle: some of the MPs will be investigated for more serious claims including phantom mortgages. I doubt, sadly, that police inquiries extend to the likes of the UK Met Office on matters of FOI.
Interesting exchange. “Yeah Whatever” is apparently very unfamiliar with the issues so many have faced when it comes to Phil Jones and CRU.
13. At 7:09pm on 19 Jun 2009, BishopHill wrote:
Yeah Whatever
The raw data is what was requested. You have provided a link to processed CRU data. The CRU refuses to release the raw data and the code they have used to process it through to the outputs you link to…
15. At 7:24pm on 19 Jun 2009, yeah_whatever wrote:
…The raw data is available.
You can get it.
Except you don’t want to, because you then have to either admit you don’t want to know in case you’re wrong or are too lazy…
John Mitchell’s replacement as Met Office Chief Scientist, Julia Slingo, seems to have some rather flexible views about the integrity of models, and perhaps on the release of information relating to methodology as well:
http://www.harmlesssky.org
There may be a rather public row brewing; see third para from the end.
Re: TonyN (#28),
I haven’t been able to trace anything else about the Met Office ‘postponing’ publication of the methodology used in the UK Climate Predictions 2009 report. I’d be interested to hear from anyone knows more about this.
The Newsnight video will be availlable here for a few days:
http://www.bbc.co.uk/iplayer/newsnight
Click on the 18th June programme below the window and the item on the Met Office report starts about 32 mins in with a fairly pedestrian report by Susan Watts. Myles Allan’s outburst is far more interesting, and the tone of Kirsty Wark’s questions to the minister even more so.
Richard111:
June 18th, 2009 at 11:48 pm says,
I noticed yesterday that both BBC and ITV news channels were broadcasting highly alarming reports on future climate change.
Panic, my thoughts also.
ITV are now to be funded by part of the BBC`s licence to enable them to survive, they will soon also be using BBC`s news reporting for all local news and probably for some of the main news items. ITV will no longer be independent.
When I read that these organizations avoid disclosing data, I get more distrustful of them and of their work. And I don’t see these actions winning the minds and hearts of the general public. Don’t they consider these things?
Most posters have missed the truly extraordinary thing about this. The Met Office is now forecasting that (from the Times)
“One in six homes in England is at risk of flooding and some £20 billion is needed by 2035 for defences to protect properties from rising sea levels and extreme rainfall, the Environment Agency warns today.
“Spending on flood defences must rise from £570 million a year to £1 billion a year — and that will protect only 490,000 homes that are at the highest risk of flooding.
The threat to entire communities is particularly alarming as vital installations for electricity, water and gas are at significant risk of flooding, which could switch off taps and leave thousands of homes and buildings without light or heating, the agency warns.
“Thousands of schools, nurseries, hospitals and GP surgeries are also at very high risk of being swamped by floods — but rescue operations could be thwarted because of the high number of emergency services such as police, fire and ambulance that are also at risk from flooding.
Related Links
“Some 2,500km (1,550 miles) of railway, 4,000km of roads and scores of premises earmarked for use as flood evacuation centres are also threatened as are hundreds of seaside camping and caravan parks and prisons.”
What is being forecast is a genuine national catastrophe which is to occur in the next 25 years, threatening life and property fully on the scale of WWII, perhaps greater. And yet, incomprehensibly, the authorities (University of East Anglian, Met Office, Environment Agency, whoever…) will not make available one of the key data sets which would allow independent scrutiny of these forecasts.
Is there any other area of public or corporate life where money would be demanded, and disaster forecast on this scale, while refusing to allow any independent examination of the data to justify the forecasts and the expense?
Decades of underspending on coastal defences in the UK combined with decades of irresponsible building on flood plains have combined to magnify the costs of flooding. AGW provides the perfect excuse for these abysmal planing decisions and the authorities are latching onto AGW with great enthusiasm.
Did Prof Mann ever provide his code for the hockey stick?
MikeN
Most of it was provided, in response to a request from the US Senate. Certain parts, like the calculation of the confidence intervals, are still withheld.
I’ve done some experimentation. It seems I’m not allowed to link to the evidence. They have allowed me to say pretty much everything I said before, but without the link. I then tried to post to the Phil Jones quote where it appears in a submission David Holland made to the UK Treasury on the Stern report. It’s gone for moderation again.
So get that – a document at HM Treasury is deemed too dubious to appear on the BBC website!
Re: Bishop Hill (#37),
Can you crosspost your exact words. It’s hard to see how BBC can justify blocking a link by you to Climate Audit or to the David Holland Submission under their House Rules or Editorial Guidelines:
http://www.bbc.co.uk/messageboards/newguide/popup_checking_messages.html
http://www.bbc.co.uk/messageboards/newguide/popup_house_rules.html
http://www.bbc.co.uk/messageboards/newguide/popup_editorial_guidelines.html
These links to the bbc will work better, and should last:
For the scene setting
http://news.bbc.co.uk/1/hi/programmes/newsnight/8109923.stm
For the Q&A with the Environment Secretary, Hilary Benn
http://news.bbc.co.uk/1/hi/programmes/newsnight/8110064.stm
Matthew:
Many thanks and very much better.
If Bishop Hill linked directly to the PDF that would likely constitute a violation.
Per your third link.
Re: Bob Koss (#41),
A pdf is linked in comment 52 of the thread, which weights against your hypothesis.
Re Bishop Hill (various): Fascinating and sad, fascinating and sad.
I’m afraid I don’t have my exact words. I must get into the habit of taking copies. As you rightly point out, other posters seem to be able to post links to PDFs.
The only other thing I can think of is that I said explicitly in the post that Phil Jones was withholding the data (with a link to the CA pdf of CRU correspondence to back up my assertion). If they thought this was potentially libellous, you could understand their holding it back for someone more senior to look at.
In my second withheld post, however, I didn’t say anything about Jones, but linked to David Holland’s comments at HM Treasury. This again was referred to the moderators. This means that it can’t be saying explicit things about Jones on the comment.
So maybe it’s linking to potentially libellous stories about Jones? But it can’t be that either, because Per linked to CA’s story on Jones and that was deemed acceptable.
I’m bemused.
Regarding access to UK Government data:
http://news.bbc.co.uk/1/hi/technology/8096273.stm
in which “Sir Tim [Berners-Lee]refused to be specific about which items of data he would help make public, but said he was interested in suggestions from bloggers about what they would like to see.”
In the BBC report one-off weather events are used as evidence of climate change which is complete non-sense, and flooding was the main example. All of the events shown could easily be explained by natural variation, the result of building on flood plains, changes in water course management, lack of maintenance, etc. You can quickly check the validity of the BBC article by reading the following paper: http://www.cru.uea.ac.uk/cru/data/riverflow/flowreconstruction/ReconstructedFlowsTR.pdf. If you scroll to the graphs at the end, you will see no significant trend in any of the river flows across England and Wales over the period 1865 to 2005. River flow by definition is closely linked to rainfall in the UK and therefore you can be pretty certain that there is no trend there either.
The BBC’s Science Correspondent, Pallab Ghosh, has some interesting background on the regional UK Climate Predictions 2009 controversy here: http://news.bbc.co.uk/1/hi/sci/tech/8106513.stm
http://petitions.number10.gov.uk/CRUSourceCodes/
We the undersigned petition the Prime Minister to Force the Climate Research Unit, or other publicly funded organisations to release the source codes used in their computer models. The Met Office, the climate research unit and various individuals at numerous academic institutions are refusing to release the source codes used in their climate research models.
These are tax payer funded institutions, which are influencing government policy decisions which will affect the day to day lives of us all.
With the Prime Minister’s belief in a new age of transparency, it is unsurportable that these publicly funded organisations, are not open to public scrutiny .
Re: JC (#49),
The British Government has responded to the petition:
“We the undersigned petition the Prime Minister to force the Climate Research Unit, or other publicly funded organisations to release the source codes used in their computer models.”
Details of Petition:
“The Met Office , the climate research unit and various individuals at numerous academic institutions are refusing to release the source codes used in their climate research models. These are tax payer funded institutions, which are influencing government policy decisions which will affect the day to day lives of us all. With the Prime Minister’s belief in a new age of transparency, it is unsurportable that these publicly funded organisations, are not open to public scrutiny.”
The Government is strongly committed to the principles of freedom of information, and the Environmental Information Regulations 2004 specifically implement our international obligations over access to environmental information. The Met Office’s commitment to openness and transparency in the conduct of their operations and to the sharing of information is set out clearly on their website (http://www.metoffice.gov.uk/corporate/legal/foi.html).
Simple and transparent licences are in place to facilitate the re-use of the Met Office’s meteorological and climate data, and large quantities are freely available for academic and personal use, for example through the UK Climate Impacts Programme and the British Atmospheric Data Centre.
The Met Office’s climate models are configurations based on the Unified Model (UM), the numerical modelling system developed and used by the Met Office to produce all their weather forecasts and climate predictions.
You may be interested to know that the UM, including source code, is available for external use under licence. For general research, the licence is free; the Met Office just asks individuals to submit an abstract describing the research to be undertaken, and to provide an annual report describing the work undertaken, the results achieved and future work plans.
To improve access to their climate models, the Met Office has worked with Reading and Bristol Universities and NERC to develop a low-resolution version which can be run on a PC and is available to all UM licence holders.
Further Information on how to apply for a research licence can be found on the Met Office website.
http://www.metoffice.gov.uk/science/creating/working_together/um_collaboration.html
More secret public data:
http://badc.nerc.ac.uk/data/cet/
[Steve: you’re abusing the term secret here relative to other situations.]
I’ve now had a reply from the Met Office to my enquiry as to who was placing restrictions on the data.
Re: Bishop Hill (#52),
I do not believe any institutions or individuals actually placed any restrictions on the data. Climate data is not sensitive or related to national security. A restriction on data is contrary to the standards of science. The claim data was provided to Jones with restrictions is not credible IMO. If it had happened, there would be written agreements detailing exactly what the restrictions are. Someone should file an FOI request for these agreements. I bet Jones cannot come up with even one. Yes, I read the link by curious above and I think the excuses from CRU about why they cannot provide these documents would be highly entertaining.
Ex Civil Servant writes on UK FOI:
http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article6702869.ece
Ron
I’ll look into it.
One Trackback
[…] Jones may have drain the moat and sell access to his climate data lists and code to pay the […]