It must be humiliating for the UK Met Office to have to protect Phil Jones and CRU. Even a seasoned bureaucrat must have winced in order to write the following:
Some of the information was provided to Professor Jones on the strict understanding by the data providers that this station data must not be publicly released and it cannot be determined which countries or stations data were given in confidence as records were not kept.
Here is the complete text of the UK Met Office’s most recent refusal of their station data.
Our Ref: 22-06-2009-131902-003 23 July 2009
Dear Mr McIntyreRequest for Information – Information not Held and Refusal to Disclose Information
Your correspondence dated 9 June 2009 has been considered to be a request for information in accordance with the Environmental Information Regulations 2004. The Ministry of Defence is permitted to withhold information where exceptions are considered justifiable.You asked “You stated that CRUTEM3 data that you held was the value added data. Pursuant to the Environmental Information Regulations Act 2004, please provide me with this data in the digital form, together with any documents that you hold describing the procedures under which the data has been quality controlled and where deemed appropriate, adjusted to account for apparent non-climatic influences”.
Your request has been assessed and this letter is to inform you that the Met Office does hold some information covered by the request. We do not hold documents describing the procedures under which the data has been quality controlled or adjusted to account for apparent non-climatic influences.
The information held by the Met Office is withheld in accordance with the following exceptions pursuant to the Environmental Information Regulations Act 2004:
• Section 12 (5) (a) Information likely to prejudice relations between the United Kingdom and any International organisation;
• Section 12 (5) (e) Confidentiality of commercial or industrial information where such confidentiality is provided by law to protect a legitimate economic interest.
• Section 12 (5) (f) (i) (iii) The supplier was not under legal obligation to supply the information and has not consented to its disclosure.As the above exceptions are qualified exceptions, a public interest test was undertaken by the Met Office to consider whether there are overriding reasons why disclosure of this information would not be in the public interest. The Met Office has duly considered these reasons in conjunction with the public interest in disclosing the requested information, in particular the benefits of assisting the public having information on environmental information, whereby they would hope to influence decisions from a position of knowledge rather than speculation.
Access to environmental information is particularly important as environmental issues affect
the whole population.Consideration of Exception Regulation 12 (5) (a)
Much of the requested data comes from individual Scientists and Institutions from several countries. The Met Office received the data information from Professor Jones at the University of East Anglia on the strict understanding by the data providers that this station data must not be publicly released. If any of this information were released, scientists could be reluctant to share information and participate in scientific projects with the public sector organisations based in the UK in future. It would also damage the trust that scientists have in those scientists who happen to be employed in the public sector and could show the Met Office ignored the confidentiality in which the data information was provided.We considered that if the public have information on environmental matters, they could hope to influence decisions from a position of knowledge rather than speculation. However, the effective conduct of international relations depends upon maintaining trust and confidence between states and international organisations. This relationship of trust allows for the free and frank exchange of information on the understanding that it will be treated in confidence. If the United Kingdom does not respect such confidences, its ability to protect and promote United Kingdom interests through international relations may be hampered. Competitors/ Collaborators could be damaged by the release of information which was given to us in confidence and this will detrimentally affect the ability of the Met Office (UK) to co-operate with meteorological organisations and governments of other countries. This could also provoke a negative reaction from scientist globally if their information which they have requested remains private is disclosed.
Consideration of Exception Regulation 12 (5) (e)
The information is also withheld in accordance with the exception under regulation 12 (5) (e) because the information comprises of Station Data which are commercially sensitive for many of the data sources (particularly European and African Meteorological services) release of any data could adversely affect relationships with other Institutions and individuals, who may plan to use their data for their own commercial interests. Some of this is documented in Hulme, 1996 but this is not a globally comprehensive summary.The Met Office are not party to information which would allow us to determine which countries and stations data can or cannot be released as records were not kept, or given to the Met Office, therefore we cannot release data where we have no authority to do so. Competitors or collaborators could be damaged by the release of information which was given to us in confidence and could affect their ability to trade.
The Met Office uses the data solely and expressly to create a gridded product that we distribute without condition.
Consideration of Exception Regulation 12 (5) (f) (i) and (iii)
The information is also withheld in accordance with the exception under regulation 12 (5) (f) (i) (iii) as Professor Jones was not legally bound to release the data to the Met Office and has not consented to the disclosure to any other party. As stated above in 12 (5) (a) Some of the information was provided to Professor Jones on the strict understanding by the data providers that this station data must not be publicly released and it cannot be determined which countries or stations data were given in confidence as records were not kept. The Met Office received the data from Professor Jones on the proviso that it would not be released to any other source and to release it without authority would seriously affect the relationship between the United Kingdom and other Countries and Institutions.I hope this answers your enquiry.
If you are not satisfied with this response or you wish to complain about any aspect of the handling of your request, then you should contact me in the first instance. If informal resolution is not possible and you are still dissatisfied then you may apply for an independent internal review by contacting the Head of Corporate Information, 6th Floor, MOD Main Building, Whitehall, SW1A 2HB (e-mail CIO-XD@mod.uk). Please note that any request for an internal review must be made within 40 working days of the date on which the attempt to reach informal resolution has come to an end.
If you remain dissatisfied following an internal review, you may take your complaint to the Information Commissioner under the provisions of Section 50 of the Freedom of Information Act. Please note that the Information Commissioner will not investigate your case until the MOD internal review process has been completed. Further details of the role and powers of the Information Commissioner can be found on the Commissioner’s website, http://www.ico.gov.uk.
Yours sincerely,
Marion Archer
FOI Manager
Update Aug 4. Following reply sent:
Dear Ms Archer.
This is to acknowledge receipt of your letter of 23 July 2009. I request that you reconsider your decision for the following reasons.
You say that there is a strict understanding between CRU and data providers that station data not be publicly released. CRU’s actions show otherwise. CRU and Dr Jones have routinely posted station data online since 1990, with 1990, 1996 and 2003 versions being online at the time of my request to you. The 1990 version is online at CDIAC in the U.S. ( http://cdiac.ornl.gov/ftp/ndp020 ); from 1996 to until July 31, 2009, the 1996 version was online at CRU ( formerly accessible through http://www.cru.uea.ac.uk/advance10k/climdata.htm); from early 2003 to July 27, 2009, the 2003 version was online at CRU (formerly at ftp://ftp.cru.uea.ac.uk/data/). These latter two files were removed from public directories only after FOI/EIR requests for station data were submitted to the Met Office and CRU. Obviously any of the “lost” confidentiality agreements did not present a problem for placing data online up to 2003. Would you please consider your refusal in light of this additional information regarding online availability of earlier versions of these data sets between 1990 and July 31, 2009.
In addition, you stated that the Met Office had entered into an agreement with Dr Jones and/or CRU which included a proviso that the data “not be released to any other source”. Under the EIR, would you please provide me with the date of this agreement, the parties to the agreement, a copy of the clause containing the precise language of this proviso and a copy of the entire agreement or any other document evidencing that the Met Office had entered into such an agreement.
Yours truly,
Stephen McIntyre
Climate Audit 
111 Comments
If this means “no records were kept” of a binding agreement, someone has been a little lax in their responsibilities.
Jones covering his A$$.
This reminds me of the reporter who did FOI requests for 2 years to get UK politician’s spending reports and was denied at every turn. She finally bribed someone and broke the lid off the huge fraud that was being perpetrated by politicians spending money on porn and personal items. Do you know anyone @ the Met who takes bribes, Steve?
National Security…
On brief reflection, things can only get worse for the Met Office, because they are using data to which they have no demonstrable right of access. If they are no records of the terms under which data was made available to Jones, he can give the Met Office no warranty that they can hold or make any use of the data whatsoever. So, now just FOI the terms under which Jones supplied the data.
In the US, you can get access to proprietary commercial data that the govt is using if you sign a non-disclosure agreement and are not in a position to benefit economically from the info. I would think that you(Steve McIntyre) would fit those requirements, so maybe you could ask them if yolu could sign a non-disclosure agreement. It would allow you to do the audit, and even discuss the results, as long as you do not disclose the proprietary data or results of an analysis that could identify specific proprietary aspects of the analysis. The nuclear people do this all the time, publishing plots of experimental results that are normalized, without the actual values, to show how well their analyses compare to data from proprietary experiments.
A very British farce, almost perfect Sir Humphrey !.
It was once explained to me when I asked why information given as fact, relating to public electric supply reliability had altered completely over a period of one year.
“The first conclusion reached was the result of a “report into arrangements for incident management”, the second conclusion was the result of an “inquiry following an incident “. Most amusing was that it was the same consultants who produced both!
So the Met Office’s documentation of these binding agreements is “Phil Jones says there were some, but he doesn’t remember how many or which”?
Oh, isn’t that just lovely. Blocks release of *any* data, because of some indistinguishable (they claim) subset.
Tho please note while they are offering multiple reasons to turn down your request, they are also claiming that they would be justified in doing so for the sole reason that Phil Jones has not consented. His reason for not consenting is really irrelevant to their ability to use that reason for denying the request, as I read it.
RE Antionio Sam, #4,
Sam was joking, but it’s not entirely inconceivable that Bismark, say, classified Tanganyikan weather records as Top Secret in order to keep the Brits in the dark as to what kind of uniforms to wear in the event of war. If no one bothered to declassify these records afterwards, Jones may have been able to obtain them only by promising to burn them after reading.
But if something like this is the case, there is no reason that the IPCC should be allowed to use indices like CRUtemp that depend on such closed access data in its Assessment Reports.
It’s too late to block AR4 etc. from using the CRU indices, but there is still time for sponsoring governments to require AR5 to use only open-access temperature indices that disclose both their raw data and their adjustment methods. And to declassify all historical weather records in their possession.
So… what is the procedure for requiring Dr. Jones and/or the Met Office to ask all of the data suppliers for permission to release the data, and then releasing that for which permission has been given? Might cost a bit in time and effort, but then, records should have been kept in the first place on such precious information. The ones failing to do their job right in the first place should bear the cost of cleaning up their mess.
Furthermore, the damage to international relations that might occur from climate predictions based on an erroneous UK dataset ought to trump any presumed ones from releasing historical temperature series.
Would this be a relevant issue for those of us resident in the UK to pursue through our elected representatives? In other words, is this information important, or is the fact of refusal in itself the interesting data?
These exercises reveal the bureaucratic silliness, resistance and ends that these organizations will go not to comply with these requests – both in other countries and especially in my country, the US. Unless one wants to go to rather marginal excuses, these actions are difficult to defend and they certainly open the whole process to questioning the motivations of the withholders. I therefore, find these efforts of value in revealing these weaknesses in a system that is supposed to lead to more transparency.
My question is, for Britain in this case, and the US in others that have been discussed at CA, what are the chances of obtaining the requested information without bringing a law suit against the organization in question. It would appear that if the dog ate the documents excuse can work in denying access without a lawsuit, that most of these cases must fail at this level unless the request is something very simple and non controversial.
The only hope outside a law suit would appear to be publicizing the requested organizations reactions to the request with the hope that that publicity would embarrass the organization to do what is right. Even that approach seems to have only infrequent success. Often it is met here with defenders posting at CA and giving the requested organizations every benefit of the doubt, indicating that the few people outside the consensus in these matters are rather alone in their concerns about transparency.
Steve, just FYI if you hadn’t seen, the Science & Public Policy Institute released a study yesterday estimating funding to global warming alarmists at $79 billion, over a thousand times more than money given to skeptics by evil oil companies. You and CA were cited in one section on the hockey stick graph.
http://scienceandpublicpolicy.org/originals/climate_money.html
This relationship of trust allows for the free and frank exchange of information on the understanding that it will be treated in confidence.
My irony meter just exploded.
This is beginning to look more like the Wizard of Oz everyday. Who is that man behind the curtain collecting super duper secret information? Information that, if made publicly available, would jeopardize the very nature of science?
OOps, the blockquote got screwed up…
I don’t understand. The request to the Met Office was for a copy of the data sent to Peter Webster. If the restrictions on the data hold for SM, why did they not prevent release of the data to Webster?
Re: Bishop Hill (#17),
To me, that’s the central argument in Steve’s favor. You can’t (by law) release the data without restriction to one party and then argue that you can’t release it to another party because its confidential. Its either confidential or its not. If its confidential, then you must protect it for it to remain confidential. Once you release data without restriction to one party, then the toothpaste is out of the tube — you can’t put it back in.
Can you show that the information was released to Webster without restriction?
I think you just made some pretty big progress in pinning them down.
Re: Bishop Hill (#17),
Bishop – that’s a separate FOI to CRU. I haven’t heard back on that one. This is a parallel one to the Met Office.
This adds a bit more fuel to your submission to EPA Endangerment Finding .
Perhaps you should submit an addendum, including the Met Office refusal letter.
Steve, if you decide to follow the whole procedure proposed by the Met Office and if your request at the end is not satisfied, you can sue UK regulations to the european commission considering the following directive about access to environnemental information…
It can be efficient … long but efficient …
Sorry, the link :
http://europa.eu/legislation_summaries/environment/general_provisions/l28091_en.htm
A public institution should not use data that it can’t disclose.
My email …
Re: Willis Eschenbach (#22),
I agree. I do think that given how important this data is, its a situation worth litigating.
I keep coming back and re-reading their response. Its absolutely preposterous.
Perhaps you should submit a Freedom of Information request to Phil Jones’ employer:
http://www.uea.ac.uk/is/foi
I think the last discussion was: The UK Met Office Deepens The Moat
Let’s see…
The MET office knows some of the data is confidential, but they don’t know which is because they don’t have any records. Also, they don’t seem to know where some of the other data came from or if it is confidential. And to top it off, they don’t have any records of agreement. This would imply gross incompetence on the part of the MET. And besides, they don’t want to hurt relations with other international organizations (IPCC) by releasing data that might hurt the AGW cause.
Can you say “Keystone Cops?” This would be funny if trillions of dollars, euros, and other weren’t on the line.
I am sorry to say it but the the way these exchanges are developing does not surprise me. The Met Office is under the the juristiction of the Military of Defence and the final block against releasing the requested data will be that such data can not be released as such an action: “…could jeopardise the defence of the Realm.”
Good luck Steve, but I fear all your efforts will be in vain.
They’re hiding something embarassing.
What else shall one conclude?
Keep plugging at them Steve. It aint right that they are not providing what is rightfully ours.
Re: Pierre Gosselin (#28),
Yeah. This problem is getting pretty common these days.
@No. 3
Interesting proposition.
If you need us hitting the tip jar, Steve, just say so.
Well I don’t know Phil Jones at all, but either he’s got something to hide or he is so behind the times that others would dismiss him as a dinosaur.
Perhaps someone who has worked with him in the past/ been taught by him might offer some insight.
Araucan
has provided a possible alternative avenue.
UK is in the EU. It stated in the provided link:
I hope this appears in a legible format.
James Bond went out and stole the secret air temperature data from SMERSH, the KGB, the Saudis, and the American Allies all through the 1960’s, and now the Brits don’t want to admit it…
What could possibly be so sensitive and top secret about global temperature data?
Clearly we have a government entity here that is wrongfully hiding something from the public and stonewalling.
Okay, let me see if I have this straight. Phil Jones claims to have promised not to release certain data, let’s call it “sensitive climate data” (that phrase cracks me up) or possibly “incredibly shy climate data.” Anyway, Jones would be happy to comply with this quite reasonable request for data but his hands are tied because of his promise. What’s a bureaucrat to do?
Aren’t we talking about a time series here? Is he not still getting data from the same people. Has he forgotten who he providing him with data? Why can’t Jones at least provide the names of all data providers and then Steve McIntyre or some other interested person can contact them all to find out who is restricting data, why and what the restriction are exactly?
Jones could certainly be more helpful if he wanted to be. He is acting like he has something to hide.
Seems to me that it is time to engage specialist lawyers (much as I would normally be reluctant to suggest such a thing). In my experience there are numerous lawyers who are sceptical. Perhaps such a lawyer might be interested in helping on a pro-bono basis.
Alternatively, I would think that a specialist fighting fund to support determined legal action would attract considerable interest from patrons of this blog, WUWT, and other supportive blogs.
Interestingly, such action would attract publicity to the issue, which is likely to be the most powerful influence on the politicians and press, who, with some notable exceptions (Australia’s Senator Steve Fielding, US Senators Barton and Imhofe, journalists Christopher Booker, Andrew Bolt, George Will, Lawrence Solomon, Thomas Fuller) have simply swallowed the IPCC yarn, hook line and sinker.
i wonder if this is a verifiable statement, and if it is FOI’able ?
per
SM,
you have, I assume, the listing of stations from CRU. You also have a listing of stations from GISS and others
who release their data. I suppose a cross check of those could be helpful in determining which countries ( if any)
and which meterological organizations, if any, would have put a restriction on Jones ( if at all). If one could make a stab
at which organizations ( again if any) have required confidentiality with Jone, then one could inquire with them.
Also, Think of this. Jones continues to get get data from these sources. It would seem a reasonable request for him
to ask the CURRENT suppliers of data if they require confidentiality. In fact, Since he has lost his records it would
see that one could demand that he or the organization he works for should be required to bring their records
up to date
You can get a list of CRU stations. Go to:
http://www.cru.uea.ac.uk/cru/data/landstations/
and download the file:
crustnsused.txt
I analysed these last year. At the time CRU listed 4137 stations. By comparing them with the statiosn of the same WMO number in the GHCN data set I found:
Rural: 955
Semi-rural:471
Urban:936
Unclassified: 1775
It may be that by cross-checling names and/or grid references which I did not do more links could be identified.
It’s not what you want but it is a start.
Re: Anne T Cyclone (#38),
I’m familiar with that file. That file was produced in response to prior FOI requests and has been discussed before. It took several years to get. I know what’s in it.
If there were a valid reason why I should be content with this listing, then maybe I’d be content with it. But there is no valid reason.
Plus we recently learned that Jones sent CRU data to PEter Webster. There’s an argument that this voids his “confidentiality” claim.
For British citizens/residents:
http://petitions.number10.gov.uk/CRUSourceCodes/
…..and I know I missed out the request for the data , but it will not allow me to edit.
I wouldn’t conclude the FI office or officer is hiding something embarrassing. But don’t rule out others.
A simple explanation: The bureaucracy is acting in a most efficient manner.
Nothing is more convenient and less troubling to the immune civil servant than to reject such inquiries. OTOH producing the information may make enemies or violate some technicality.
Here Archer is just saying “If you don’t like the answer you can keep trying. Won’t disturb me at all.”
All very polite. And we should be glad of that, some governments behave otherwise.
#37 I don”t believe for a minute that some other entity told Jones not to release their data. It’s just another dodge.
Steve, If this communication from the Met Office wasn’t in the context of “climate science” it would be very difficult to believe.
It would perhaps be useful if several people could submit requests for copies of the data referred to. I would do this very happily right now, but am not sure of the appropriate address, or indeed a name, apart from the signatory of your letter. I may try to find something but it would be a tremendous help if you were able to publish an address here. I would not use email but a standard hand written letter, addressed to someone high in the hierarchy of the Met Office, since my experience has been that this form of communication is more likely to get read and acted upon than an email.
Having read the letter again I am even more staggered by the crass reasoning they seem to have used. Barely believable that it is presented as being serious. In particular I have been studying the sentence “We considered that if the public have information on environmental matters, they could hope to influence decisions from a position of knowledge rather than speculation.” I have to use McEnroe’s famous statement “You cannot be serious!” when considering the implications of such a totalitarian pronouncement.
I look forward to more postings on the subject.
Also think about pinning Jones down a bit. According to the MET Jones has informed them that the data is confidential and that he didnt keep records. You should probably FOI that statement. Also, usually with every request for documents I ever got I had to sign a statement stating that I made a diligent search. Failure to keep good records ( in my case ) could result in charges. So, folks should start to go after the communication with Jones about this matter.
What rules of due diligence is he subject to. Was he advised to make a complete search. Are there codes of conduct with regard to the keeping of records. has he supplied the data to others ( webster) FOI those communications. the cover up …
The “I don’t remember” thing is still a red herring tho. ANY of the three sections cited can be used to deny the request.
Section 12 (5) (f) (i) (iii) is going to be the killer no matter what else can be disposed of as ridiculous, or even consents laboriously reacquired station by station.
Steve,
OT if I may. I recently began learning R and found it to be more well suited for many of the calculations that I’ve used MATLAB for. Importing data is a trivial matter. I just loaded the ncdf package and was shocked and amazed that I could load the entire HadCRUT file! MATLAB would have spit out an out of memory message. I’m very close to considering myself a convert. I wish I had taken the advice you had given me about one year ago to start learning R.
We have multiple people involved in this who are direct public employees.
The current excuse is that they apparently think they have a lot of time invested in this data, so they personally “own it.”
What’s the current penalty for appropriating public property for private use? Sounds like a nice companion piece for those recent stories about politicians buying porn and dredging out moats with government funds…
This alone should be the subject of a laser-focused FOIA request — request a copy of the written agreement (or any correspondence or contemporaneous notes) between Jones and UK Met that restricts redistribution of this data.
Somebody in the UK familiar with the various press organizations should e-mail Steve’s FOI request, the Met office response and W. Eschenbach’s email response to the Met office ruling. Since the recent scandal with Parliament avoiding an FOI request, there is a chance the Press might pick up the story and run with it. It should be pointed out that the scandal in Parliament was only discovered using a bribe and not FOI laws. And the question should be have FOI laws and its administrators once again failed the public with this ruling.
Folks, I get very tired of people making suggestions as to what they think that I should do. IF you think that you have a good idea, then do the FOI yourself.
Aside from saving me some work, it has other more important effects – it shows that other people besides me are concerned about this. And it forces them to answer you.
The FOI officers are: Met Office marion.archer at metoffice.gov.uk and
CRU david.palmer at ues.ac.uk
So from now on, instead of writing in with a suggestion, I’d like to see people say – I have just the following to request under the Environmental Impact Regulations (which is the relevant FOI regulations) seeking xyz information. Don’t worry about whether it’s the best possible phrasing. Stop talking about it – do it.
Hulme 1996 mentioned here without a citation is, I presume, a chapter by Hulme in the Physical Geography of Africa ed, Adams (See CRU publications list for Hulme.) If any reader has handy access to this, I’d appreciate any information on what conceivable relevance this has to teh FOI request.
Re: Steve McIntyre (#54), I have just picked up a copy of The Physical Geography of Africa (William M. Adams et al., eds, 1996). Chapter 5 is Climate Change Within the Period of Meteorological Records by Mike Hulme. This starts with a brief history of meteorological records in Africa, referring to problems of homogeneity of data and urban warming due to site changes. There are maps of locations of stations from the 1890s to the 1980s, and a nice picture of a Stephenson screen in the Sudan. A time series of temperature and rainfall in Cape Town is shown, with the comment “Data extracted from the CRU/DoE global data set described in Jones et al. (1985, subsequently updated). There’s a long discussion of changes in rainfall, and a brief discussion of temperature with the comment “The gridded global data set of Jones et al. (1985, updated) can be used to extract regional time series”. Two time series are shown, as annual data points and “smoothed with a filter which suppresses variations on time-scales of less than 10 years”. There is a brief discussion of glaciers, and a longish discussion of possible causes (land use changes, ENSO, and CO2). He concludes that it’s not possible to say much at this stage.
The key reference is Jones, P. D., Raper, S. C. B., Santer, B. D., Cherry, B. S. G., Goodess, C. M., Kelly, P. M., Wigley, T. M. L., Bradley, R. S and Diaz, H. F. (1985), A Grid Point Surface Air Temperature Data Set for the Northern Hemisphere, US DoE report no. TR022.
Re: Jonathan (#71), just a thought, but isn’t Hulme 1996 likely to be a misprint for Hulme 1994? I’m thinking of The cost of climate data-a European experience, M. Hulme, Weather 49, 168 (1994).
Re: Jonathan (#71),
The 1985 reports by Jones et al. were for DOE under contract. That data may be available under US FOI. See:
Re: David L. Hagen (#107),
PS those reports are available via NTIS as reports# DE85015609 and DE86007205
RE #52, #38,
See “A First Look at the CRU Station List”, Oct. 3, 2007.
It seems that our own Willis Eschenbach used the UK FOI to pry the CRU station list loose from Jones’s cold, dead fingers.
It will undoubtedly take even greater efforts to get the actual data out of him. It’s worth trying, but meanwhile, let’s get the secretive CRU indices banned from AR5!
I have tried getting temperature data from some countries (France was one). Some data is free. But some requires payment to be made.
If any of the data you are requesting is restriced but provided to CRU free or has been paid for, then surely CRU cannot give this data away free? Unless the databank has been set up to separate free data from restricted data then separating these for passing on would be difficult and time consuming.
Re: thefordprefect (#56),
If meterological organizations expect public policy to be based on this data, then it has to be public. If they want to keep the data private, then don’t use it for public policy documents.
Re: thefordprefect (#56),
Pure speculation. We don’t need apologists, we need the data.
Re: Jeff Alberts (#60),
http://www.meteo.fr/meteonet_en/services/cli.htm
Data costs! Unless I have missed the free download section
Please note that I am not saying refusal is right, I’m just saying providing may not be legal.
If you want to see the absurditity of copyright look at what you agree to when you buy a DVD of a movie. You cannot (in uk at least) loan (for no payment) the DVD to a friend etc. Even Charity shops selling 2nd hand DVDs are probably breaking the law!!
Re: thefordprefect (#77),
Utter rubbish. For example, from here:
Sorry for the slightly OT comment, but there is an important point buried in it. People often just assume they know the law – perhaps some bloke at a bar imparted some urban myth wisdom, or perhaps people confuse their own opinion for the law. Always, always, always, check the law. Do not assume it coincides with your opinions, common sense or any assumed morality.
Re: Spence_UK (#79), I have a recording of an official of FACT telling me that loaning is not legal. Therefore your quote is of great interest, thanks!.
snip – please do not speculate like this
Agree with SM in #58. Wherever data is DUV (Data Unavailable for Verification) then the research should not be used by the WMO, the EU, the IPCC, the USA, the UK, the EPA or any taxpayer-funded body to shape or inform public policy.
I have already done the FoIs that are being discussed here – to the Met Office for the agreement with Jones and to CRU for the agreements with national met offices putting restrictions in place. Answers should be due in a couple of weeks.
I’m still not clear if anyone has ever requested Jones’ code. If not I will request that too. Is anyone aware of such a request ever having been made before?
Bishop,
Can you post your requests so that others can follow suit. I recall when Steve did that a bunch of us took his request,
made our own versions and got some results ( AR4 FOI).
Request to CRU:
Dear Sir or Madam,
I gather from Dr Phil Jones’ correspondence with Douglas Keenan
(see
http://www.climateaudit.org/corresponden…)that
restrictions have been placed on redistribution of climate data by
some of the countries that have supplied this data.
I would like to receive copies of all agreements or other
correspondence where such restrictions have been placed.
Yours faithfully,
Request to Met Office:
Further to my recent enquiry regarding CRUTEM station data, firstly
thank-you for your prompt reply. Could you please send me a copy of
the agreement/letter with Dr Jones in which he makes these
restrictions on redistribution of the data.
I’d like to draw attention to the fact that 16 years ago Jones DID make all the station data publicly available, here:
http://cdiac.ornl.gov/ftp/ndp020/
I can not understand what could possible have changed since then, but I’m no legal expert.
Re: Jean S (#67),
do I read it correctly, 1 station for SH Jan 1851
RIO DE JANEIRO BRAZIL
or
HOBART,TASMANIA AUSTRALIA
and 103 for SH Dec 1990 ? (86, 551 for NH) Interesting set, now need to Matlab the Fortran code .. 😉
Re: UC (#73),
So far as I can establish, the first systematic Australian records date from 1856 +/- 1 year, Melbourne. The discovery of gold in several countries in the 1860-1880 period provided new wealth. This possibly contributed to more science funding and more recording becoming affordable.
Since we are all guessing a little, is a simple explanation possible for CRU? Did someone use copyrighted data illegally and fears retribution following disclosure?
I have an email from P D Jones where he claims he longer has the data used in a publication. If he does not have it, he cannot pass it on. End of story for that piece. End of credibility for the paper.
The simple approach is to discontinue use of the HADCRU data. If it cannot be replicated, it has little to no scientific integrity and no proven validity. But this needs to be told to the public, which some of us have been busy doing for a few years. The views of some decision makers here have now swung over. More are following at increasing pace.
Steve’s correct. Don’t load him up. Do it yourself. It’s not hard. Weight of inquiry numbers will tell, or someone will crack the secrecy formula with serendipity or good thinking.
Ask your Country’s Minister for Environment to make a request for the data on behalf of your country’s people and to inform parliamentary debate.
Those that know what to do with this information may like to get some political help. Nigel Lawson now Baron Lawson of Blaby is not convinced by the “science” behind AGW.
“Lord Lawson claims climate change hysteria heralds a ‘new age of unreason”
http://www.telegraph.co.uk/earth/earthnews/3338632/Lord-Lawson-claims-climate-change-hysteria-heralds-a-new-age-of-unreason.html
As Margaret Thatcher’s Chancellor during the 1980s, he may have some influence.
http://biographies.parliament.uk/parliament/default.asp?id=27077
Political interests – Climate Change
See links below.
UK Freedom of Information Act and Environmental Information Regulations
Click to access ico_foi_eir_how_to_comp_final.pdf
https://forms.ico.gov.uk/freedom-of-information-or-eir-complaint.aspx
UK Minister with responsibility for the Met Office
Under Secretary of State for Defence and Minister for Veterans
Kevan Jones MP
http://findyourmp.parliament.uk/constituencies/north-durham/messages/new
Good Luck
PS Don’t mention helicopters.
UK John:
I agree, Sir Humphrey would no doubt approve.
James Hacker: How am I going to explain the missing documents to “The Mail”?
Sir Humphrey Appleby: Well, this is what we normally do in circumstances like these.
James Hacker: [reads memo] This file contains the complete set of papers, except for a number of secret documents, a few others which are part of still active files, some correspondence lost in the floods of 1967…
James Hacker: Was 1967 a particularly bad winter?
Sir Humphrey Appleby: No, a marvelous winter. We lost no end of embarrassing files.
James Hacker: [reads] Some records which went astray in the move to London and others when the War Office was incorporated in the Ministry of Defense, and the normal withdrawal of papers whose publication could give grounds for an action for libel or breach of confidence or cause embarrassment to friendly governments.
James Hacker: That’s pretty comprehensive. How many does that normally leave for them to look at?
James Hacker: How many does it actually leave? About a hundred?… Fifty?… Ten?… Five?… Four?… Three?… Two?… One?… *Zero?*
Sir Humphrey Appleby: Yes, Minister.
Note how the Met Office’s FOI Manager, Marion Archer, moved the goal posts.
Ms. Archer claimed three bases to reject release of data. The language of these regulations is very precise; each word has precise meaning.
Archer then explains that “scientists” might not want their “confidential” data released. Were any of the scientists asked ? Of course not, that would require Jones to have given notes to the Met Office with the scientists’ contact information.
But even if they had asked, it would be irrelevant.
A scientist does not = an International organization.
A hundred scientists do not = an International organization.
Then she concludes that “international relations” could be negatively affected between the UK and foreign governments, limiting the UK’s ability to protect national interests.
A bit speculative for a low-level functionary. That seems like a determination for the Foreign Office.
Anyway, a foreign government is not = an International organization.
The ICRC is an International organization, as is the World Bank. They are chartered, lack sovereignty, and operate in many countries.
Archer mentions hypothetical difficulty dealing with “meteorological organizations”, but doesn’t identify a single International meteorological organization that would actually object to the release of data, such that release of the data would cause so great a rift that it would “be likely to prejudice relations” with the International organization.
Is station identification data commercial ? Has it ever been bought and sold ? Is there a commodities market that Jones is trying to corner ? What’s the going rate for station futures ? [answer: neg 2% economic growth] I thought Jones, et Al had to actually PAY money to get this stuff published in a journal – how could it be commercially valuable ? Met Office gives their “gridded product” away for free.
There must be a commercial market, though, because Archer tells us that “competitors” could find “ability to trade” impaired.
Who knew all those reclusive scientists sitting on old, dusty boxes of temp records were greedy capitalists at heart.
Once properly formatted and published, those Tornetrask records will command more than Harry Potter and Thriller, combined. Now there’s an “economic interest” that needs protecting.
Governments deprive people of “economic interests” all the time, however. [zoning, taxing, banning, regulating…]
Note that the regulation only applies to economic interests where confidentiality is “provided by law .” That means there has to be some OTHER area of UK law that provides confidentiality – such confidentiality not being lost by virtue of the FOI Act.
Generally, these would be trade secrets – processes and recipes that could be patented, if the owner wanted. Examples would be the recipes for Coke and Kentucky Fried Chicken.
In order for the law to “provide” confidentiality, the owner of trade secrets has to do something not commonly done, and assert ownership by taking steps to protect the secret. The trade secret has to actually provide a real economic benefit to the owner, it can’t be something trivial like using blue paper clips or calling your customers “guests.”
It’s well-known that Coke and KFC have unique, closely guarded secrets that generate millions in revenue.
Can the same be said for this station data ?
Well, no, because Archer tells us we don’t even know who purports to own this stuff. How is this information in any way unique , how do we know the originators want it kept confidential [other than by Jones’ convenient explanations…], have any of the originators derived real economic benefit from this data ?
Archer takes the backwards view that ALL data has presumed confidentiality provided by law – until the Authors can be ferreted out and affirmatively reject confidentiality. [Difficult when Jones won’t tell you who they are…]
The opposite is generally true, “under the law “. In a lawsuit for theft of trade secrets, the owner of the secrets would have to demonstrate their recipe or process had the attributes of a trade secret.
But even if all the Authors were identified, and could demonstrate that they possessed genuine trade secrets, they still probably wouldn’t be protected by UK law.
UK law protects UK trade secrets, not Chinese, Bolivian, or Antarctic trade secrets. It may limit UK corps’ and subjects’ ability to steal trade secrets abroad – in the same way US law proscribes US corps from giving bribes to foreign governments.
These are international data, held by international scientists – they’re probably NOT provided confidentiality by non-FOI UK law.
I’ve just fired off FOI’s to the Met and UEA. Two prongs, one requesting the data and methodology, the other questioning the use of the data if the provenance cannot be certain and the Met’s creation of derivative commercial works if they’re uncertain as to what can/cannot be released. I suspect that as the Met’s budget is under pressure, they’re afraid their profits may suffer by this release. I suggest they’ll suffer more if they can’t be trusted.
Additionally, I’m a party donor and will be attending a fundraiser shortly. I’ll raise this matter given there’s an obvious and urgent public interest in releasing this information. If they persist in stalling, it simply damages the Met’s credibility even further.
To echo Steve’s comment, rather than disussing it here, anyone in the UK should send FOI requests and copy their MP’s and open minded journalists like Christopher Booker at the Telegraph.
Dear Steve
Sorry to be a thicko but please can you tell me what I specifically need to ask for? Is it as follows edited from your intro above.
Please provide me with CRUTEM3 data that you hold. Pursuant to the Environmental Information Regulations Act 2004, please provide me with this data in the digital form, together with any documents that you hold describing the procedures under which the data has been quality controlled and where deemed appropriate, adjusted to account for apparent non-climatic influences”.
Anthony Watts has suggested contacting the UEA. I am happy to contact the Met (MOD)and UEA. If there is no joy I will then contact my MP.
While I am at it I may also ask for details of expenses:-£
Untill it is released when any one quotes`Hadcrut or links to it on a blog. I will just rubbish the link by saying they have something to hide and the findings cannot be relied upon.
Willis has sent an email which I am happy to use if he his?
One thing to note beaureacrats hide under procedure and my view is that direct requests may carry more weight than trying to support your efforts directly?
Steve when people suggest you do something just remember they are trying to delegate upwards 😉
Did Webster get his data directly from Jones rather than the Met Office? Because if he did, that may not count as precedent re that regulation that requires them to get consent from Jones before releasing the data. If *they* (Met) didn’t release it to Webster, then you can’t argue that they are applying their rules differently for Steve.
You can easily make Freedom of Info Act requests from the website linked in my name. These can be made anonymously if you wish and are publically accessible (request and response).
FOI requests for the met office can be made here:
http://www.whatdotheyknow.com/body/met_office
And FOI requests for Cru can be made here:
http://www.whatdotheyknow.com/body/university_of_east_anglia
You don’t have to be a UK citizen to make the requests either so anyone can do it!
Just a reminder that most world weather station data is available from within Mathematica 7.0. I tried it when CA was discussing Siberian anomalies and I could download all stations in that region with up to the hour results. There is also WeatherUnderground. I would be really great if a volunteer team took up this public data for an analysis. I’m not in a position to do so, but would love to help out.
Re: Craig Loehle (#84), I’ve used weather underground quite a bit. Amazing how many of the “lost” stations report to weather underground. And it’s not like they do so in some weird, secretive way – they do so using their GHCN ID’s.
.
It’s astounding.
There was a CRU FOI response to a question about the impact of adjustments to the HADCRUT3 global temp index earlier this year. See
Sorry, try
Or even http://www.whatdotheyknow.com/request/impact_of_adjustments#incoming-13778
Re: Varco (#87),
Ok, one can (for example) adjust past to be colder and present warmer, and then show adjustments in a histogram to show that there’s no effect. For example,
adjust=sort(randn(100,1));
%temp=temp+adjust;
hist(adjust,10)
Just as a suggestion to other UK residents.
Worth emailing David Cameron and Liam Fox ( Tory leader and shadow defence). While not in power at the moment, they probably will be and Cameron has made alot of noises about governments being open and publishing data on the internet. That promise was mainly to do with government spending projects, however proper phrasing and a reminder of the political damage caused by the blocking of the Expenses FOI could cause them to push this matter. This will take a bit of time but better than relying on emails to Brown and Milliband.
Jonathan
Ir seems as if many contributors are not aware of the fact the UK Met Office, and many other Met Services around the world, charge for data. To buy, say, 10 years of monthly data for 4 stations with 4 parameters at each station, would cost thousands of pounds. This is something which consultants designing flood protection schemes, for example, have to pay. If the Met Office, under the freedom of information act, was to start releasing data it holds then it would lose a sizable income stream.
Steve: If Met Offices are worried about climate change, then they should make relevant data public. If they are more concerned about selling data to consultants, that’s their prerogative, but then such data should not be used in data sets being used by IPCC.
Re: Anne T Cyclone (#89),
That’s just a problem of market forces and the way formerly public bodies have been turned into ‘commercial’ ventures. Taxpayers fund the data collection and processing and there is an undeniable public interest in ensuring that data is trustworthy and credible. If it isn’t, then the data and products have a diminished value anyway, so it should be in the Met’s interests to be more open and have their product validated.
Unfortunately given it’s under new management, science and public interests are swayed by profit motivations.
Re: Anne T Cyclone (#89),
This comment muddies the waters of the topic at hand without providing more details. Is not the Met Office “competing” against sources of the same data that are offered free of charge? Are we talking here about the data of the type requested by Steve M? If this were the case where information is withheld in freedom of information requests why would not this simply be the one and only explanation given to the requester? Has this explanation been used previously to anyone’s recollection?
Varco
That was me! They were clearly going to obfuscate, and I didn’t have time to follow it up.
One has to love this statement.
This government agency puts out data with their name behind it and they don’t even know the exact procedures involved in how it’s processed.
Mind you that this data is going to be used to set binding restrictions on just about anything that emits GHG’s with stiff financial penalties for failing to meet target cuts in said GHG’s.
Nice.
National Review’s global warming webpage “Planet Gore” has picked up on this story from Anthony and WUWT:
http://planetgore.nationalreview.com/
Just watching a repeat programme on the BBC Neil Young
Don’t be denied
Hey take care.
I wonder what the Canadian WMO representative would have to say about this sort of thing? I would expect he might be able to loosen their hold on this information. Give him a call.
With regard to UK Met Office charging for data.
As no one else had mentioned this as a reason for non-disclosure under FoI requests I thought it was worth mentioning. As I pointed out, it is clear that if data were made available free or at nominal charge then this would mean the Met Office would lose a lot of money. Many other countries also charge for met data. It is therefore understandable that countries who charge for data would require the CRU not to provide that data under FoI requests.
To be clear, I do not support this policy but it seems to me that it is a sufficient explanation of why CRU does not release data.
Re: Anne T Cyclone (#100),
Anne, if it is true that there are charges for the data, then all they have to do is say so.
Re: Anne T Cyclone (#100),
Without details of the specific data for which the Met offices charge and how this might affect Steve M’s request and further whether the charging issue has ever been used to deny an FOI request, your comments remain too general to add anything to the topic of this thread. I suppose I could do my own research on this issue, but you brought it up.
This is a total red herring until the Met Office uses this excuse. Let’s stick with the excuses that they give. If they use this excuse, we can deal with it then.
Re: TAG (#104),
You’re right that they mention it.
However, if they are relying on this, then let’s find out which nations, if any, are withholding data for commercial purposes and get them to waive this privilege,
As noted in a new post, some of the non-GHCN versions in CRU come from Sweden, Denmark, Norway, Canada… and I am highly doubtful that such nations will assert commercial priority as an excuse to avoid disclosure.
If the various met offices want to assert that temperature data is confidential for commercial purposes, so be it. I don’t think that they;ll like the public attitude.
Re: Steve McIntyre (#106), as previously mentioned Hulme 1996 is probably a misprint for Hulme 1994; this states that “Data Agreement Forms” were required to obtain data from Germany, Norway, Spain and the UK. Data was not acquired from the Czech Republic, Hungary, Romania and Ukraine because of the large fees requested.
Ref 103 … They said precisely this …
“The information is also withheld in accordance with the exception under regulation 12 (5) (e) because the information comprises of Station Data which are commercially sensitive for many of the data sources (particularly European and African Meteorological services)”
It’s interesting to note that when this story broke on slashdot (), the main respondents were realclimate fans. Now after a good few hours the most popular responses seem to be from sceptics. I conclude from this that the warmists have some kind of early warning system that allows them to get in first with their replies, but that the majority opinion is firmly sceptical.
… and of course, I firmly screwed up the link in my post above, so would appreciate an edit ;).
“The Ministry of Defence is permitted to withhold information where exceptions are considered justifiable” and they will lie to you and make up stuff like “It would also damage the trust that scientists have in those scientists who happen to be employed in the public sector and could show the Met Office ignored the confidentiality in which the data information was provided” in order to obfuscate and divert attention away from what they consider classified. It would be ironic if Dr. M were spirited away and given “special rendition” to someplace secret like Guantanamo, never to be seen again, because he crossed some hidden line in an effort to make seemingly innocuous climate info public. And especially since any bad guys already know the info in context anyway – classified material more often than not just keeps government screwups hidden from the public, not real enemies. Given what’s happening with glaciers, the arctic summer ice cover, collapsing Antarctic ice shelves, record droughts, sea surface temperature, and all the other observations of global warming, does it really matter what this raw data is?
Follow-up sent today as follows:
Re: Steve McIntyre (#116),
The reply should be interesting. How do you avoid the document?
As a late arrival, not a scientist but convinced that this issue is primarily driven by politicians, not scientists, my first question is, what records were not kept? Science without thorough records is not science. Without records there can be no integrity.
I then make the observation that the Met Office’s claimed principal concern seems to be “commercial interests”. This suggests that the science is bound up in the prospect of profit, or loss if some “commercial interest” was to sue for damages. This does not promote integrity in the science.
However the public availability of slabs of the information seems to destroy the Met Office’s case anyway.
15 Trackbacks
[…] UK Met Office Refuses to Disclose Station Data Once Again […]
[…] UK Met Office Refuses to Disclose Station Data Once Again […]
[…] Meanwhile, once again the Met Office has refused to disclose the data on which CRUTEM3 is based. Steve McIntyre from Climate Audit asked for the data together with any documents describing the procedures under which the data had been quality controlled and where deemed appropriate, adjusted to account for apparent non-climatic influences. Their response is revealing. […]
[…] warming, uk met office trackback UK Met Office Refuses to Disclose Station Data Once Again by Steve McIntyre on July 23rd, […]
[…] UK Met Office Refuses to Disclose Station Data Once Again – Steve McIntyre har otur igen. […]
[…] UK Met Office Refuses to Disclose Station Data Once Again […]
[…] only do the UK Met Office predictions fail, but they refuse to disclose Station Data. UK Met Office Climate Prediction […]
[…] an wet summer? The same one that had to back off on their unhinged Arctic predictions? The one that refuses to release their raw data? The one that is throwing out yet another “we only have 10 years left” prediction? […]
[…] Neither the Climate Research Unit (CRU) at the University of East Anglia nor the British Met are able to provide their raw data to other research scientists because of the confidentiality agreements that Professor Phil Jones at CRU entered into. Unfortunately, Jones did not keep records of those agreements and, according to the British Met, can neither identify the countries with the confidentiality agreements nor provide the agreements. Earlier this year the British Met wrote the following to Steve McIntyre at Climate Audit: […]
[…] Neither the Climate Research Unit (CRU) at the University of East Anglia nor the British Met are able to provide their raw data to other research scientists because of the confidentiality agreements that Professor Phil Jones at CRU entered into. Unfortunately, Jones did not keep records of those agreements and, according to the British Met, can neither identify the countries with the confidentiality agreements nor provide the agreements. Earlier this year the British Met wrote the following to Steve McIntyre at Climate Audit: […]
[…] last summer, the Met Office had turned down my FOI request for station data, saying that the provision of station data to me would threaten the […]
[…] last summer, the Met Office had turned down my FOI request for station data, saying that the provision of station data to me would threaten the […]
[…] station data on the premise that such a request would somehow threaten UK international relations. UK Met Office Refuses to Disclose Station Data Once Again Climate Audit But a month later after Climategate, station data are released en masse to the public. Met […]
[…] the Met Office FOI thread, he’s seen giving advice on how to proceed: […]
[…] their refusal letter, the Met Office described adverse consequences of disclosing CRU station data, an event that […]