From an emailed PRESS RELEASE on March 5, 2010
Climate scientist delivers false statement in parliament enquiry
It has come to our attention, that last Monday (March 1), Dr. Phil Jones, head of the Climatic Research Unit at the University of East Anglia (CRU), in a hearing with the House of Commons Science and Technology Committee made a statement in regards to the alleged non-availability for disclosure of Swedish climate data.
Dr. Jones asserted that the weather services of several countries, including Sweden, Canada and Poland, had refused to allow their data to be released, to explain his reluctance to comply with Freedom of Information requests.
This statement is false and misleading in regards to the Swedish data. 
All Swedish climate data are available in the public domain. As is demonstrated in the attached correspondence between SMHI (Swedish Meteorological and Hydrological Institute), the UK Met Office and Dr. Jones (the last correspondence dated yesterday March 4), this has been clearly explained to Dr. Jones. What is also clear is that SMHI is reluctant to be connected to data that has undergone “processing” by the East Anglia research unit.
STOCKHOLM INITIATIVE
Göran Ahlgren, secretary general
Kungsgatan 82
12 27 Stockholm, Sweden
===================================
They included attached PDF files. which I have linked below. Click for PDF files:
Request_from_Professor_Phil_Jones_regarding_the_release_of_data_from_the_HadCRUT_dataset__dnr_SMHI_
Climate Audit 
222 Comments
“What is also clear is that SMHI is reluctant to be connected to data that has undergone “processing” by the East Anglia research unit”.
LOL! That is too good!
Francis, What you and Anthony (and everyone else) seems to have missed here is that the question posed to Dr Jones was did you have problems releasing the data because some countries didn’t want it released. The context of their question was before the Climategate emails became public. The letter from Jones was dated after Climategate broke. He pulled a fast trick on everyone and everyone, including Anthony, fell for it.
The point is that SMHI NEVER told Phil Jones that their data was not to be made available. There were no agreements not to release it with SMHI, so there was no FOIA issue as claimed by Jones (and covered by CA previously). Requesting that data that was “value added” (i.e., adjusted in who knows what manner) by CRU should not be attributed to SMHI as their data is a no brainer.
Jones trying to cover his FOIA derriere after the fact makes no difference as regards his ethical shenanigans–either before or after Climategate became public. He doesn’t merit being called a scientist when he still shows every intent to continue following Team rules and be less than fully factual.
It is clear that SMHI told Phil Jones not release their data because it was not really their data, it was their data with his adjustments. But that is still missing the point… telling Steve Mc or anyone else years ago that they could not have the data due to an agreement with an individual national weather organization was the basis of the question… Jones dodged the question and claimed that all of these countries denied publication of their data but forgot to mention that those responses were to his recent letter to them and had nothing to do with the question.
I had misunderstood the main thrust of your original point. I couldn’t agree with you more. Unfortunately, the “consensus” refers only to the rules by which The Team will continue to operate as in the past. My guess is that the only change is that they’ll save their incriminating e-mails for their private accounts from their home computers in the future. The more things change the more they stay the same.
Since FOI includes a legal process, even those emails can be subject to the subpoena process. The same goes for the legislative process.
On the other hand, I’m sure we would both agree, if Phil Jones would have answered the question, he would have put his head in his hands and cried for mercy and proclaimed something like this:
Sir, I couldn’t give Steve the data because he would find something wrong with it… Just like he did with NASA… and I would have looked like a bozo or a moron, you know, all those names Mike and I liked to call him.
Authentic gems, these documents. The one under the link “Data_from_the_HadCRUT_dataset_100304”, dated 4 March 2010, is IMHO the most important of the three. Dr Jones may be starting to feel increasingly and uncomfortably lonely.
But we’re supposed to feel sorry for the poor, beleaguered climate “scientist”. Bollocks.
We are not supposed to centre our comments on individuals and morals, or to use this blog to vent feelings, but to concentrate on facts that get unfolding and being revealed about how climate science has been conducted lately. The Swedish Met Office, for one, had clearly shown what their position was, and made it even more clear in its latest and very recent message to Dr Jones.
Nicely put.
The smoking gun in the letter by Phil Jones asking SMHI to release their data AFTER CRU has fiddled with it.
Jone wrote: “It is therefore highly likely that the version we hold and are requesting permission to distribute will differ from your own current holdings”
Let me rephrase this, “You don’t REALLY want to have your reputation damaged by us releasing data we have fiddled with, do you.”
It looks like Jones was deviously trying to get SMHI to bless his fudge and they recoiled from the trick.
Claiming refusal to release these data as respect for proprietary interest seems a devious misrepresentation.
If you had bothered to read the first PDF, the Swedish Met office said they did not want Jones to release the data. This makes the story, well, nothing. Jones was right, you are wrong. Admit it.
Not exactly.
1) Jones asked the Swedes if CRU ‘homogenized’ data could be attributed directly to the Swedes.
2) The Swedes said no to this request.
3) On the basis of this ‘no’ in a different context, Jones and Acton conveyed the false impression to the parliamentary inquiry that the Swedes would not allow the release of their raw data.
4) The Swedes complain in the press release about being misrepresented.
This is clearly documented (assuming the documents are genuine).
1) No he didn’t.
The request letter is asking for permission to release the underlying HadCRUT data for the SMHI stations that CRU previously had received monthly data for, also acknowledging that CRU’s data isn’t the same as the SMHI data.
2) They said ‘no’ to the request to release data. Reading the latter letter they might have assumed some kind of incorrect attribution, but that’s not really relevant, they said ‘no’ anyway.
“SMHI has in a letter, dated November 30, from UK MetOffice, received your request regarding the release of data from the HadCRUT dataset. [..] SMHI do not want the data to be released on your web site.”
3) No false impression there, the letter clearly states that “SMHI do not want the data to be released on your web site.”. Again, maybe SMHI misunderstood the original request, but presenting their release denial as something else would be giving a false impression, not presenting the obvious denial as a denial.
4) It’s a nice way of saying “sorry, we didn’t mean to be rude but now when we see the mess we got ourselves into we’d be happy to help (but it wasn’t our fault, we were just being misinterpreted by you all” 🙂
Anyway, they gave CRU the release acceptance and CRU is free to release their own modified data along with links to the original (modified?) data by SMHI. So let’s be happy for that.
Actually, the Swedes make their position quite clear. You seem to have neglected to quote the stated basis for their refusal. They say, “[g]iven the information [from Jones’ letter] that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold …[we] do not want the data to be released … .” That phrase, the likely difference between CRU’s version and the SMHI raw data, is the basis for the denial. They do not want adjusted data attributed to them, even if the “adjusted” data derives from their own raw data. They seem to be saying that they understand what Jones asks them for is permission to attribute the modified data to SMHI. Very few scientists would be willing to have data that is not their own attributed to them.
You are arguing about the reason for refusal, which might well be, as you say, a misinterpretation (or possibly even correct interpretation) of Jones’s request. It doesn’t change the fact that they did refuse. If they had wanted to accept with a specific attribution condition, they would have done so, but they didn’t.
There is no mention, however, of attribution in Jones’s request. The request is for releasing “Data from the HadCRUT dataset […] under your present day purview […] relating to your stations”.
Sidenote: I’m just speculating here but “Under your present day purview” might mean that to receive the original data (maybe not with SMHI, but the same letter was probably sent to many countries) CRU had to agree to some terms of use which covered the use of the original as well as any derived data based on the original and that’s the reason they have to ask for permission to release their own derived (or altered) numbers from the holder of the original data. Someone here surely knows how it really works? 🙂
This is all a bunch of hokem; Phil had a scientific responsibility. With the FOI request, he attained a legal responsibility as well.
Posting the information to a website has nothing to do with it, and isn’t a valid defense.
He had an obligation to provide the raw data, his homogenation methodology, and the resulting homogenized data. Whether the SMHI didn’t want his homogenized data posted on his website is completely irrelevant and his argument that he was restrained in any way from releaseing public domain data is obtuse.
This man needs put out of science and possibly put into jail.
As the second PDF, dated March 4 2010, makes clear, SMHI has no objection to CRU releasing their holdings, provided it is made clear that CRU may have altered the numbers (as CRU admitted in their original request) and that these are not the raw data. So as of March 4, Jones (or Acton’s) statement of March 1 is no longer valid.
However, as of March 1 Jones or Acton could not have seen this letter, and could only have based their statement on the first PDF, dated Dec. 12, 2009, which does seem to bar them from posting the data on their website even with disclaimers.
It’s pretty obvious that Jones and Acton’s interpretion of the first letter led to SMHI’s prompt retraction of the impression given in the first letter.
I think the Swedish group’s press release headline, “Climate scientist delivers false statement in parliament enquiry”, is therefore overstated, given the subsequent timing of the contradictory letter.
Also, Anthony’s headline, “Phil Jones called out by Swedes,” gave me and perhaps others the wrong impression that it was the Swedish Met office that had called Jones out, and not what is apparently a private Swedish group.
Hu – if you check what the Swedes originally said ‘no’ to in December 2009 (this was a request by Jones the last pages of the DOC111209 pdf) you will see that it very clear that they were saying no to the distribution to their data after the CRU’s homogenization.
The Swedes were surprised to find that this was turned into ‘we cannot release the raw data’ for the parliamentary inquiry. This is where Jones and Acton misrepresented what the Swedes had said.
I think this does all come down to the first letter, myself, and Jones’ bad grammar and attitude. Jones appears to have been asking for permission to release his work, based on the Swedish figures as well as others. It does, though, read as if he wants to release the Swedish ‘underlying data’.
The bad attitude part is that clearly the Swedes thought ‘underlying data’ meant the actual underlying data, and not the abstracted, processed data that Jones considers to be raw material.
I don’t think we need to assume any malice here. Jones thought he’d made a reasonable request that had been turned down. He didn’t realise that he’d totally bollocksed-up a simple letter, and that the response he got wasn’t to the question he meant to ask.
As I keep saying, assume that anything that can be explained by simple incompetence has no need of a more complicated or cynical explanation.
It was a piece of incompetence that worked out rather conveniently for Dr Phil, though, didn’t it? When we have earler read of his ingenious suggestions for frustrating the FOIA requests, in this case it doesn’t require the mind of a swivel-eyed conspiracy theorist to give rather less weight to the snafu principle, and rather more to the possibility that he knew exactly what he was doing.
“It was a piece of incompetence that worked out rather conveniently for Dr Phil, though, didn’t it?”
Is that an example of confirmation bias, or is there another name for it? If it had proven inconvenient to him, he’d have been far more likely to notice that there was an error.
To my mind, there is a common thread running through every Jones incident: he’s basically an idiot. I’ve seen little which in a broader context indicates malice on his part, but plenty that shows that he can’t get anything right. As the old saying goes, his wife must go on top, because he can only screw up.
It really doesn’t stretch my credibility at all to believe that you don’t need to be very stupid to fail to see the consequences of your actions. Everything Jones has done is consistent with believing in his own work (because he’s too stupid to see the flaws in his methods–consistent with choosing those methods in the first place because he thinks they’re right) and failure to understand the consequences to his credibility of not following the true scientific method.
An episode like these letters doesn’t contain enough information to analyse it fully. You have to look at context and past history to work out what Jones was trying to say, let alone to ascribe motivations for him saying it.
ZT:
Here is a link to the the terms of a license which must be agreed to before the data can be downloaded. It specifically, in section 3.2, prohibits the licensee from distributing the raw data to third parties although it is OK to redistribute processed data.
The letter of March 4th put the CRU under the terms of the current license agreement for non-commercial usage. Jones was correct in his answers given on March 1st and Watts and McIntyre owe him an apology. We’ll see if that apology is forthcoming. I’m not holding my breath.
I would also point out that this is unlikely to make Steve or anyone else who has been clamoring for the data happy. Only the homogenized and QC’ed data can be released directly by CRU. For the raw data one must still go to the SMHI. This is a similar restriction to that placed on data access by Environment Canada and several other national met services. For whatever reason, mostly because they sell data to commercial forecasters, they wish to maintain control over the dissemination of their raw data.
Re: Rattus Norvegicus (Mar 6 21:39),
You are incorrect in your interpretation of the SMHI restrictions and the restrictions placed by Enviro. Can. You fail to notice that maintaining control over the dissemination of data is completely different from the licensing policies, especially wrt published scientific work. UEA/CRU had ample opportunity (years) to clear up any of the supposed technicalities wrt their contractual obligations and failed to do so. Instead they chose to “hide behind” them.
Jim, or is that Cow, chip, the maintenance of the dissemination of the data is explicit in the terms of the SMHI license agreement. I know you have to use Google translate to read the terms, but legalese translates amazingly well.
As far as terms regarding published scientific work, the SMHI terms don’t have any exceptions for that unless the released data has been subject to further processing. The terms for the release of data which is unavailable w/o payment to Environment Canada are unclear. Clearly the terms in the linked agreement farther down in the discussion are at odds with the request for payment. I suspect that the license terms for restricted data are different, but I have no way of knowing this at this time.
I “suspect” you are dead wrong. Jones’ doctored data is not the main thing they were after. What was wanted was the raw data and what Jones et al did to produce the doctored joke his product has become. If Jones was doing science by any normal definition this information would have been availoable to the public at the time of the original publication and we wouldn’t be having this discussion at all.
Hu:
Please reread the original response where SMHI says not to release data because it does not match the original data. They state the data is available on their site for non commercial use. So they needed to call out Jones for false statements. False by omitting the reason for refusal. Jones admitted that he had fiddled with the data in his original request!
I may be wrong but, I watched Phil’s testimony. I do think he mentioned that certain countries didn’t want him to release data that was already publicly available. This must be what he was refering to.
What we should be interested in is looking at his data and comparing it to the original swedish data. Then we can see what changes he made to the raw data.
Having read the letter a couple times I would say both the Swedes and Jones can claim to be interpreting the Dec letter fairly because the wording is sufficiently ambigious that a reader has to fill in the blanks based on the their own preconceptions.
The scam is over mate. Move on
To see how wrong Rattus is skip down to the post by Hoi Polloi Posted Mar 7, 2010 at 9:54 AM.
Rattus, you are completely wrong. They didn’t want Jones releasing data that he had fiddled with and claim that it was SMHI data. The second pdf clearly states that this was their intention. They clearly state that their data should be publicly available and they ask him to provide links to their data. They just don’t want anyone to think that their data actually resembles anything that he would be presenting.
Exactly. Beyond that, this is long after intial FOI requests, and, correct me if I am wrong, has nothing to do with and does not prove any intial non-disclosure agreement, which if lacking then permission was not needed in the first place.
They just didn’t want their data being affiliated with CRU’s website. They had no problem with CRU providing a link to the SMHI site, where people would know they were getting the real raw data.
Acton’s remarks were totally disingenuous.
Chris, you are wrong. Look at the terms of the license agreement for non-commercial use. There might have been a language gap here, but the letter of the 21st of December was pretty clear. In it they were placing the CRU under more stringent terms than the standard license agreement for non-commercial use. The letter of March 4th placed CRU under the normal terms of the agreement.
>”Subsequent quality control and homogenisation of these data have been carried out.<"
Quality control? Of someone else's data? I don't understand such language.
Does that mean: If we found your data to be incorrect, we applied the necessary adjustments to correct it?
Has Prof Jones written in the same terms to all the national met authorities? If so, have they all noticed the precise terms of the letter as have the Swedes? When/if CRU release the data for all countries, will it be pre or post homogenisation? Will this be clearly stated?
Of course he has. One wonders why other countries agreed to him releasing fuddled data. Co-conspiritirs is a term that comes to mind.
Peter, I think you have hit upon the key point. If the various Met Offices agreed to the terms of this letter, then CRU will be able to release the fiddled-with data and claim ‘this is the data that the Met offices have authorized us to release’.
I did an FOIA on this a while back.
I have the orginal request letter which was an attachment to the lts mail in the
stack.
Taken before the Science and Technology Committee
(Science and Technology Sub- Committee)
on Monday 1 March 2010
http://www.publications.parliament.uk/pa/cm200910/cmselect/cmsctech/uc387-i/uc38702.htm
Q94 Chairman: Why did you hide it then?
Professor Acton: Unfortunately, several of these countries impose conditions and say you are not allowed to pass it on, so there has just been an attempt to get these answers. Seven countries have said “No, you cannot”, half the countries have not yet answered, Canada and Poland are amongst those who have said, “No you cannot publish it” and also Sweden. Russia is very hesitant. We are under a commercial promise, as it were, not to; we are longing to publish it because what science needs is the most openness.
STill, Acton did not say Sweden would not let them release CRU fiddled Swedish data. Neither did Jones. Clearly they both tried to give a FALSE impression about the data.
Mr Grisanchich,
According to themselves, CRU is not in possession of the raw SMHI data, so they cannot be asked to release data that they don’t have. They, however, seem to want to release data they (according to themselves) have, hence the request was sent to (among others) SMHI for permission to release CRU’s own ‘fiddled data’ that is apparently based on the monthly data they have received from SMHI.
I can’t imagine why they aren’t keeping all the originals somewhere safe ‘unfiddled’ (maybe there is some strange but good reason?), but I think your impression about what impression they gave is a bit misinterpreted?
Clarifying myself here: with “raw SMHI data” I meant the actual station measurement data– CRU may well be in possession of the original SMHI monthly mean station data but the request to release was for the ‘fiddled’ version of that.
Hang About. Who is the Stockholm Initiative:
About us
Our time is characterized by a continued rapid development of the global economy. The key factor enabling previous and continued growth is access to efficient energy sources. At present, 80 % of the global energy supply is provided by fossil fuels.
For more than 20 years, a few dozen researchers, but above all, politicians and media, have spread the notion that carbon dioxide emissions will cause a global climate catastrophe. Politicians now regard this as a fact above discussion. The issues are now what measures should be taken to reduce and eliminate such emissions with the purpose to control the global climate.
However, climate and weather patterns do not follow the predictions. Moreover, new research clearly points to the fact that the ongoing and planned policy measures will have none of the desired effects on the climate.
Still, such measures are now being considered for global binding conventions, and involve political control over energy sources and markets. This will cause catastrophic side effects on the global economy and the environment as have always been the consequences of command economy measures, clearly demonstrated by actions already taken. For that reason, it is of utmost public interest to thoroughly examine the foundation of the present climate policies and their economic and environmental consequences.
The Stockholm Initiative is a non-profit organization dedicated to such an examination. Our ambition is to promote climate-, energy-, and environmental policies based on what we know rather than on what we don’t know
—————-
Points for media
•Contemporary climate changes are neither unique nor alarming.
•No relation between carbon dioxide concentrations and climate changes has been verified.
•There is no scientific consensus regarding the role of carbon dioxide as a dominant climate driver.
•The climate models on which the climate alarms are based are of no value for making predictions.
•Ongoing and planned climate policies have catastrophic side effects such as environmental damage, starvation and poverty for the world’s poorest
Why didn’t The SMHI release these documents.
Their referenced pages for data cover 1961 to present
Jones data goes back to 1800s and has taken 30 years to collect
Hmm! Strange stuff.
As Rattus says the document completely backs up Jones claim that data release had been refused:
“SMHI do not want the data to be released on your web site.”
I do not think that could have been clearer!!!
Following this with a letter on 4th March saying what they really meant to say is no help after the hearing has happened.
TFP:
You seem to have missed the original reply stating that the adjusted data was not to be released and referring to a SMHI web site that is under construction for the raw data.The rest of your copy and paste, while true, have little to do with this issue. SNMI DO NOT WANT “Corrupted” data released from CRU.
Of course some realize what the first reply actually meant.
“Hang About. Who is the Stockholm Initiative:”
Doesn’t matter. They’re playing a similar role to CA in all this. The letters in question are to/from the Swedish Met Office equivalent. All the Stockholm Initiative has done is draw attention to the issue at hand. You can question their motives for doing so, but that doesn’t dismiss the issue.
The Stockholm Initiative have at least one name that should be familiar to the readers of the Climategate letters: Wibjörn Karlèn.
Another name is Nils-Axel Mörner; the founding father of sea level studies.
They are serious people.
This underscores the true value of the recent hearings. Despite the lack of aggressive followup questioning in some areas, the overall result is that Jones and CRU were forced to decide on a positions and state them for the record in a very public way. That opens the door for very clear and public rebuttals from other parties (in this case the Swedes) thereby slowly and convincingly exposing the problems with the stated CRU positions.
The more precisely and publicly Jones and the CRU (and, for that matter, any “experts” claiming authority on the climate science debate) are forced to publicly and precisely select a position and defend it, the faster it will get precisely and publicly rebutted by those in a position to do so.
That, combined with the decision to
characterize some of the CRU’s output as a ‘product’ and thus broach the issue of ‘product liability, is sure to lead to some interesting timed ahead.
There seems to be some confusion – so here is a more detailed summary.
1) In November 2009 Jones asked the Swedes if CRU ‘homogenized’ data for their stations could be distributed and attributed directly to the Swedes. “Subsequent quality control and homogenisation of these data have been carried out. It is therefore highly likely that the version we hold and are requesting permission to distribute will differ from your own current holdings.”
(see http://wattsupwiththat.files.wordpress.com/2010/03/doc111209.pdf)
2) In December 2009 the Swedes said no to the Jones request: “Given that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site”.
(see http://wattsupwiththat.files.wordpress.com/2010/03/request_from_professor_phil_jones_regarding_the_release_of_data_from_the_hadcrut_dataset__dnr_smhi_.pdf)
3) In March 2010, on the basis of this ‘no’ in a different context, Jones and Acton conveyed the false impression to the parliamentary inquiry that the Swedes would not allow the release of their raw data.
(see http://www.publications.parliament.uk/pa/cm200910/cmselect/cmsctech/uc387-i/uc38702.htm)
4) After this misrepresentation, the Swedes complain in a letter about being misrepresented.
(see http://wattsupwiththat.files.wordpress.com/2010/03/data_from_the_hadcrut_dataset_100304.pdf)
While I would agree that the letters are about Jones trying to get the Swedes to give his monthly massaged data official sanction, something else is also revealed here.
The something else appears to be a rather shifty move by CRU to release their massaged data to the public with some claim that it is some kind of “official” version of the “raw” data.
Do you guys research anything that gets published on this website? Clearly, Mr McIntyre has a gigantic axe to grind against Jones but regardless:
1) They contacted the countries regarding release of the data.
2) They released most of the data.
3) The data that was not released was public domain (at least mostly)
4) Jones has said that some countries had problems with him, putting the data on his website. Exactly like how Swedes say in there letters. See http://www.youtube.com/watch?v=pBInhAVeixk for the first two minutes. So this is a non-issue.
Re: Lotharloo (Mar 6 03:13), Rattus Norvegicus (Mar 6 00:26), thefordprefect (Mar 6 01:12),
You all missed the pea under the thimble.
“The data” means different things in each of your statements. Was it the raw data, adjusted data, or combined.
I’ve seen this in other contexts. One set of parties is most interested in raw data. The other set of parties is interested in adjusted data. They communicate assuming the other party shares the same interests. Bad assumption. In one case, I saw miscommunication continue literally for two decades before things were sorted out.
In this case, if we add the appropriate tag (“raw” or “adjusted”) to “data”, it all becomes clear:
The Swedes were very happy to make their (raw) data public. And are happy for others to reference their (raw) data.
Jones values his (adjusted) data more than the Swedes’ (raw) data.
The Swedes do not want to be associated in any way with Jones’ (adjusted) data.
Jones used Swedish reluctance to be associated with his (adjusted) data, to claim they were withholding the (raw) data that had been requested.
Note the Swedes were very clear in their first letter that they didn’t want their (raw) data published on Jones’ website. They in no way prohibited release of the (raw or adjusted) data, only prohibited Jones from placing it on his website! And, they were clear that the issue was Jones’ non-maintenance of their (raw) data sets. They had no confidence that he’d actually release their (raw) data.
It’s a huge issue.
Re: MrPete (Mar 6 20:18),
I’ll add that the “data” has been interpreted by the check-kiters as being what is referred to as “metadata”. Some FOI requests were merely for the station ID’s so that the relevant raw data could be obtained (even from individual NMS).
Their ambiguous uses of the term “data” is part of the team’s common subterfuge. Call it all “data” and refuse to release any information using any excuse to “hide behind”.
Re: Jimchip (Mar 7 12:05),
Completely true.
As far as I know, 100% of the FOI’s regarding metadata were initiated after FOI’s for the real data were denied…in an attempt to find another way toward the data.
Thus, one could say:
a) If the raw data had been released, there likely would have been no data-FOI’s.
b) If the raw data had been released on data-FOI, there likely would be been no metadata-FOI’s
The only reason the FOI’s multiplied was stonewalling.
Lotharloo:Agreed whatever we think there from the correspondence there is scope for UEA misunderstanding the Swedes. But in some senses this is a moot point, what was required was the raw data, the data Jones used to produce his papers. Apparently it’s been lost, but Jones didn’t tell anyone this until after Climategate broke. Why?
Hazarding a guess it means that any paper published using the Hadcrut data set should now be withdrawn because they’re not verifiable doesn’t it? If that’s the case there is a scientific scandal of immense proportions. But who knows what passes for normal in the whacky world of climate science?
The issue is that you need to know exactly which data sets were used. Its like a chemist says ‘all of the elements used to create my new compound can be found in the periodic table’, — well you sorta, kinda need to know which ones (along with the exact method) to reproduce the compound.
We believe that he has very little confidence in his own data because he agreed he wrote “Why should I give information to you when all you want to do is find something wrong with it?”
But of course everyone else is supposed to have confidence in these value additions.
CRU should publish their code and the raw data. The raw data is by y definition equal to the data published on the Swedish site. All massageging/homogenisation/etc should be clear in the code CRU should publish, but don’t publish. And don’t publish because it is not standard in “climate science”, but who defines standards in climate science? Pretty much CRU?! If not alone certainly together with IPCC, they could have introduced that standard.
This may or may not be important, but i cannot understand why Phil jones statement that “peer reviewers have never asked for any information” from him, has not been picked up as important. It does seem very strange to me, but maybe i am missing something.
Re: Peter Oneil (Mar 6 03:59), The follow-up question should have been: “If the peer reviewers cannot check your science and methodology because they don’t have access to either your data or code, then just what are they peer reviewing? Your spelling and grammar?”
You’re confusing Peer Review with Climate Science Peer Review. It’s similar to the distinction between Science and Climate Science, in that the General Public who have no interest in checking up would imagine the two versions are carried out to similar high standards.
That Jones would not have had such requests is probably because he has been peer reviewed by internationally renowned colleagues with whom he is quite familiar, and whose work he in turn may well have been involved in peer reviewing. They all know each other and trust each other to do a good job. The right sorts.
Climate Science Peer Review is generally there to provide a fresh set of eyes to catch the odd spelling mistake, identify a lack of clarity in the odd sentence, spot an incorrect reference, make sure the formatting and fonts look good, and most importantly to give friendly colleagues a well deserved pat on the back.
It is only when a paper under review has the wrong conclusions (and when despite having given it the thumbs down a journal still thinks they might publish some shocking piece of work) that it becomes necessary to dig a bit deeper and discredit the science fundamentals. (This may or may not include reading the paper if it comes from certain authors.) However, showing that the paper conflicts with one’s own or ones friendly colleagues eminent peer reviewed science is usually sufficient to achieve that result. Ergo, no need to ask for codes or data.
This isn’t just science, this is Climate Science.
I don’t want to take this off topic but I do want to plant a big flag on the question asked by Peter Oneil.
If peer reviewers never asked for the information it has therefore never been peer reviewed …..
“Given the information that the version of the data from the SMHI stations that you
hold are likely to differ from the data we hold, SMHI do not want the data to be
released on your web site.”
What do the defenders of UEA candor make of this? It is plain and simple. They are refusing, and the reason for their refusal is that the data is likely to be different from the data they hold. They do not discuss why it is different. They simply say that you should not release it, because it is on your own account, different.
So the question to be asked is also very simple. Did UEA, in their communication with the Committee, make clear that the Swedes had refused because the material which it was proposed to release was different from the material they, the owners of it, held?
Or was it said or implied that there was some other, or some unknown, reason?
The answer is yes. Professor Acton: Unfortunately, several of these countries impose conditions and say you are not allowed to pass it on, so there has just been an attempt to get these answers. Seven countries have said “No, you cannot”, half the countries have not yet answered, Canada and Poland are amongst those who have said, “No you cannot publish it” and also Sweden.
They did not say ‘you cannot publish it’. What they said was, you cannot publish something which is different from it as if it were the same as it.
The interesting thing from a social point of view is how Rattus and Ford react to this. Rattus makes a clearly false claim, that UEA was right to say that the Swedes had forbidden release of the data. They had not, and this was, as it stood, at a minimum a seriously misleading thing to say. They had forbidden release of some other data, attributed as their data, when it was not their data.
Ford rather than consider the textual evidence before him, which clearly shows this, veers off into the question of who supplied the textual evidence, what other positions do they hold, and at some point will emerge triumphantly saying, you see, they are bad parents, have poor personal hygiene, and are funded by Exxon. So that’s all right then, it does not matter what is in the evidence. If its been put about by people with poor personal hygiene, we do not have to consider it.
It is a bit like rejecting gravity and calculus because Newton sometimes skipped baths.
But you see in a nutshell why climate science and AGW are falling into disrepute. One reason is that people keep denying the evidence when it is plain in front of them. The second reason is that they decline to discuss the evidence, instead discussing irrelevancies. So what one would say to Rattus and to Ford is: you are doing more damage to your cause by this style of argument than the opponents can do. You would do better to just admit that UEA was economical with the truth, regroup, and move on. As it is, you make an impression of being, well, sorry to use the expression… in denial!
“They did not say ‘you cannot publish it’. What they said was, you cannot publish something which is different from it as if it were the same as it.”
Yes, they have said that.
See http://www.youtube.com/watch?v=pBInhAVeixk for the first two minutes
Yes, they have said that.?
WHO have said WHAT.
You point to a video of jones,
You have jones testifying to what the canadians purportedly said.
But play that video on and then read my piece on Jones’ lies
Re: michel (Mar 6 04:04),
Sweden: UEA claims the Swedes said “No, you cannot [pass on the data]”. Those circumstances are now public. UEA’s claim was sufficiently misleading for SMHI to rebuff it.
Canada: UEA claims the Canadians said “No, you cannot [pass on the data]”. Those circumstances are not clear.
Poland: UEA claims the Poles said “No, you cannot [pass on the data]”. Those circumstances are not clear.
Country 4: UEA claims that Country 4’s Met Office said “No, you cannot [pass on the data]”. That country’s identity is unknown and the circumstances are not clear.
Country 5: as for Country 4.
Country 6: as for Country 4.
Professor Acton:
* Would you kindly agree to the release of the correspondence between CRU and Canada on this question?
* Would you kindly agree to the release of the correspondence between CRU and Poland on this question?
* Would you kindly identify Country 4 and agree to the release of correspondence?
* Would you kindly identify Country 5 and agree to the release of correspondence?
* Would you kindly identify Country 6 and agree to the release of correspondence?
Regarding Sweden, Jones request was not to “pass on the data”, it was to release data from (CRU’s) HadCRUT dataset that was related to the SMHI stations. SMHI might have misunderstood that and therefore changed their refusal to a qualified acceptance in the latter letter.
From a previously unpublished CRU mail.
Dec 1, 2010
>
>Our parlimentary adviser has asked me to inform you that we have started
>sending and got as far as the letter M. The request letters are
>therefore now likely to be in the public domain.
>
The letter from P D Jones to the SMHI was sufficiently confusing that the SMHI may well have believed that Jones was proposing to release CRU gridded date which had been mixed with other data as if it were SMHI raw data. Read the last two pages of DOC111209.PDF as if English were not your native language.
If you consider that Jones had vested interest in receiving confirmation that certain country’s data was not allowed to be released, then it does present the possibility that Jones had a motivation to be ambiguous in his correspondence with SMHI and others. I am not accusing Jones of this, but it does make me wonder.
Nowhere in Phil Jones’s letter does he request that raw data from sweden be released, He stresses that any data that he would release would be adjusted in some way. That is the data for which approval was refused.It is not possible to claim that the Swedes refused when they were never asked in the first place.
Peter, your point is THE point, precisely.
thank you very much. I am not alone,
Only those wanting to cloud the issue are arguing otherwise.
On a point of order, I don’t think it’s justified to universally ascribe opposing views to malice. More to the point, it’s neither constructive nor dignified, and, as such, is a net drain (in a very small way) on CA’s credibility.
What malice is being refered to?
Deadwood’s comment that I referred to basically says that anyone who disagrees with X view is doing so because they wish to obscure the truth. There is no reason to think that people are malicious merely because they disagree with you.
Phil Jones is not personally responsible here. CRU have only 3 staff. The FOI officer is not one of them. The UEA have responded through FOI that the information was not available because of agreements which would appear not to have existed.
Where is the letter/email to the Swedes informing them that the release was required due to an FOI request.
To me it is not believable that the Swedes have not been aware of circumstances and ramifications until now. They are covering there rear ends and Jones is the fall guy. Not that I have much sympathy for him
CRU actually shows 3 full-time and 1 part-time support staff.
http://www.cru.uea.ac.uk/cru/people/
However CRU is an integrated part of the UEA Environmental Science faculty which has a considerable number of other support staff who can be drawn upon, exactly as they are in normal day-to-day CRU operations…
http://www.uea.ac.uk/env/people/People/Administrative+and+Support+Services+Staff
You don’t mention of course that Mr. Ahlgren has nothing to do with SMHI and instead belongs to a Swedish denialist group that is merely putting their spin on the SMHI/Jones correspondence.
Oh shucks, Libby. All of this time wasting and stalling while the planet doesn’t warm and the ice again thickens in the Arctic. Your days of harrassing skeptics are numbered… I guess you’ll be forced to go back to the Bigfoot cover-up.
What is the time-line here? I though Prof Jones voluntarily stood down from his position at CRU until the completion of the inquiry. If so why is he still writng letters on CRU letterhead as if he was still the boss and on active duty? Has he actually been reinstated without any notice to the public? (what was the line from Blackadder: “Ive got a plan so cunning you could put a tail on it and call it a weasel.”
Snip me entirely if you think this is too harsh.
he stood down as director. He is still employed by them. So, amazingly, he is still working.
The letter was written on letterhead that says:
Climatic Research Center
Professor P D Jones, Director
Jones initial letter is undated and addressed to Dear PR with no company or organisation or their address. We do not know it’s date.
That letter is forwarded by John Hirst, of the met office to SMHI on the 30th of Novemnber.
It was announced on the 1st of December that Jones had, on that day, stepped aside as director of CRU.
There does not appear to be any inconsistency here.
The letter was written in early november. If you read the last email
in the climategate files you will see a mail between jones and thorne.
that mail has an attachment. I surmised that this attachment was the request.
That attachment is the letter send out. ( I have other mails not in the climategate files )
A file of 172 countries was prepared by Mike Salmon. individual mails were send out.
Re: Steven Mosher (Mar 7 03:37),
From Thorne: “Shorter, simpler, requests are more likely to get read and
acted upon was the specific advice from international relations.”
It looks like it didn’t work out well for CRU as far as the SMHI is concerned.
“Given the information that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site.”
This is the key sentence from SMHI. Jones should have answered on the hearing: “I was not allowed to publish data other than the original SMHO data.” He did not and “homogenized” the SMHI quote. Seems like he’s a pathologic homogenizer…
The SMHI data is not all available on teir site as THEY need to homogenise it from ancient paper records. Jones did this as he wanted the data – a reasonable situation
Re: Hoi Polloi (Mar 6 07:12), Serial as well!
“Mr. Ahlgren has nothing to do with SMHI and instead belongs to a Swedish denialist group..”
BCL, WTF does what group he belongs to have to do with anything? What we, the rest of us, are talking about, is what is in the letters which have been made public. Do you think that Newton’s personal hygiene had anything to do with how useful calculus is?
Not by the way that there are any denialist groups. This is just a silly and perjorative way of describing people who differ from you. There are no witches either.
The press release by Göran Ahlgren, secretary general, of the STOCKHOLM INITIATIVE is “false and misleading in regards to the Swedish data.”
FALSE: All Swedish climate data are available in the public domain.
From the 12/21/09 SMHI letter “The datasets are for a shorter period, but the web site is under development and more data will be available for download in the near future.”
MISLEADING: What is also clear is that SMHI is reluctant to be connected to data that has undergone “processing” by the East Anglia research unit.
None of those words come from they SMHI letters and that statement mischaracterizes a neutral reading of their letters.
From 12/21/09 “Given the information that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site.”
SMHI clarified their position on 3/4/10 by writing, “It has never been our intention to withhold any data but we feel that it is paramount that data that has undergone, for instance, homogenisation by anyone other than SMHI is not presented as SMHI data.” They go on to request the data be properly described and that links be provided to SMHI’s version of the original data.
The value of this press release is the that letters are interesting. They demonstrate that there is an agreement, oral, written or gentlemens, not to disclose the data without permission. This confirms an element of Jones’ claims.
They further demonstrate that he was happy to have those agreements as an excuse to willfully avoid his obligations under the FOI rules. Attaining the permission to release his version of the data, apparently the only version that he retains, required but one letter (and a bit of testimoy before a MP committee). Hardly the burden he repeatedly claims.
The press release, however, deserves a [snip].
Ok, only partially false on the first. The data from the stations Mr. Jones requested was there, just it did not currently cover the entire period, but soon would.
As to this being MISLEADING: What is also clear is that SMHI is reluctant to be connected to data that has undergone “processing” by the East Anglia research unit.
Your statement…
“None of those words come from they SMHI letters and that statement mischaracterizes a neutral reading of their letters“, is misleading, a characterization is not a quote, the characterization is accurate.
What part of “Given the information that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site.” do you not understand as saying we do not wish you to release the data as our data, given that you have changed it. The press release is very accurate, and the 3/4/10 letter confirms it. No sniping required.
And where is the original non-disclosure agreement?
C’mon – stating that SMHI was “reluctant to be connected” with data that had been “processed” is pejorative beyond what can inferred from those letters.
The first letter is almost a form response conveying not a single explicit statement that requires characterization. It was straight forward and to the point. The second letter was apologetic and accomodating.
“…may have been misinterpreted…”
“…the formulations may have been a bit harsh…”
“We see no problem…”
As for the agreement, we agree. And it seems safe to assume if it exists in writing it will be the subject of a future FOI (or voluntary) release.
Sir I disagree with this…”C’mon – stating that SMHI was “reluctant to be connected” with data that had been “processed” is pejorative beyond what can inferred from those letters.
I do not see excess disparaging enhancement in the commentary.
The facts are:
1) In November 2009 Jones said: “Subsequent quality control and homogenisation of these data have been carried out. It is therefore highly likely that the version we hold and are requesting permission to distribute will differ from your own current holdings.” (See DOC111209 above)
2) In December 2009 the Swedes said no to the Jones request: “Given that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site”. (see Request_from_Professor_Phil_Jones_regarding_the_release_of_data_from_the_HadCRUT_dataset__dnr_SMHI_ above)
SMHI clearly did not want the data released in an altered form and the The Swedes said ‘no’ to the request for CRU-homogenized data to be released – and Acton and Jones misrepresented that ‘no’ to the UK parliamentary inquiry (by pretending it applied to raw data).
As the second PDF, dated March 4 2010, makes the above interpretation cogent and percise, SMHI has no objection to CRU releasing their holdings, provided it is made clear that CRU may have altered the numbers (as CRU admitted in their original request) and that these are not the raw data.
I can already see why our host gets tired of refereeing disputes.
We agree that that the SMHI data isn’t entirely available and the Goran Ahlgren’s press release was at least half false in that regard.
We also agree that “SMHI has no objection to CRU releasing their holdings” as long as it is noted that the underlying Swedish data has “undergone processing” and as long as there is a link to the version of the data posted on SMHI’s website.
SMHI’s request was professional, appropriate and consistent with their obligations as keeper of the Swede’s data. SMHI’s stipulations in no way suggest a prejudice against CRU or their data.
As for misrepresentations to the UK inquiry, well that kind of discussion occurs above my pay grade.
And apologies for reverting back to DGH. It was an accident. I thought that had been rooted out of all my forms.
I think it is good to work out the process of clarification, in this we get to the heart of the matter, as you stated; “MISLEADING: What is also clear is that SMHI is reluctant to be connected to data that has undergone “processing” by the East Anglia research unit.” wheras my reading is that clearly SMHI did not want their data “connected”
(IE. represented, without clairification that it had been altered)to the East Anglia research Unit.
The 3-4-10 letter clearly puts the correct connotation to the earlier communication.
So I guess we can agree to disagree. It does take time to communicate, and that is worth the effort.
Cheers
I’m a “skeptic” but I agree that there`s nothing there in this case. This story has been mis-characterized as an official Sweedish rebuttal letter but it came from a skeptical group called the “Stockholm Initiative”.
However, some countries do not seem to trust CRU’s manipulations. I believe that Canada probably has the same position as Sweeden since their own data is also in the public domain.
The “Stockholm Initiative”, did not write the letters, they broke the story, so please rethink your position.
Re: Syl (Mar 6 08:14),
There are two related but distinct questions.
1. What information (stations / months & years / raw temp data) from Sweden, Canada, Poland, and Countries 5, 6, 7 is available?
2. What information (stations / months & years / raw temp data) from Sweden, Canada, Poland, and Countries 5, 6, 7 was used by CRU to construct their models?
The long-running dispute about transparency, traceability, and FoI requests has centered on Question #2.
Prof. Jones’ 30 Nov 2009 request to SMHI (Sweden) didn’t specifically address those issues. If it had, SMHI couldn’t have responded as it did.
Most importantly CRU didnt tell people that FOIA could trump their commercial rights. They are obliged by FOIA guidelines to inform third parties of this.
The FOIA became complete in the UK in 2005. It is a bit harsh on data providers if their data suddenly becomes available because of a change in law. Is it retrospective?
The UEA is a university. Learning is their commodity. The people at faul would be the FOI officer not Jones.
1106338806.txt
1182342470.txt
1214229243.txt
**1219239172.txt**
1226451442.txt
**1228330629.txt**
1228922050.txt
ho hum
Since the context of those hearings was about other scientists being able to engage in replication of Jones work, it is perhaps relevant to ask Steve.
Steve, could you replicate Jones work without access to the “value added” data that Jones holds that while derivative of the Swedes data, is not the same?
As for Canada, we make them come to a gov. website and charge $100 per request.
http://climat.meteo.gc.ca/newPriceAnnounce_e.html
Big City Liberal, you mean the Canadian Government actually wants to make a buck off its weather data? So it would prefer that the CRU sends people to the EC website so it can collect the $100 rather than sharing the data with them, thus preventing EC from collecting $100?
Earlier today I sent all the Swedish correspondence, together with quotes from Professors Acton and Jones to the Muir-Russell enquiry and also to the UK Science and Technology Committee, both of who are investigating the actions of CRU.
I made no particular comment, save for the observation that there appeared to be an inconsistency between the statements made by UEA and those from Sweden.
It is up to those two enquiries to determine the truth of the matter.
Makes you wonder if Sweden is the only country that has picked up on the fact that the data CRU is asking to release is not the countries data but rather CRU’s “value added” data which is not the same thing. If and when CRU finally releases the data from countries that have given permission, the question will remain as to just who’s data it is – the countries or CRU’s “value added” data.
The Swedish response of assuring that a distinction between the raw Swedish data and the CRU value added data be made sounds very reasonable to me. That logic might also be applied to other requests that CRU might have or will make of other national weather groups.
Before we get way off the topic here in offending and defending Phil Jones and CRU, I would think we should note that the current situation involves the inability of CRU to provide to the all of the historical the raw data used in producing the value added data set. Whether losing that data by CRU says anything about their abilities and attitudes is a point for debate elsewhere, if we are concerned with retrieval of the raw data and making it public for independent study and analyses.
Phil Jones has stated that he judges that the raw data can be retrieved or reconstructed with some effort. I would think that CRU, if serious about its temperature data set and its use by others, would make every effort to retrieve/reconstruct it in order to maintain its prestige and as a measure of repentance for its apparent sloppiness in losing it along the way.
I would think that this Swedish reply would provide some hints as to how that raw data might be retrieved and with how much effort. Without the raw data, I would assume that climate scientists would be much inclined to use one of the other available data sets and perhaps start using the satellite based sets more.
You wrote “Phil Jones has stated that he judges that the raw data can be retrieved or reconstructed with some effort.” I agree that Jones said that or something very similar in an email, but I doubt if the raw data can be produced by reverse engineering because of the programing problems encountered by “Harry”—Jones in his letter to the SMHI says that “…quality control and homeogenisation of the data has been carried out.” I am not sure that Jones knows what was done to the raw data, when it was done, or why it was done. I also don’t know if data can be un-homogenized.
Re: Kenneth Fritsch (Mar 6 11:18), and Robert Burns,
There’s a doc in the Crutapes called jones-foiathoughts.doc. It’s properties dates say Created: 04/23/2007, 10:38:00, Climatic Research Unit. The ‘creator’s’ “Options appear to be:” 1. Send them the data. 2.Remove data they had to pay for (no comment on whether what they had to pay for kept them from sending it, anyway.
Last but not least,
“3.Send them the raw data as is, by reconstructing it from GHCN. How could this be done? Replace all stations where the WMO ID agrees with what is in GHCN. This would be the raw data, but it would annoy them.”
Are WMO ID’s enough for reconstruction, as claimed? Why not option 1. Send the data?
Hard to believe Canada won’t share data. I wonder who they asked. I wonder what a FOI to EAU would find.
Hard to belive, because ii is not true
“Canada releases its temperature data to anyone who requests it,” Brigitte Lemay, a spokeswoman with Environment Canada, said in an e-mailed response. “We have in the past and we will continue to make our data public. All Environment Canada official climatic data is made available without restriction to the public through our Web site.”
Source:
http://www.businessweek.com/news/2010-03-01/russia-canada-won-t-reveal-data-climategate-school-says.html
“Canada has said they would rather we sent requests for their data to their Web site,” Jones said. “They don’t want it on our Web site.”
thefordprefect (Mar 6 08:41),
Do you have a reference which verifies that CRU actually transcribed the early Swedish data?
There are a couple on GISS:
Harnosand Sweden 62.6 N 18.0 E 645023610010 19,000 1880 – 1991
Jonkoping Fly 57.8 N 14.1 E 645025500003 131,000 1951 – 2010
Both magically start before 1961 I would guess that the additional data is available from the individual Met. sites.
I obviously have no idea what CRU have got, but it took years to obtain it!
Re: thefordprefect (Mar 7 18:54),
Ok, thanks. I expected that you were just referring to CRU having obtained early data, not that they were necessarily the people who transcribed it. I grant that CRU put in effort gathering the data, but so has Steve M.
So CRU objects to being the clearinghouse for worldwide climate data? That seems sensible since they clearly don’t seem up to the task. Something goes with this though. They make claims in papers that they can’t substantiate with data.
The facts are:
1) In November 2009 Jones asked the Swedes if CRU ‘homogenized’ data for their stations could be distributed and attributed directly to the Swedes. Jones said: “Subsequent quality control and homogenisation of these data have been carried out. It is therefore highly likely that the version we hold and are requesting permission to distribute will differ from your own current holdings.” (See DOC111209 above)
2) In December 2009 the Swedes said no to the Jones request: “Given that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site”. (see Request_from_Professor_Phil_Jones_regarding_the_release_of_data_from_the_HadCRUT_dataset__dnr_SMHI_ above)
3) In March 2010, on the basis of this ‘no’ in a different context, Jones and Acton conveyed the false impression to the parliamentary inquiry that the Swedes would not allow the release of their raw data. Acton said “Seven countries have said “No, you cannot”, half the countries have not yet answered, Canada and Poland are amongst those who have said, “No you cannot publish it” and also Sweden.”
when talking about the data provided by other countries, as in”…data provided by over 150 countries…”.
(see the inquiry transcript)
4) After this misrepresentation, this week, the Swedes complain in a letter about being misrepresented.
(see Data_from_the_HadCRUT_dataset_100304)
I tried to post this yesterday – but with links it was held up by the spam filter.
If the documents are genuine, it is crystal clear that the inquiry was misled by Jones and Acton.
The previous post id get through and I commented on it above 🙂
Mikael – the November 2009 request from Jones is clear (if this is what you disagreed with – perhaps you had not read it at that point?)
The Swedes said ‘no’ to the request for CRU-homogenized data to be released – and Acton and Jones misrepresented that ‘no’ to the UK parliamentary inquiry (by pretending it applied to raw data).
The climateers may be worried by this turn of events as it clearly demonstrates the fact tampering tendencies of the UEA and the CRU.
1)
I agree that Acton made it sound like they were refusing the release of the original (“unhomogenized”) data received from SMHI, so his statement is, on the surface of it, misleading in that respect.
ACTON: “its data is the copy of data provided by over 150 countries, whose national meteorological stations turn the data into the average for a month and then pass it. There is no sort of primary source here.”
ACTON: “Unfortunately, several of these countries impose conditions and say you are not allowed to pass it on, so there has just been an attempt to get these answers.”
2)
The original claim says Jones made the false assertion– this is clearly wrong (and failing to correct someone else’s misrepresentation is not making a false assertion).
3)
The original claim (and Acton!) confuses Swedish “monthly mean station raw data” with “HadCRUT underlying monthly mean station records”. The first can (apparently) be obtained directly from SMHI, but the latter (quality adjusted and homogenized version) can be released only by CRU.
It is obviously important that CRU releases the adjusted version (since this used to calculate the global temperature anomalies), but it would also be beneficial if they could publish (if they have not) the original SMHI record along with the documentation of adjustments they have made to that. If this documentation does not exist the adjustments can be checked and analyzed by obtaining the original data from SMHI and comparing that to the HadCRUT records. (Right?)
4)
I had read the request letter and I disagreed with your initial attribution argument as there is nothing in that letter that says CRU wanted to release their own derived data and attribute it to SMHI.
Mikael>
To my mind, the problems arise because the final paragraph flips its meaning depending on whether or not it’s read in direct conjunction with the preceding paragraph. My experience of Jones’ writing ‘style’ leads me to believe he meant it to be read that way. Grammatically, though, he’s indicated (by the paragraph break) that it shouldn’t be — but a lot of none-too-bright native English speakers have no idea how a paragraph is supposed to work.
It seems to me that, given the worst possible interpretation, this is at most a minor smudge contributing to the general blackening of Jones’ reputation. It’s not worth the scrutiny it’s received – presumably we’re only arguing in the absence of another post…
Mikael – I see you point of view – I think that we agree that the net effect of Acton and Jones in the parliamentary inquiry session was to shift blame and responsibility to others, and thereby mislead the inquiry.
SMHI says the data will be “downloadable for non-commercial purposes”. That does not match the meaning of “public domain” of being usable for any purpose. Being available to the public is not the same as being in the public domain.
Why is releasing (or linking) to the original data, and then releasing the changed data (and methodologies) along side it so hard? That is what anyone would do normally, and you don’t even need permission to link. This is just more smokescreen. I’m betting his data is much ‘warmer’ than the original, and I’m betting the same goes for his Canada data.
This from wuwt suggests 4.4degC/Century (don’t need to increase this!!!:
Harry Lu (10:09:44) :
OK you lucky people
Here are a couple of plots showing what the swedish met office is offering.
The top plot shows a spageti plot of most stations with 1961 to 2008 data intact.
The second averages these all togeter
Note that Upsaala was experiencing warmth during the LIA unlike the rest of Europ. (or perhaps measurement problems?
Note that the slop of the data for the bulk of the stations 1961 to end is 4.4degC per decade.
This data is all from SMHI page
http://www.smhi.se/klimatdata/meteorologi/dataserier-for-observationsstationer-1961-2008-1.7375
debreuil, I’m guessing that this is the center of the data release problem for CRU and they’re spinning despirately to avoid just that comparision
A thorough reading of the CRU email dump shows that this concern seems to be a 2nd order issue for these people (it may not be, of course)
But the primary concern that comes through constantly is the desire to protect what is perceived as their IP (ie. personally developed code, modeling, analyses etc) – to the point that it is suggested that if the RMS Journal insisted on showing all raw and treated data, code in published papers then no more papers should be submitted to the RMS
This is a very grey issue. I’ve had the experience of attempting to publish a peer-reviewed and approved M.App.Sc thesis which contained very precise geological mapping of commercial leases (the data was collected with the eager co-operation of the Companies who owned the leases) … but they didn’t want the data publically available (competitors, customers etc). The legal comment on this was that the Companies owned the data … on that basis, CRU owns the data, code etc, not the individuals that developed them
I suspect that if Jones had asked for such an endorsement of CRU’s value added “raw” data as recent as one year ago he would have gotten the full and enthusiastic approval of most, if not all, the national weather agencies.
Of course this would have been a major blow to their credibility if the climategate leak had unfolded in the same manner as last November, but who’s to say what difference this might have made if the FOI had been answered with “full” disclosure of the value added data with official sanction of the contributing agencies.
I have just done a perfunctory check of the Environment Canada web-site, which contains a great variety of current and historical weather data, including temperatures.
The standard agreement on data distribution is at the site and is quite easy to understand. It prohibits redistribution of data UNLESS certain conditions are met: no fee to be charged for the data, altho a fee may be charged for “value-added” data; Environment Canada must be acknowledged as the source of the data; and the receiving party must agree to the same restrictions.
From which I conclude that it is not that difficult for a researcher such as Dr. Jones to release any data obtained from that site (assuming that the EC site is what he referred to in his response).
But of course, Janice, you don’t provide a link to that.
Gosh that was hard. This is the first hit I got from a Google search of “environment canada redistribution agreement”:
link
I’m not very adept with computers and don’t know how to do that. However, if you google “Environment Canada” and go to the web-site, the agreement is very easy to find; i.e., if I can find it quickly, anybody can.
The Swedish license agreement is here:
http://data.smhi.se/met/climate/time_series/html/essential20.html
Dataserier 1961-2008
3.2 The Licensee owns no right to use the data or products provided under this agreement for commercial purposes and not for development or production of meteorological, hydrological and oceanographic value added-value services. The licensee does not own nor authorized to redistribute, sell, assign or otherwise transfer data products or documentation without further processing to third parties unless the parties have received written permission from SMHI.
Phil Jones was telling the truth.
Its a fair point. I suspect that UAE believes (1) they don’t have the right to redistribute the raw date without the permission of the Swedes, and (2) they don’t have the right to distribute derivative works based on the raw data without permission from the Swedes. Since they no longer have the raw data, they are asking for permission to distribute the derivative work.
I think confusion was created when UAE implied that the request sent to the Swedes was for the right to redistribute raw data. It was not. The request was to distribute the derivative work that CRU created based on the raw data.
This is not what Jones was referring to in testimony to the parliamentary inquiry when he said:
“We sent out some emails and letters to Met services around the world in November and we have got replies from many of those now and, through the auspices of the Met Office, we have now released 80% of the data on their website together with a programme that analyses the data, produces the derived product and produces the global temperature average.”
Jones and Acton conveyed the false impression to the parliamentary inquiry that the Swedes would not allow the release of their raw data. In fact, the Swedes provide open access to their raw data on the web.
Jones and Acton are pretending that the raw data is not available and that they are being prevented from publishing it (‘…we are longing to publish it…’) by an external organization. They are using the ‘no’ obtained in a different context to excuse their reluctance to describe their processing in detail.
Again, as mpaul said, the data on SMHI’s web is (according to the letter) not the data that CRU was requesting permission to release. In theri first letter SMHI did not allow CRU to release their (CRU’s) own derived station data for the Swedish stations (which apparantly may fall under some terms of use attached to the original station data CRU based their derived data on).
J Bowers: We don’t really know that this is the current agreement (if they have any) between CRU and SMHI. If there is one, it was probably made quite a while ago, possibly with different terms? The basic idea should be the same though, as stated on CRU’s data availability page.
Mikael Lönnroth: “If there is one, it was probably made quite a while ago, possibly with different terms?”
1968 to 2008.
Really, what’s the big mystery? It follows the pattern of standard agreements.
Sorry 🙂 I’m just saying that you can’t automatically assume that the newest version of a license agreement regarding _downloading_ data from SMHI’s website is the actual agreement between the CRU and SMHI regarding the usage of data they may have received since 19xx by other means than downloading.
The 1968 to 2008 is bound to mean the time span of the available data, not that they in 1968 wrote that license statement because they were clever enough to foresee the invention of the internet and their own site in the future 🙂
Anyway, the CRU web says “Since the early 1980s, some NMSs, other organizations and individual scientists have given or sold us […] additional data for […], often on the understanding that the data are only used for academic purposes with the full permission of the NMSs, organizations and scientists and the original station data are not passed onto third parties.”, so they are nevertheless in a generalized way clearly working under the assumption of terms like the ones mentioned in that license agreement.
The latest version of the UK’s license agreement has been changed recently. The previous version spelled out in no uncertain terms that there would be consequences for releasing the data, it was to be for academic purposes published in the recognised publications, and had to be deleted once it had served its purpose. They even bolded parts as emphasis.
Ok, thank you for the clarification.
No, § 3.2 does not constrain use of “homogenized” data, which is even more obvious in the original wording:
3.2 Licenstagaren äger icke rätt att använda data eller produkter som tillhandahållits i enlighet med detta avtal för kommersiella ändamål och inte heller för utveckling eller framtagning av meteorologiska, hydrologiska eller oceanografiska värdeförädlade tjänster. Licenstagaren äger inte heller rätt att vidaredistribuera, sälja, överlåta eller annorledes överföra data, produkter eller dokumentation i obearbetad form till tredje part med mindre än att man erhållit skriftligt medgivande från SMHI.
Dr Jones seems to have interpreted this wrong.
Agreed Kari.
“Licenstagaren äger inte heller rätt att vidaredistribuera, sälja, överlåta eller annorledes överföra data, produkter eller dokumentation i obearbetad form till tredje part med mindre än att man erhållit skriftligt medgivande från SMHI.” It should translate as:
The licensee does not own nor authorized to redistribute, sell, assign or otherwise transfer data products or documentation in [b]non-reworked form [/b] without further processing to third parties unless the parties have received written permission from SMHI.
Environment Canada website is http://www.climate.weatheroffice.gc.ca It is a very comprehensive website for raw temperature data Records go back to the 1800s in some cases with large numbers of reporting stations in all provinces. It is open public access to anyone.
I don’t see that UEA is referring to releasing raw Environment Canada data on the UEA website. Why does anyone think they would want to this? Obviously Environment Canada doesn’t want they website data to be duplicated somewhere and what would be the point?
Seems to me that UEA is playing games and perhaps is trying to get some form of approval if they release their own manipulated version of Canada data.
Interesting that countries with cold climates Canada, Russia, Sweden are referred to as “not wanting to permit release”. I’d suggest that these are prime countries for detailed comparisons of raw temperature data and UEA/NASA concocted data for the same reporting stations, if and when UEA is forced to release this.
I’d be willing to bet that the raw data will have been manipulated upwards. Presumably this is why UEA doesn’t want anyone to be able to make the comparison.
It could also be that Environment Canada has been involved in the manipulation of Canada’s tempertaure data and doesn’t want it to be available for comparison. One of the recently released batch of NASA emails is from Dr Jones mentioning that Lucie Vincent of Environment Canada has done good “homogenization work”!
The timeline of the correspondence and Hadley Met data releases deserve a little more attention. Without rehashing too much “Dr. Jones asserted … to explain his reluctance to comply with Freedom of Information requests.”
The MET Office claims on their website that they released some data “in December” and 1500 stations more on 4 Feb 2010. The MET letter is from 30 Nov. so it appears they had started a process to obtain permission (whether is was needed or not) and it could have been just a CYA attempt to hope somebody would back them up after the fact and after the Crutapes were released. Looking at both the Swedish document and website and the Canadian Climate Weather Office site it looks like there are few restrictions, especially for non-commercial use. It would be interesting to actually see the agreements that UEA/CRU used to justify previous claims of “contractual” limitations or “confidentiality”. Any claims based on DOC111209 that UEA/CRU would mount in the future or that perhaps was ‘spun’ during testimony fall short…The inquiry isn’t about what CRU did in Dec., 2009 or Feb. 2010. There’s a longer history.
An important FOI request that was denied was from Steve McIntyre. One CA link with a timeline, CRU Refuses FOI Request, Nov. 21, 2009, is here. Regardless of what the current CRU/MET claims might be there have been several past ‘excuses’ for not releasing data. The last MET office letter listed above, DOC111209, with Phil’s attached letter looks to me like an attempt on Phil’s part to achieve some verification for data or documents that he didn’t have or didn’t want to release and it didn’t work.
Here is the email I referred to from the batch recently released by NASA that is from Phil Jones mentioning to Environment Canada homogeneity activities. Have to wonder what “good homogeneity” is. Upwards adjustments????
On 8/15/07, Phil Jones p.jones@uea.ac.uk wrote:
Jim, Gavin,
Your recent finding of NCDC/GHCN not continuing to adjust records in real time and its impacts on blog sites has alerted me to inform you of a few things we’ve been doing over the past year. Don’t pass any of this on via Real Climate or whatever. Eventually, we will get around to documenting all we’ve been doing.
1. You may have noticed that Canada has changed loads of its WMO IDs. We’ve been in contact with Lucie Vincent there who’s also been doing some homogeneity work (which is good by the way). As a result of this we are applying adjustments in real time to about 40 stations (mainly in the east of the country) in order to use her long adjusted series. Why she adjusted records to an earlier period still isn’t clear to me. We are also getting on top of their station number changes, which appear related to automation – and giving the new AWSs a new number.
I may have missed something in all this – frankly the cataract of ‘gates (can’t we just call the whole thing Fabrigate?) has become hard to stay abreast of – but given what we know of Jones’ enthusiasm for ingenious ways of frustrating the FOIA, it looks rather as though:
*Aware that he would be asked how his most recent attempts to secure all the permissions necessary to release all the data (anything less being an insufficient basis for replicating his work), Dr Phil set the Swedes a carefully worded letter, containing a request he hoped and believed they would refuse.
*The Swedes, realising that associating themselves with the CRU’s “value-added” “quality-controlled” (= “adulterated” to thise of us here on Planet Earth) data was not likely to lend lustre to Swedish climatology, obliged, allowing Phil and the preposterous Acton to show a clean pair of hands to the Committee.
*The Swedes then realise they have been had for Patsies, and write a clarification, too late to confound Dr Phil and his handler, in camera, but not too late, one hopes, to elude the retrospective gaze of Graham Stringer MP as he battles to convince his more gullible colleagues that if it walks like a duck….
Alternatively,…
The license on Dataserier 1961-2008 from SMHI clearly states that data may not be distributed (3.2)
November 30th 2009
Request made by UEA to make data more widely available.
December 21st 2009
Request rejected.
“Given the information that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site.”
March 1st 2010
Parliamentary hearing.
March 4th 2010
Clarification by SMHI.
“We understand now that our response to your request forwarded by UK MetOffice 30 November 2009 may have been misinterpreted, maybe due to the fact that the formulations may have been a bit harsh.”
Jones isn’t psychic. SMHI recognised that they have probably given the wrong impression.
When people say why doesnt McIntyre just get the data himself?
Guess who wrote this email?
Guess when he wrote it?
Phil –
We have part of that information for GHCN.
In our 1997 GHCN version 2 paper (attached), we include a list of sources of data, a
one page table. If we had to do it now, the list would be far longer as it would include
all the GSN sources, France’s digitization of colonial African data which we then had
to contact the individual countries to obtain permission to use, John Griffith’s colonial
era data, Fred Wernstedts data, and on and on.
What we don’t have is a list of people whom we’ve contacted to try to get data. I’m
sure that would be an order of magnitude longer list as I actively beat the bushes
searching for more data usually to little avail.
Sometimes we received data from multiple sources. I think we preserved at one stage
the record of which data point came from which source. But it raises questions as far
as that person’s request, what if one version of the data was requested not to be
released and another one wasn’t, can you release it?
Part of me would like to encourage you to give him whatever lists you can put
together and ask him not to only ask for permission for you to release the data but to
confirm the exact data points offered for release by providing the person with the
data. He would quickly see how difficult this task that you and I have spent decades
working on is.
Another request from Mr. [snip] . As this one is quite different
that
the other, I will treat it separately.
Do we actually have the requested information? I am assuming that
we know
where we get this information from so we should have a list of where
we
get the data; what I am not sure of is whether we have a list of who
CRU/UEA has approached to provide the data.
Deadline is 11 September BUT we have the option under Reg. 7(1) of
EIR
to
extend this deadline to 40 days where we ‘reasonably believe the the
complexity and volume of the information requested means that it is
impracticable either to comply or… to make a decision to refuse to
do
so” within the standard 20 day period. This should be
considered with all
requests under EIR where applicable (Note – this provision does not
exist
under FOIA)
Cheers, Dave
Oh, I’ve been so busy I forgot about some of these mails.. I have to go through them, now
> >Phil, Peter,
> >
> >I talked to Carolin about this issue. She perfectly understands your
> >concerns and would in principle be willing to help, but advises that
> >WMO is currently not in a position to write such a letter – this appear
>
> >to be too delicate a subject (unfortunately).
****************
there’s more
For a typical country, there is an agency collecting raw temperature, by which it is meant the numbers as read from the instruments.
A number of host countries then apply a homogensation or other adjustments, like correcting scribal errors, deleting values that seem wrong, correcting wrong signs, converting from degrees F to C and so on. Some go further and apply corrections related to adjacent stations or even to distant stations, station relocations and introduction of new equipment. Some average to produce a range of data from near-continuous to monthly aggregated.
The host country can be using a near-continuous epgrade of data, working back through old paper records and applying recently-considered corrections like urban heat island. There can be changes in the status of station quality and some stations can be deleted and others added. Thus, we have to drop the concept of “raw” data being passed to global compilers like CRU, NOAA, etc. The data passed can have had quite substantial manipulations, both before it was passed to the adjusters and after. There could be many un-numbered versions from a single country, which had not been updated by the adjusters or even passed to them.
As such, if I understand correctly, the home country data can be copyrighted (as it is more than a simple list and requires intellectual input from the compilers)and the terms of the agreement can include that it not be altered without prior approval and that it not be passed to a third party without prior agreement of the provider.
Right now, we have a few major adjusters whose records show on close examination(if the available data allow such interpretation)that their adjusted data can be based upon obsolete data from the provider, or even now-rejected data. One suspects that some classes of homogenisation done by the home country have been repeated by the adjuster, to produce a result that is probably wrong by any test.
We also have a probable example (there might be more) of a copyright agreement being infringed as Steve noted in the case of Peter Webster of Georgia Tech.
I use “probable” to cover the possibility that at the time Webster was sent the data, at least one related country had an agreement operating, this time with CRU. That agreement might not have been of the type that starts “We, the following named Parties agree ….” but rather of the type that has a notice attached to the data. One example that I show here is from Australia’s Bureau of Meteorology (noting that it might not have existed or might have had different words at the time of the Webster disclosure).
http://www.bom.gov.au/other/copyright.shtml
Note that this specifically excludes non-authorised transmission to third parties. Note also that copyright law can differ between countries and is a topic for specialists. I call this a Type 2 agreement for simplicity here.
The point of the above is that there is an element of time sequences involved in the Jones/Acton statements before the Willis Committee. Let us presume that in January 2009 or so, CRU was sent some data from Australia’s BOM. It would be possible that it had an attached Type 2 agreement. Let us next suppose that Jones passed on the information, be it in identical or altered form, to his colleage Michael Mann. On the face of it, this would constitute a breach of Type 2.
Before the Willis Committee, it was claimed that agreements with some counties had prevented CRU from releasing raw data. Perhaps it was not a discussion about Type 1 agreements between CRU and host countries. Perhaps it was about the likely infringement of copyright over possibly many years by CRU failing to obtain the permissions in the Type 2 agreements, be they called copyright or another form of agreement to avoid technicalities re copyright laws.
In my interpretation, the factor used by Jones to refuse release of raw data was simple. By releasing raw data he would immediately confess to copyright or similar infringement of type 2.
There exists correspondence between myself, the Australian BoM and CRU/Hadley that might have tricky wording, but on a straight read it says that there was no type 1 agreement between Australia and CRU/associates. No agreement at all.
Since we know the global data are manipulated, that they include Australia and that (maybe) some such data were passed to Webster, then there seems to be a prima facie case that incorect statements were made to the Willis Committee. Unless further evidence comes available, I would claim that this was known an least to Jones; and that further, he could hardly fail to know that he had omitted to conform with the Type 2 agreements.
Further, if Jones/Acton claimed they had not lost raw data, that would also be false because there is a faint chance that they had seen much truly raw data.
Although I use Australia in this example, I suspect that many other countries would have similar Type 2 restrictions that were ignored; I do not mean to exclude other country circumstances here.
Material in this blog suggests that Sweden had a Type 2 agreement that was breached. Their remedy was to write a Type 1, more specific agreement. IMO, a normal action.
It would be interesting to know if Canada and Poland also had Type 2 agreements in force, and in breach, for some time before Climategate accelerated the excuses.
I’d like to call attention to the following relevant report on the status of the peer review process – brought up at:
http://klimazwiebel.blogspot.com/2010/03/frank-furedi-turning-peer-review-into.html
Thanks Pierre excellent article, but I have to ask: what did you say to upset Dr von Storch so much?
i don t understand, why CA would repost such a misleading article.
the SMHI did not allow Jones to publish their swedish data in dezember.
i also am a little confused, that the same people who always wanted the data, “as the CRU uses it”, are now attacking them for their wish to publish exactly taht modified data.
Perhaps we might talk about misrepresentation, when the Swedish group’s press release headline, “Climate scientist delivers false statement in parliament enquiry”, carries more than an overstatement, “given the subsequent timing of the contradictory letter.”
Perhaps we might talk about misrepresentation, when Anthony’s headline, “Phil Jones called out by Swedes,” gives Hu McCulloch and others the impression that the Swedish Met office called Jones out, and not a private Swedish group.
Perhaps we might not, and hand over to lawyers discussions of misrepresentation, like every discussion based on legalistic lingo.
What you all miss is that smhi together with the other nordic countries did their own analyses for the entire nordic region and guess what: the nordclim trend differs a lot from the IPPC trend, which is the same as the cru-trend.
Not missed by me – this is what I would expect. I have looked on the http://www.smhi.se website (English version) for this analysis but it’s not obvious. Could you post a URL for it.
Click to access rapport0900.pdf
It is the latest temperaturereport made by nordklim. It doesn’t contain raw data but they describe what they do pretty well. But what’s interesting is the output. Very funny looking hockeysticks. The trends are all over the place. Considering SMHI is pro AGW you must admit they’ve got balls letting that report hang in the open
Andreas – that report is dated 17.11.00. Is this the “latest” you meant?
I agree “latest” wasn’t the best description since it’s 10 years old but the study includes the whole century and smhi obviously doesn’t think the historic temperatures has changed unlike giss, noaa, cru who rewrites history all the time.
In case anyone missed this, Willis Eschenbach on WUWT (5.3.10):
geo (18:46:15)
“Bottom line, Jones can’t make the data available that is required to check his calculations without the Swedes approval, even if it is a “different” dataset than what the Swedes keep themself. The Swedes still have partial ownership rights in Jones massaged version. The FOIA were about the data that Jones uses for CRUTEM, and he can’t release it without the Swedes approval even if they are very concerned that whatever indignities have been inflicted on it since they handed it over be clearly identified as not their responsibility.”
Well, since I made the first FOI for this data, I can assure you that you are wrong. Jones refused to release the data to me because he said that some of it (2% by his figures) was covered by confidentiality agreements. (Of course in the fullness of time we found out that he couldn’t release it because he had lost it … but I digress). He didn’t say at the time “I lost it”, he said “confidentiality”.
So I said fine, then release all but the 2% … but he said he couldn’t do that because of sunspots, no, because I might try to find fault with it, no … hang on, I’ll look at my correspondence … oh, yeah, because it was available on the web … somewhere. When I asked where, they wouldn’t answer. Useless.
“Further, it is not enough to say “there is no evidence the Swedes put limits on it originally”. They wouldn’t need to. You don’t get to give other peoples stuff away to third parties just because they didn’t say you couldn’t –they have to affirmatively say you *can*. They don’t lose their rights because they didn’t make a point of asserting them at the time.”
Nonsense. Jones made a clear distinction to me between the 98% of the data that theoretically could be released because it was not covered by a confidentiality agreement, and the 2% that could not be released because it was covered. Your claim is simply not true.
So then I and others filed an FOI asking for the confidentiality agreements. They pulled up three or four pieces of paper, only one of which restricted redistribution. Ooops … sorry, they said, but they had lost the rest of the confidentiality agreements. None of these was from Sweden. So all of their excuses turned out to be totally bogus.
Finally they ‘fessed up that they can’t locate the raw data that their whole edifice is based on. The dog ate their homework. So they were lying from day one, just shucking and jiving to avoid telling the truth — they lost the data.
Now, they’ve written to Sweden to ask if they can release the massaged Swedish data. The Swedes, reasonably, said no way. You can’t release the munged data and call it the Swedish raw data.
So in fact, Jones et al.’s whole CRUTEM and HadCRUT “adjusted” datasets are merely apocryphal. The raw data is lost, gone. They released what they claimed at the time was the raw data, but it’s not. It is merely the CRUcified data, so it is useless.
So there is nothing scientific about the CRUTEM and HadCRUT adjusted datasets, because they cannot be replicated. And replication, as we know, is the core of science.
As a result, the UK Met Office (the “Had” part of the HadCRUT “adjusted” data) has decided to start over, and reassemble the raw data from scratch. They know, even if you haven’t noticed, that everything that the CRU now has has is useless unverifiable junk. Despite your claim, they know that there is no way to “check [Jones’s] calculations”. The game is over.
End of story.
We can all agree that CRU was looking for any excuse for NOT releasing any data. To me, the most extraordinary comment is that they “lost” the original data. This alone should reduce the precision and probablities in the conclusions reached of ALL studies produced with processed raw data.
The IPCC should acknowledge this in the next report.
Re: Syl (Mar 7 10:10),
Exactamundo.
There are so many other issues, but confidence in scientists comes from the PhD after their names – yet also from their individual histories, which includes their responsibility level.
Anyone who loses their data should pretty much not be trusted with data in the first place. That alone makes one a non-scientist. Should Jones et al be drummed out of the Corps? Oh, man, the injury to their science is SO huge; any rehabilitation of the individuals is probably impossible – even as they do damage control, the ship is sinking. At least the adults are beginning to wake up to the pranks the children have been up to.
…Even more than replicability (which is WAY up there in importance), science is about quantifying that which is studied. Without the numbers = each and every VALUE recorded and each and every AMOUNT of adjustment – is there any science to begin with?
If the numbers/values/amounts are lost, why should the work itself not be tossed out?
Evidently the Met Office is at least superficially of that same mind… but time will tell – and the whole world is watching now!
Yet the public, so many of the public, still accept the almost daily pronouncements that the world is going to hell in a warm handbasket, in spite of the scandal. Give some credit to the spinmasters.
But public be damned, what will collapse their house of cards will be their fellow scientists, who in ever increasing numbers (numbers are important in more ways than one!) are rowing away from the disease-ridden ship that is HadCRU and leaving it to its fate.
I would still like to hear Steve comment on how in the way he usually does business he would deal with all these countries insisting on distributing the data directly themselves, without granting right of subsequent redistribution (which the Swedish website apparently specifically points out this is forbidden without written receiving written permission).
Do we even know how many different data providers there are? How many different formats they provide in?
How could someone like Steve do his thing the way he usually does it (which includes either direct redistribution of data, or pointing at links where it can be gotten) if the number of data sources and formats in unwieldly and no right of redistribution has been granted?
Re: geo (Mar 7 10:39),
You provided the answer yourself:
It’s not that hard. Just make a list of links.
Even better — provide an R script that downloads and assembles the data from the links.
That’s what Steve has done on any number of occasions.
It’s called Reproducible Research: provide the code, provide the data. Enable others to do what you did, to check and extend your work.
Possibly. How many links do you think there will be? What’s the most Steve has done for a single project? I like the idea of the R script –could it step through “check the box to agree to our license” kind of speed-bumps?
I like what Steve has done, don’t get me wrong –I’m just wondering how doable it is if the number of discrete, separately stored sources gets too large, and where that “too large” point is.
Oh, and as I pointed out to Willis after that –given everything Willis has said about Jones, it is really time to recognize he shouldn’t be using Jones as an authority to prove anything, including the legal state of play of redistribution rights to third party data.
Can we find common ground on “throw dirt in CRU’s face and move on, leaving the issue of CRUTEM’s accuracy unresolved”? I would be willing to do so. I’m not sure everyone would. I think some people for at least historical reasons would like to get to the bottom of how accurate or inacurate CRUTEM really is/was.
I saw the Met office is going into business for themselves (which Steve had previously called for). That’s a positive. It’s not clear to me yet if they will succeed in overcoming all the redistribution rights issues, or merely end up another non-transparent “trust me” database as to the 100% availability of all the underlying raw data that would be necessary to truly “audit” their results.
Is GISStemp raw data 100% available? If so, then it proves it can be done.
“Is GISStemp raw data 100% available? If so, then it proves it can be done.”
The short answer is “no.”
It has been “adjusted” many times by Hansen et al. The original data was overwritten.
“Is GISStemp raw data 100% available? If so, then it proves it can be done.”
All the primal raw data has been conveniently “lost”. So probably it’s impossible to falsify the GISS or CRU data. There’s only one solution to stop the ever last discussion; start from scrap with methods open to everybody.
Re: Hoi Polloi (Mar 7 09:54),
I like this Apr 20, 2007 one from CA: Measuring Precipitation on Willis’ Boots, here.
I wonder when all the angry visitors, frantically bailing, will recognise that they’re trying to bail an estuary as the tide is coming in.
Give it up guys.
Go and enjoy your day and reflect on just how you’ve been deceived.
Mikem
I would agree with you here that the blog heading and PR headline are a bit over-the-top. As someone who gets very annoyed at the hyperbolic “we’re all going to die” rhetoric from the usual suspects, I have to say this all detracts from the underlying message. (You have to admit though, it’s helped CA’s blog traffic while Steve’s out earning a few loonies.)
But back to the underying issue. I was going to sit down and compose a comment about the nuances and subtleties involved and how Jones/Acton are just paying the price for raising a red herring to the committee using Sweden’s original letter.
Then I came across this:
http://rankexploits.com/musings/2010/the-pure-anomaly-method-aka-a-spherical-cow/#comment-36770
Lucia says it more lucidly than I ever could.
(Surprise, surprise, huh? :))
Even the concept of “raw data” re historical data is interesting at this point. I doubt few people are calling for the release of photocopies of 1880 observer logs. Try paying your sizeable tax bill to the IRS in the correct number of pennies by delivery service and see what happens. The value of Jones dataset was that it was collected, digitized, and homogenized as to *format*. The scandal is all that was done to data that had been homogenized as to actual adjustments to the value of the data elements themselves.
After all the generations of the raw data moving from format to format, one could suspect that other degree of “auditing” –checking that the real raw data has survived its many transferences of format– is probably worthwhile too. I suspect we’d find if the attempt were made to undertake it that a lot of the real raw data–those old observer logs– has been destroyed after it was transferred to a more modern format, and thus the audit trail broken.
That would not be the usual government procedure. Governments take their paper archives seriously. In Washington State (USA) the government has two copies of each piece of paper it has collected or generated for the last 120 years – an original in Olympia (the state capital) and a copy in Pullman (Washington State University is set up as an emergency backup for government in case of nuclear or natural disaster).
I can understand the problem that might arise where war or other disaster has destroyed archives, but not deliberate destruction of paper records. This is what is so strange about CRU’s claim – its so far from standard archival practice and procedure.
Perhaps “a lot” is too pessimistic. But we’re talking about 150 yrs or so of worldwide data here –some of those true originals have almost certainly gone on walk-about by one means or another.
As I have said UEA is small underfunded (especially in the 80s) university. Retaining raw data would have been costly.
as an example consider early NASA lunar mission data:
http://en.wikipedia.org/wiki/Lunar_Orbiter_Image_Recovery_Project
The original tapes were carefully archived for 20 years in Maryland. When the tapes were released back to NASA’s Jet Propulsion Laboratory (JPL) in Pasadena, California, in about 1986, the decision of whether to scrap the tapes became the responsibility of archivist Nancy Evans. She instinctively decided that the tapes should be preserved.
i.e it was just one person and luck that these tapes have not been destroyed. NASA I imagine would be slightly better funded than UEA.
Raw data can mean different things.
One place to begin might be un-adjusted, un-grided data.
Another place to begin might be to go back to the weather stations and get the original data.
Gridded or monthly mean data are not raw data. The original station data is the raw data that is the subject of the FOI requests.
So what are Jones and the CRU up to changing the terms of reference from the raw data to the value added gridded or monthly mean data?
For give me for appearing to be a little cynical, but I do not think Jones is being entirely honest with what he is doing.
I do not think the issue will go away until we all start from scratch with the actual paper records and document each step that data is subject to from paper to global average Temp.
I’m a skeptic myself. However, I find that this kind of dicussion is counter-productive, even detrimental to the cause of healthy skepticism. It creates the impression of nit-picking, of trying to find fault at any cost.
I’m sure that there is enough fault to be found with Dr. Jones, but in this particular case most of the fault seems to lie with SMHI. That institution itself has admitted that its first letter “may have been misinterpreted, maybe due to the fact that the formulations may have been a bit harsh”.
However, the wording in the first letter “SMHI do not want the data to be released on your web site” is not only harsh, it is as clear as one can get and can hardly be misinterpreted.
There is also a blatant contradiction between the two letters. In the first letter, they effectively say “because your data might be different fron ours, we do not want them released on your web site”. In their second letter they say “because your data are probably different from ours, you are free to publish them as long as you name SMHI as the source of the original data”. This is in accordance with their own licence agreement, whereas their first letter contradicts it (as Rattus Norvegicus has already pointed out).
There is another contradiction beteen the two letters: The second letter claims that “the original data can be obtained” from their web site. The first letter, however, clearly states that the data on their web site are still incomplete.
In short, the whole matter was badly handled by SMHI, and they did themselves a disfavour.
So keep on criticising Dr. Jones on the basis of clear facts, but please do not make him responsible for the mistakes of others and do not indulge in pure speculation on supposed ulterior motives.
Agree completely, this kind of discussion does belong on WUWT not on Climate Audit, as it undermines it’s pristine statistical/mathematics reputation. Too much tabloid discussions here lately unfortunately…
I agree that the Swedes are responsible for the confusion here, and not Phil Jones.
However I disagree with your contention that this discussion does not belong on Climate Audit.
This blog seems to be about two things: (1) getting the data and (2) auditing the data. There are hundreds of posts on each of these two subjects. If, from the beginning, Steve only focused on auditing data that was already in the public domain, there would have been very little to write about.
Furthermore, this particular discussion is directly on point to topic (1). The excuse du jour for the secrecy of climate data is confidentiality agreements. The (second) Swedish letter directly challenges Phil Jones’ assertions on this topic.
Steve posted the letter, and then allowed many comments that, on balance, exonerated Jones of the ‘charges’ in the second Swedish letter. To my mind, that’s clearly within the scope of this blog.
Sean,
When you use the term “the data” you need to be clear about what is meant. The climate science community has generally obfuscated requests for data such as the station IDs which they used to create their gridded data; the tree ring data used in proxy calculations; and other clues which might lead one to the source data. Jones freely acknowledged this at the Parlimentary hearings last week. One can not audit a report if one is not privy to the data which supports the report’s conclusions.
Jones stated several times at the hearings that the source data is readily available. This is true. One can get data from the weather stations. What has been a closely guarded secret, until very recently, is the IDs of the particular weather stations used for papers which Jones authored (or co-authored).
From Sweden, Max Andersson writes Climate sceptics are wrong about Phil Jones and SMHI (7 March 2010)
The license agreement has already been published above. You fail to qoute 3.2 as others have, which clearly allows edited data to be published. Nothing in 3.2 or 4.1 prevents the Licensee from providing a link to said data and which parts of said data were used to generate the edited data.
I still do not understand why if the datasets themselves could not be sent to justify the FOI requests, why a link to the data and a description of what was used could not be. If the info is not on a public website at the very least a description of the data used and the country/organization it came from should be provided. Nothing in the Swedish license prevents a researcher from saying I used data from station a,b,c… from the years x to y. To my knowledge, EAU has not even provided that.
The Max Andersson blog quotes the standard agreement as rationale for refusing the request.
However, the initial request asks not for the raw data, but permission to publish the amended data. This is not the request that was made as others have pointed out.
Refusal in the response made no mention of standard confidentiality, but declined on the basis that the data would be altered and that they didn’t want to be associated with it.
I’ve posted to this effect on Max Andersson’s blog. Pending moderation so far.
The comment was added without problems (thanks Max). His response is after:
http://maxandersson.blogspot.com/2010/03/climate-sceptics-are-wrong-about-phil.html
My goodness, but Global Parsing has doomed us unless we destroy mankind…
I got this Swedish data that goes back to @ 1960, here:
http://data.smhi.se/met/climate/time_series/day/temperature/
Which are the raw temperature data for each station.
For fun, I’m going to plot the raw data as colored blobs on the gps coordinates of a satellite map of Sweden – just to see what it looks like. My computer has been chewing on the files for about 24 hours to sort data into one file, by date, with all stations’ data for that day. Then I’ll plot a frame for each day and assemble the series into a vid. I have no idea what it will look like. It’s a few days away, anyhow.
Then I’ll probably post it on the net without asking anybody’s permission – the way the best people do. Perhaps on a Russian server just cuz anybody can?
sounds good. please post the link here (as a reply to your original comment) when you finish.
sean:
52 weather stations in Sweden report temperature observations.
Daily records of all stations for each day are converted to color and plotted on a map at the precise coordinates of the observation station.
Each daily map then becomes a frame of a video.
It’s a bit degraded by the reduction and compression needed to get it on the net.
I find that following the particulars about this particular permutation in the CRU Data Dump Saga too difficult to follow.
Who said what, first? Who is correcting whom? And what interests or standing – legally speaking – support or undermine their respective opinions, regarding the Swedish authorities and/or interest groups?
Am I alone in my inability to untangle this bramble? (YES it is easy to lapse into conventional assignation of “Z motive by Y, therefore blame accrues to X.” However, like Steve’s examples, I’m trying not to do so.)
I did post a comment putting the various letters, section in the original SMHI license agreement, etc, in chronological order, but it disappeared into moderation. Be interesting to see if this makes it through.
http://www.stockholminitiative.com/eng/about-us/who-we-are/
A visit to the website of “The Stockholm Initiative” shows that not only does the group as a whole lack the technical qualifications to speak on any climate change issue, but that their mission is to debunk AGW and push the benignity of fossil fuels. Their interpretation (and that of Mr Watts) of the SMHI letter is absurd. The letter is simple enough for anyone to understand on without their dubious assistance and it clearly states that they requested Jones no release their data on his website.
Of course, nobody was asking him to post it on websites. The FOIA request was to get it, not post it on websites.
Try again….
The license on Dataserier 1961-2008 from SMHI clearly states that data may not be distributed (3.2)
November 30th 2009
Request made by UEA to make data more widely available.
December 21st 2009
Request rejected.
“Given the information that the version of the data from the SMHI stations that you hold are likely to differ from the data we hold, SMHI do not want the data to be released on your web site.”
March 1st 2010
Parliamentary hearing.
March 4th 2010
Clarification by SMHI.
“We understand now that our response to your request forwarded by UK MetOffice 30 November 2009 may have been misinterpreted, maybe due to the fact that the formulations may have been a bit harsh.”
Jones isn’t psychic, nor is Acton. Even the SMHI recognised that they have given the wrong impression.
“A visit to the website of “The Stockholm Initiative” shows that not only does the group as a whole lack the technical qualifications to speak on any climate change issue….”
So typical AGW’er argument. Attack the messenger, dont really understand the message.
Do you know any of the names on that list? Any bells ringing? No? Well then.
I am not trying to ‘sum up’ (or anything like that) but my take is that if the whole process, where transparency is important…Acton and Jones were being coy, no matter what.
What a load of cobblers.
How much data has CRU lost?
Assumptions (Lots):
Most early records would have been on paper. Perhaps one sheet/month?
1000 stations (out of perhaps 5000)for 50 years to 1980s (i.e. the first record produced in the 1930s on average)
Making no allowance for dividers for filing or shelves etc. Just the paper:
50 years
12 months/y
1 A4 sheet/month
1000 Stations
5 cm/ream (measured)
500 sheets/ream
21 width A4
29.5 height A4
sheets total 600000
reams 1200
height 60 metres
volume 3.717 cu metres
So to preserve raw data would take a storage space of perhaps a lot more than 4cu metres.
These sheets would have been transcribed into computer format, possibly with errors added and errors corrected so the first computer record is NOT the raw data.
If you had to move office and you thought that you would never go back to your 4 metres^3 of raw data because your computer data was correct. And you had admin complaining that there was not room for your tatty bits of paper. And it needed sorting and indexing to be useful. I think binning it would be a sensible option.
And if you wanted raw data the WMOs would still have a copy so nothing is lost
Now if someone comes along and demands that you provide the raw data. What do you give them – your modified computer data, or the raw data?
Well they asked for the raw so how much will that cost:
£1800 @ £1.5 per ream.
+time+photocopy cost
Probably manageable but would the recipients be happy with your 4 cu metres of data???????????
It is also interesting that with all these £1000000s being paid to UEA for climate research that they JUST purchased a new scanner in 2008 for a project and they were going to use it to ILLEGALLY(?) scan a document to satisfy McIntyre:
. This email came to CRU last night.
From: Steve McIntyre [[1] mailto:stephen.mcintyre@utoronto.ca]
Sent: Tuesday, June 03, 2008 5:09 PM
To: [2]alan.ovenden@uea.ac.uk
Subject: Farmer et al 1989
Dear Sir, Can you please send me a pdf of the Farmer et al 1989, cited in Folland
andPArker 1995, which, in turn is cited in the IPCC Fourth Assessment Report. Thanks,
Steve McIntyre…
CRU has just the one copy of this! We’ve just got a new scanner for a project, so someone here
is going to try this out – and scan the ~150pp. I’m doing this as this is one of the
project
reports that I wished I’d written up. It’s got all the bucket equations,
I wonder how long it would take to scan those half million station reports!!
The WMO does not receive or maintain copies of the original surface weather observation record forms or their summaries in any archive. The NMC (National Meteorological Centres) are responsible for archiving source records forms and transmitted datasets. Many of the NMC have failed to archive the source records forms and datasets, and in some cases the NMC are presently destroying at least some of those original source records and datasets by negligence and/or deliberate records retention policies.
Thanks for the correction DP
I could have finished this off with a couple more lines
total area 37170.00 sq metres
total weight 2.97 tonnes
That’s some weight to send to anybody
Sorry – a bit more!
The weight above assumed 85gsm paper.
600000 sheets of perhaps dog-eared paper reports (would require manual feeding)
5 seconds per sheet
6.5 working hours per day (allowing lunch and breaks)
is about 128 days photocopying making no allowance for retrieval!
One also has to assume that Jones would have to satisfy all the other FOIs with this data as well.
Phew!!!
Something a few grad students could knock out. What’s the big deal?
I hope you are joking!
Perhaps McIntyre has sufficient time on his hands to do the copying? 3 tonnes excess baggage on the return trip may hit him a bit hard, however!
Re: thefordprefect (Mar 15 12:52),
Argumentum ad absurdum.
So, I can only assume, from your rant, that CRU just plugged all those sheets of paper into their computers and out came magic. The stuff has to be in digital form for CRU to have used it.
Re: thefordprefect (Mar 14 21:20),
It’s funny how little practical sense you have.
Funny that you came up with a figure of 600000 pages. That’s exactly how many pages of raw source material a friend of mine maintained for his global almanac.
In the early days, it filled a room full of filing cabinets.
Then it was converted to microfiche form both for ease of storage and to make it sharable.
Later, it was digitized.
All by volunteers.
To thefordprefect, an absurd notion. In the real world, a practical necessity.
If that’s actually the data that needs preserving, then do so. Don’t cry that the dog ate your homework.
http://www.dlib.org/dlib/february00/kingma/02kingma.html#Table2
“The first four rows of Table 3 show the cost of producing master copies of microfiche. The cost of producing master copies of microfiche is $114 per fiche, $1.54 per image, or $333.11 per 216 page volume.”
600000 pages are equiv to 2778 volumes of 216 pages
2778 volumes at $333.11 per volume is $925,305!
The data is all at the NMCs – no data has been eaten. McIntyre has every right to go to the NMC as request copies.
The CRU decided that the data they digitised was adequate for their purpose. Who has the duty to archive this data? Surely it must be the dtat originators? Why retain the same data as the NMCs have preserved?
It’s funny that you know all this, but Jones never said a word about it.
ford prefect writes:
How was this modified computer data produced. Someone must have entered this data into the computer for processing. So the raw data must have existed in computer form at least for a time. Even the CRU must have had accesss to tape drives in the 1980s. Certainly all the computer centres that I worked with did. People had large and valuable data collections back then and they archived them as a matter of course. I can’t imagine going to management and saying ‘You know that design that you paid millions for. Well, don’t laugh but …”
Do you remember paying taxes for NASA Lunar Orbiter Missions?:
wiki
http://en.wikipedia.org/wiki/Lunar_Orbiter_Image_Recovery_Project
The tapes from the Lunar Orbiter missions were primarily used to locate landing sites for the manned Apollo missions, and once those missions were over, the data was largely forgotten.
Only chance has allowed the project to retrieve data.
And this was conveniantly stored on tape, not handwritten on hundred of thousands sheets of paper.
The RAW data so deperately desired is on no computer it is with the NMCs. Any translation of paper to digital media can be accused of bias/fraud. I’m sure the data entry clerk will have made mistakes and “corrected” obvious errors (e.g. 99C changed to 9.9C). You want the raw then the raw is what HAS to be provided else you will not be satisfied.
It seems that even in 2007 paper is still the recording medium:
http://gallery.surfacestations.org/main.php?g2_itemId=28549&g2_imageViewsIndex=3
From Orland and Watts site.
any chance of unblocking
Thanks
Nine track tape drives were common commodity items even in the 1970s. They were used in every computer centre that I was involved with. It would surprise me very much that the UEA computer center was not equipped with these drives in the 70s let alone the 80s.
What makes you think Mr. McIntyre wanted paper and why did you bother with all of that calculating? In the sample email that you provided he specifically requested a pdf.
Indeed I can’t recall any instance on this blog where Mr. McIntyre suggested he wanted paper. Raw data? Yes. But when he was searching directories across the web for data files, I hardly think he was looking for paper. Nor did he ever complain when he received digital data.
As for your suggestion that it might be illegal for Jones to scan a report that he co-authored at CRU for delivery to the UK NERC and US DOE, I have one question. Huh?
Apparently it wasn’t illegal and it wasn’t much of a burden to scan the report. Three (3) days after the request was made by Mr. McIntyre this document was created.
Click to access Farmer-1989-NERC.pdf
So, Dr. Jones was right -he wasn’t allowed to release data he didn’t own-. I guess a “sorry” post will follow.
Yes, I’m sure it will.
Won’t it?
Jones was hounded for the raw data (which he had no right to distribute – sending to another climate scientist with whom he was going to collaborate on a paper is NOT the same as distributing).
The scanned copy that Pasteur1 notes is not the temperature data requested it is a completely separate published paper.
No one wants paper copies of data. The transcription to digital format is time consuming and prone to errors. But ANY data that has been transcribed is not RAW – errors may have been corrected on-the-fly errors may have been introduced.
In meteorology data is still in the form of paper copies at the point of source == RAW.
When Jones started on his project in the 80s there must have been a lot of paper copies to transcribe (Jones says he spent many years doing this!).
My, frankly toungue-in-cheek entry just sets out a possible scenario. I do not know how muck of the pre 80s stuff was on paper and how much was provided on computer tape.
Obviously once the dta is on tape then a single reel of tape is all that is required to hold all the vast quantities of corrected/errored data. Provided these have been periodically wound through a machine, provided the tape readers are still available, provided the oxide is not shedding, provided the acetate(?) has not degraded, provided you can afford someone to nurse the tape through the machines, then copies of this almost raw data can be sent with NMC approvals to others. IT IS NOT HOWEVER THE RAW DATA
Ford – just give it up, anyone with an ounce of sense and experience can see exactly what went on here. Suggest you stop digging and let the wheels of due process take their course – given the misrepresentations that CRU have made over the FOI requests the more you go on about it the more likely the true nature of the deception will be noticed by the wider world.
Click to access agreements.pdf
Re: curious (Mar 19 18:17),
I disagree. He seems intent on totally exhausting his credibility here. I say we let him.
Has anyone checked whether P.D.J. had legal or even courteous permission to
(a) copy the data to his files;
(b) “value-add” to the data;
(c) incorporate the data into sets like global without specific acknowledgement of source;
(e) pass the data to third parties;
(f) pass the data to third parties without caveats about the adjustments he made;
(g) omit to confirm with the originators that his adjustements were acceptable, before distributing them;
(d) make assumptions as to copyright that might in hindsight be illegal?
6 Trackbacks
[…] Phil Jones called out by Swedes on data availability issue From an emailed PRESS RELEASE on March 5, 2010 Climate scientist delivers false statement in parliament enquiry It […] […]
[…] quite concur with Prof. Jones’ statement that they don’t want their data released, see here. Possibly related posts: (automatically generated)Opening Night Reviews in the UK […]
[…] Swedes wrote into Parliament to clarify (see here at Climate Audit) that they have no issues with data being public at all. The blog Bishop Hill beat me to the punch, […]
[…] DOC111209 Data_from_the_HadCRUT_dataset_100304This press-release have gained considerable attention on climate denier blogs but contains many factual errors. To begin with swedish data is […]
[…] Phil Jones caught out lying – again!, variations in CO2 measurements, Tomato cold, UAH temperature variations. […]
[…] Swedes call out Jones on data availability (ClimateGate hearing) climateaudit.org entry […]