“Community” climate scientists (e.g. Nature, Realclimate) have been quick to accept the idea that CRU was prevented from releasing station data because of confidentiality agreements with originating meteorological services.
Something that Nature and Realclimate didn’t discuss or even seem to notice – and the blind spot is symptomatic – is that Jones delivered “confidential” data to his friends. In a Climategate Letter in early 2005, Jones told Mann that he had just sent “loads of station data to Scott [Rutherford]”, Mann’s close associate.
I guess that it didn’t occur to Nature or to Gavin Schmidt that confidentiality agreements also apply to masters of the universe. If the data is as confidential as Nature and Realclimate say, then the distribution of confidential data to associates and other third parties in breach of the confidentiality agreements may warrant just as much attention as Jones’ effort to obstruct FOI requests.
CRU Statements
CRU publicly articulated their position on the supposed confidentiality agreements with national meteorological services (NMSs) in summer 2009 both in interviews with Nature and in a short statement on their website posted in response to numerous FOI requests for confidentiality agreements. (These FOI requests constitute the vast majority of all FOI requests.)
On Aug 13, 2009, Olive Heffernan reported in Nature (v. 460, p. 787):
Jones says he can’t fulfil the requests because of confidentiality agreements signed in the 1990s with some nations, including Spain, Germany, Bahrain and Norway, that restrict the data to academic use. In some cases, says Jones, the agreements were made verbally, and in others the written records were mislaid during a move.
In an associated blog post, Heffernan added:
Why won’t Jones give McIntyre the data? Jones says that he tried to help when he first received data requests from McIntyre back in 2002, but says that he soon became inundated with requests that he could not fulfill, or that he did not have the time to respond to. He says that, in some cases, he simply couldn’t hand over entire data sets because of long-standing confidentiality agreements with other nations that restrict their use.
(In passing, none of these statements should be taken at face value. For example, Jones obviously did not become “inundated with requests” “soon” after my 2002 request.)
Jones also posted a webpage on the matter at almost the same time as the Nature article:
Since the early 1980s, some NMSs, other organizations and individual scientists have given or sold us (see Hulme, 1994, for a summary of European data collection efforts) additional data for inclusion in the gridded datasets, often on the understanding that the data are only used for academic purposes with the full permission of the NMSs, organizations and scientists and the original station data are not passed onto third parties.
The “Agreements”
As most readers know by now, in response to multiple requests for the supposed confidentiality agreements with national meteorological services, CRU produced only four “agreements” (UK Met Office, Spain, Norway, Bahrain). Obviously, these agreements would not have encumbered the entire data set for very long, as waivers from Spain, Norway and the Met Office could presumably have been obtained in a day or so.
Faced with these few embarrassing scraps, CRU argued that there were many other agreements that they had unfortunately lost or never committed to writing. Indeed, they didn’t even know for sure who they had made agreements with. CRU:
Below we list the agreements that we still hold. We know that there were others, but cannot locate them, possibly as we’ve moved offices several times during the 1980s. Some date back at least 20 years. Additional agreements are unwritten and relate to partnerships we’ve made with scientists around the world and visitors to the CRU over this period.
They stated that these agreements contained language that was representative of the period and of what was used in the “lost” agreements, noting in particular that the examples showed that they “would not make the data available to third parties”:
In some of the examples given, it can be clearly seen that our requests for data from NMSs have always stated that we would not make the data available to third parties. We included such statements as standard from the 1980s, as that is what many NMSs requested.
The four agreements were placed online here.
The agreement with the Met Office (respecting data from various colonies and ex-colonies) is the clearest agreement. It does indeed include language that the data would not be made available “to third parties”. But it did more than that. It said that the data would only be used by the Climatic Research Unit and only for the “specific project” of “construction of a gridded baseline climatology for global land areas” and would “not be used unauthorized for any other project” or “passed on to third parties”:
These data are to be used by the Climatic Research Unit for a specific research project sponsored by the NERC, namely the construction of a gridded baseline climatology for global land areas. The project falls in the area of Global Environmental Change which makes this data request subject to the IACGEC Framework for Data Exchange to which the Met. Office is a signatory. The data will not be used unauthorised for any other project and will not be passed onto any third party.
The CRU “agreements” included a recent NERC form which clearly proscribed use of data for academic work outside of the original consented program:
UKMO data/software so obtained may be used solely for the purpose for which they were supplied. They may not be used for any other projects unless specific prior permission has been obtained in writing from the UKMO by a NERC Data Centre. Note that this applies even for other bona fide academic work.
The NERC form also stated that any “infringement” of confidentiality, whether intentional or merely negligent, would be taken very seriously – language that is a little ironic when one considers that CRU had left versions of the confidential station data on their FTP site for many years (see CA, July 2009, A Mole?)
NERC recognises that some data holdings supplied by UKMO under the arrangements are commercially valuable : the recipients of data are under an obligation to respect the terms and conditions of data supply, and to have regard to the security of datasets entrusted to them. Any infringement, whether by deliberate abuse or negligence, will be regarded extremely seriously by NERC, as endangering Council’s own reputation and the integrity of the NERC supported scientific community. The introduction of sanctions against individuals or Departments may be considered if breaches occur.
“Loads of station data to Scott”
For the sake of analysis, let’s stipulate the position of CRU, Nature and Realclimate – that the station data was restricted by confidentiality agreements.
Despite such agreements, on Feb 2, 2005, Jones emailed Mann notifying him that he had just sent “loads of station data to Scott [Rutherford]”, Mann’s associate. This email has previously attracted attention for Jones’ threat to delete the data rather than send it to “MM” and for Jones’ disdainful attitude towards UEA officials who would dare argue that the Freedom of Information might apply even to Jones. However, commentators have thus far not paid attention to the equally interesting information that Jones had just sent “loads of station data” to Mann’s close associate, data that would be denied to Warwick Hughes a few weeks later, to Willis Eschenbach in 2006, and to me, McKitrick, Pielke Jr, McCulloch and others in 2009. Jones wrote Mann as follows:
Mike,
…
Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites – you never know who is trawling them. The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? – our does ! The UK works on precedents, so the first request will test it.We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it – thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere to it!...
Phil
Another distribution of confidential station data to a third party occurred last summer when Jones delivered station data to a professor at Georgia Tech, who, unlike Mann, was not a close associate of Jones, and who had innocently asked for the data, implicitly accepting the quid pro quo that Jones would be granted a coauthorship on any subsequent article. Olive Heffernan reported at Nature (2009, 460, 787) as follows:
Webster says his team was given the station data for a very specific request that will result in a joint publication with Jones.
At the time of his initial inquiry, Webster would have had no reason to presume that the CRU station data was encumbered by confidentiality agreements and nothing here is a criticism of Webster’s behavior, which has been scrupulous at all times. However, Jones knew of the confidentiality agreements and it was Jones’ obligation to honor the supposed confidentiality agreements. Nevertheless, Jones delivered the “confidential” data to Webster in the expectation of coauthorship – a coauthorship that would incrementally increase Jones’ H-Index.
The Climategate Letters show that Jones’ H-Index was important to him: Jones wanted to be an AGU Fellow and looked to Michael Mann to organize the nomination. In one letter, Jones and Mann had an unsavory discussion in which Mann said that he would report a false higher value of Jones’ H-Index in connection with Jones’ nomination as AGU Fellow. (Because Jones is a common surname, Jones advised Mann that an online algorithm for the H-index yielded an higher H-Index that was too high because it included articles by an unrelated biologist Phil Jones. Mann advised Jones that he would use the inflated value in the nomination form anyway.)
In subsequent discussion at the Nature blog, Heffernan was unable to see any inconsistency between Jones’ delivery of confidential data and any obligations to the NMSs to keep his data confidential. Jones’ desire for coauthorship may be understandable in academic terms, but it is totally irrelevant to his obligations under the NMS confidentiality agreements. The interest of the NMSs would be in the confidentiality agreement, not Jones’ H-Index. An increase in Jones’ H-Index is not a valid reason to breach the confidentiality agreements. Even Nature should get this.
There was an even more remarkable breach of the confidentiality agreements 20 years ago. The collection of station data by CRU was financed by a unit of the US Department of Energy (the Oak Ridges Nuclear Lab) – DOE financing continued for over 25 years. Notwithstanding confidentiality agreements in effect in 1990 (and CRU says that many agreements date from the 1980s), Jones delivered the data set to the Oak Ridges Nuclear Lab (Carbon Dioxide Information Center), who placed the data set online. This version is available to this day ( http://cdiac.ornl.gov/ftp/ndp020/ ).
Conclusion
The language said to be standard in CRU confidentiality agreements stated that the data was to be used only by CRU and only for the construction of a gridded temperature index and not to be used for other purposes and not to be passed on to third parties.
Jones flouted these terms. He passed confidential data to the Oak Ridges Nuclear Lab in the US, who had funded the collection of the data. He passed confidential data to his close associates, Michael Mann and Scott Rutherford. He passed confidential data on to a relative stranger in circumstances where he expected to increase his personal H-Index.
The reaction of the Community to these revelations is just as important.
Jones’ violations of the confidentiality agreements didn’t bother Gavin Schmidt, who said that there was nothing-to-see:
CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on
Similarly, Nature said that there was nothng-to-see-move-along:
But for much crucial information the reality is very different. Researchers are barred from publicly releasing meteorological data from many countries owing to contractual restrictions.
The idea that masters of the universe should be constrained by confidentiality agreements doesn’t seem to have occurred to them. It should have.
Postscript
There’s an interesting example of this attitude in one of the Climategate Letters that arose out of the very first post in the current FOI sequence. On May 11, 2009, as mentioned in a CA post here, I had noticed that the UK Met Office webpage on CRUTEM stated:
To obtain the archive of raw land surface temperature observations used to create CRUTEM3, you will need to contact Phil Jones at the Climate Research Unit at the University of East Anglia.
No mention of restrictions due to confidential agreements. On May 12, 2009, Peter Thorne of the UK Met Office wrote to Phil Jones ( see Climategate Letter):
Mr. Fraudit never goes away does he? How often has he been told that we don’t have permission? Ho hum. Oh, I heard that fraudit’s Santer et al comment got rejected. That’ll brighten your day at least a teensy bit?
The Team has such an elegant rhetorical style.
Thorne was correct that our IJC comment had been rejected, but, at that point, I hadn’t commented on the status of our Santer comment. On May 28, 2009 (CA here), I stated:
Ross and I submitted a comment on this topic to the International Journal of Climatology, which we’ve also posted on arxiv.org. I’m not going to comment right now on the status of this submission.
It appears that Santer co-author Thorne was a reviewer of our submission, a reviewer who recommended rejection and who then raced to tittle-tattle about the rejection to his confidante, Phil Jones.
Climate Audit 
199 Comments
At the minimum this is another example of sloppiness by the Team. It also highlights their arrogance and disdain of anybody that they considered to not be a “climate scientist.” At this point I’m not shocked, but I’m still disappointed. We really deserve much better for our public expenditures.
I concur.
Some day, deniers will learn that interpreting written agreements cannot be done simply by a reading of the text. There is too much noise in the signal. What CRU and The Team have done is to apply smoothing routines and Bristlcone proxies to distill the true intent of the parties to the agreement. The smoothed data clearly shows authority for the team to use data in any way they see fit, including withholding of the data from third parties, but not requiring it.
Steve, I’m sure they would happily provide you their source code for the smoothing if you submit an FOI request.
Did anyone ever write to some of the originating meteorological services to find out if they do indeed have confidentiality agreements with the CRU? Maybe I missed it but the originating services seem to be pretty quiet about these supposed confidentiality agreements.
Yes.
Steve: May your 2010 be filled with the data you’ve been asking for for years.
Regards
Has anyone done an FOI request to get the inundating pile of requests?
Yes, I have, for the time being I asked for the simple total number of FOIA requests they’d received between January 2005 and November 2009 (I suspect they’ve been inundated with FOIA requests since the publication of the climategate files).
I received a prompt reply referring to a complaint they’d received from someone else two weeks previously about refusing to respond to FOIA requests. Very baffling.
I have written back to them telling them about the mix-up, and referring to my first request which was simply the total number of FOIA requests. It does look as though they’re inundated at the moment though if ther’re making this kind of simple mistake.
Everyone must be very worried about the Sir Muir Russell investigation. The terms of reference are pretty wide, which at least shows that the University’s Vice Chancellor is aware of the gravity. In the UK this sort of investigation rarely comes out in favour of the obvious conclusions, so it will be interesting how Sir Muir, a physicist, and from what I hear, well respected, senior civil servant, will come up with.
Don’t look for anything too dramatic though, he is a brave man indeed if he is willing to take on the new fundamental environmentalists.
Bonfire, petrol. You’ve been hinting at this post, presumably while mulling how best to express your disgust at having confis used against you, and I’m very glad to see it.
Steve, a minor point, but merely publishing a paper does not increase your H-index. If that paper is subsequently cited then the H-index might rise, but the paper will have to acquire a number of citations greater than your current H-index before it can contribute, and in Jones’s case this is quite large. Full details of the H-index can be found at http://arxiv.org/abs/physics/0508025
You may recall the comment in CRU’s response to my appeal of their refusal of my FOI request:
But even this statement seems to suggest a technical error rather than concern about a serious breach.
Steve: Funny that they didn’t consider the NERC policy quoted on their website:
The usual glittering scalpel in evidence – a classic demonstration of how much more damaging putting things *in* context is turning out to be.
But perhaps a slight qualifier is in order: the fact that no exchanges asking for permission to share are evident in the leaked emails, doesn’t mean that permission wasn’t sought and granted.
I know where I’d place my bets, but absence of evidence isn’t evidence of absence.
It is whenever months are required to turn up something in your defense which should have taken minutes.
I’ve done more than my share of data forensic work for the courts. Taking months to present what is then lauded by the defense as reason for a “simple and obvious dismissal” does not go unnoticed. A solid timeline in the hands of an effective prosecutor is as damning a piece of circumstantial evidence you’re ever likely to see in a court of law.
When the political will shifts back to the right in the United States, I would expect a host of the Climate Grifters to see the inside of a court house. Ponzi science is slowly turning into a hanging offense in the eyes of those Americans who already can’t afford the enormous energy bills this winter alone is causing.
Throughout the Climategate Letters, there are many incidents in which Jones and associates disregard confidentiality. I cited one example in another thread – where Jones sent Mann confidential reviews of a McKitrick and Michaels article for MAnn’s use with his department chairman.
Another letter you should consider is the peterson to Jones letter where peterson says he doesnt want NCDC mentioned WRT confidentiality. Peterson and karl appear to disagree on the issue. based on the time frame it looks like jones was asking paterson and NCDC for cover, or at least confirmation that other people had confidential data as well.
It’s hard to see how station data gathered by public agencies qualifies as “confidential” information in the first place, alleged agreements and contracts with CRU to the contrary notwithstanding. It is also difficult to see how the apparent unauthorized release of this data (to Rutherford and perhaps others) was to the detriment of the Met Office and other confiders. If it were, would not the agencies attempt to estop future disclosure and seek redress?
Strategic disclosure of reviews of articles to Mann is more seemly, although here too who was harmed by the release? In my way of thinking, ALL of this information is or should be in the public domain.
Researchers have a very elegant solution to the confidentiality issue: fund studies privately. If one chooses to accept public funding of any kind, one should be prepared to accept the consequences. FOI is the public’s form of peer review. Peers as in tax paying citizens.
Jones’ pre-Climategate behavior amounted to a sophisticated shell game. His antics were exposed not a moment too soon.
All of the US data has been posted online for years, this then implicitely gave them permission to release all of their US station names, and any adjustments they might
have made, especially since this was funded by Uncle Sam. They can’t hold up the release of the vast majority of countries data simply because they have agreements with a few. This was clearly a rope-a-dope strategy devised to delay, delay, and delay any look into the specific stations they were using. Clearly the ‘confidentiality’ defense is nothing but a ruse. I guess their thinking is: anyone we decide to give it to is ok, anyone we don’t want to give it to let’s invoke the C-word. The question is, what is so scary about the station lists? Why
would they want to destroy it instead of releasing it? Most of the stations I have looked at so far in North America show warming from the 1800s up to the 1990s. Is it the North-Face-of-the-Eiger fall-off since 1998? I don’t get it.
Steve: It’s really hard to understand. If you take their excuse at face value – as I have here, it really exposes them to more serious complaints than FOI obstruction. In most walks of life, giving confidential data to friends – giving them sort of inside edge – is much more serious than obstruction of critics; You have to wonder at a strategy that exposes them to that sort of potential complaint.
In terms of what they were afraid of, reading the Russian IEA paper released in mid-Dec 2009 may have an answer.
I am curious about the US station data. Prof. R Carter of Australia has called it the “gold standard”. When I look at the US annual temperature time series, for summer (i.e. when the extremes of heat would presumably be most harmful), I’m not seeing much of a warming trend. In fact the 30s where as warm or warmer than the recent decade. If you start with the 30s there is really no warming trend at all for 70+ years in summer. Precipitation may have increased slightly over all, in the summer growing season it doesn’t seem to have much of a trend, just correspondence with lower levels at the temp extremes indicating drought.
The warming seems more pronounced in the fall, winter, and spring. This is when urban heat island would most likely effect results. Is there some reason the effects of CO2 etc. would abate in the summer?
I can add to that.
My data is from NCDC 1880 – 2009
Northern hemisphere.
Summary of Highs
Dec, Jan, Feb 51.8%
March April, May 23.6%
June July, August 1.8%
Sep, Oct, Nov 22.7%
100.0%
6 mths Nov – April 88.2%
6 mths May – Oct 11.8%
100.0%
Summary of Lows
Dec, Jan, Feb 48.8%
March April, May 20.2%
June July, August 5.4%
Sep, Oct, Nov 25.6%
100.0%
6 mths Nov – April 84.5%
6 mths May – Oct 15.5%
100.0%
Steve: Totally OT. Why do this?
“In most walks of life, giving confidential data to friends – giving them sort of inside edge – is much more serious than obstruction of critics . . .”
Yes; in high finance I believe it’s referred to “insider trading.” If you’re caught, the consequences can be severe.
These people on both sides of the atlantic seem to be truly messy bookeepers, losing or mislaying files all over the place.
I hope the enquiry into jones and mann’s work is thorough and the results are made public within a reasonable time.
BTW does anyone know whether that early record of temp was really deleted/ lost? ( because of storage problems?) or is that just another story dreamed up along the way.
I hope the enquiry into jones and mann’s work is thorough and the results are made public within a reasonable time.
When a publicly funded institution investigates one of its own, its primary concern is usually its own reputation, on which much funding depends. Given that, I’ll be very much surprised if either investigation turns out to be anything other than a whitewash that clears both individuals of any significant wrongdoing.
The full extent to which the game is rigged in favor of the alarmists was made clear to me by the fact that Wegman’s confirmation of the flaws in Mann’s methods made absolutely no difference to Mann’s employer, his fellow scientists, the IPCC or the media who covers the whole circus. If these guys can get away with what Mann and his cronies have done to date — in full public view — what possible import can a few indiscrete e-mails have?
Jones may get thrown under the bus for his overt conspiracy to evade FOI laws — though it will be stressed that his errors do not impact the science at all — but Mann and everyone else is bulletproof. Just my opinion. I hope I’m proven wrong.
Vito,
I think just about everyone agrees that taxpayer-funded work should be readily available to the taxpayers who funded it. But the obligation for transparency goes further. Government policy makers should never rely upon purported scientific findings which cannot be replicated. In other words, we may not be able to force someone to make available his privately funded research, but we should never make policy based on such work. The source of the funding shouldn’t matter when it comes to policy. Either the public has access to everything necessary for replication or the findings aren’t appropriate for use in making policy.
As a side note, replication isn’t limited to scientific fields. When setting land speed records at Bonneville, the vehicle and driver are expected to turn around and do it again in order for the record to be official. In drag racing, the driver and vehicle are expected to repeat the run within a few percentage points of the record or exceeding the record for the record to become official. Once is a fluke; twice is taken more seriously.
Neville – In my opinion, not so much messy bookkeepers, but both career academicians and rank amateurs when it comes to any sense of record keeping, due diligence, and just plain old common sense. The old phrase “idiot with a badge” comes to mind…
Steve
How about
Masters of the Climate rather than Masters of the Universe?
Mann’s use of what he knew was a falsely high h-index is breathtaking, given his letter yesterday in the WSJ.
Pat Michaels
There is another minor mystery here. What is the full name of P.D. Jones? I have seen variations like Phil and Philip, but a while back there was also a strange one. The relevance is that some official agreements require the full name of the signatory, one reason being to avoid mixups with others. If the full name was “Philacre” or similar, and it was not used where “print full name” was on the form, would this invalidate the form – such as a confidentiality agreement?
My apologies for the guesswork portion here, but I’m raising it to seek the truth.
Seems to me, in trials, judges and jurors catching witnesses in this kind of knowingly false statement are entitled to dismiss any and all statements by the witness.
With the world as their judge and jury, at a minimum, it is time to flush every statement connected with Michael Mann and start over with more scientific and impartial researchers.
Note that Phil Jones was indeed appointed a fellow of the American Geophysical Union: http://www.agu.org/about/honors/fellows/alphaall.php#J The AGU is a “non-profit corporation” in the “District of Columbia”. One could ask for the letters in nomination for his support to see whether Mann maintained his deception in the eventual letter; or ask the AGU to check this particular fact.
Agreed.
This is an interesting letter:
http://online.wsj.com/article/SB10001424052748703478704574612400823765102.html
‘Society relies upon the integrity of the scientific literature to inform sound policy. It is thus a serious offense to compromise the peer-review system in such a way as to allow anyone—including proponents of climate change science—to promote unsubstantiated claims and distortions.’
?!?
Had a look at your “interesting letter” link. You’ve got to hand it to our Michael, he’s sure got gall ! However , I thought the Editor’s Note was the most telling part.
Editor’s note: Several weeks ago, we invited Mr. Mann to write a feature explaining his email exchanges that were revealed as part of the University of East Anglia Climatic Research Unit’s document disclosure. He declined.
Re: Hillbilly (Jan 2 04:50), Poor Michael Mann. He is so deep in the hole, his only reaction is to dig faster. His letter is proof that it is better to remain silent and let your opponents think you a fool, than to open your mouth and remove all doubt. I almost feel sorry for him (but not really).
It actually shows Mann’s mindset, he has no fear of being caught, and when he is caught, a la hockeystick, he simply brazens it out. This charactersistic has led to producing the hockeystick, for most people the air-brushing of the MWP would have been too great a step to make, especially if when caught you are likely to finish up in the Scientific Hall of Infamy, but not Dr. Mann. Maybe this self assuredness has given others the confidence to follow suit, leading to a folie a plusieurs where each member is supported by the seeming self-confidence of all the other members. It naturally follows that what we see from the Climategate papers is their normal behaviour, they will hide, lie, break the law and confidentiality agreements and generally do things they would never have done in a thousand years because of their support group. Who knows what’s driving them in their extraordinary behaviour.
Of course insider trading on the stock market usually involves friends or other people in a privileged group getting info before everyone else for financial advantage.
That type of advantage can’t be the case here but it’s probably the wrong thing to have done and should be reported by the msm at least.
Just followed through the link to the discussion concerning Phil Jones’ “H” index.
How can Mann possibly spin this?
Presented with an explicit and, at least on this occasion, honest correction by Jones, he unambiguously asserts he will use the incorrect and misleading figure.
No interpretation could classify this as anything other than a knowing deception – a blatant lie.
He can’t. And, because of his actions, he has probably given anyone who was unfavorably reviewed by an academic committee that he sat upon entre’ into the process, he has really opened up a can of worms here! Also, what about those who received favorable reviews? IE his climategate pals?
Gah, sorry I didn’t spot the previous comment. I didn’t intend to labour the point.
Jones: “I think this got rid of some biologist from the 1970s/1980s, so go with 52.”
Mann: “OK–thanks, I’ll just go w/ the H=62.”
What? Why? !!!
About the data given to Scott – maybe Jones was the corresponding author for that paper. Surely data-sharing would be permitted if that is the case.
Mann: “OK–thanks, I’ll just go w/ the H=62.”
Uh… could be an unfortunate typo. 5 is next to 6 on the keyboard and what do we know about Mr. Mann’s typing skills? At the very least, if there is a public document in which Mr. Mann asserted what Jone’s H index was it should be inspected to see if the number 52 or the number 62 was used.
Like mine: Jone’s should be Jones’
Re: JT (Jan 2 13:53),
Actually it should be Jones’s, but that is a can of worms best left unopened.
Re: jeez (Jan 2 15:56), NO. It should be Jones’. Read about grammar somewhere. This is 4th grade stuff, actually.
No, Jones’s is correct – spell it like you say it.
As in the forthcoming movie : Phil Jones’s Diary.
Oh dear me. Ahem. Let’s see if we can straighten you boys out here.
Punksta is correct, inasmuch as it should be spelled how it’s pronounced, but Steve is the one who spells it correctly. That’s “Jones'”, pronounced as spelled, rather than as “Jone-ziz”. Some people have argued that the spelling Jones’s is correct by analogy to other possessives, but they still pronounce it as “Jones” and not “Jone-ziz”.
“This is 4th grade stuff, actually.”
Sorry, old boy, but if you’d stayed at school past 4th grade, you might have learnt the right answer 🙂
This is a very fair point. I actually think it reads a bit more naturally if you assume there was an unfortunate typo and he meant to say 52.
But if there were any follow-up documents that would handily damn or exonerate on this specific point.
Yes, it reads more naturally as a typo (or as if he confused the two numbers). I was wondering why they were even discussing it (as if there was an alternative, so to speak), until I figured out that this means that he has 52 publications with 62 citations each, so his real H-index (if Mann’s definition is correct) is most likely higher than 52 (for example something like 57 publications with 57 citations each), since the list contains only publications with a minimum of 62 citations. This would explain why he assures that this is “almost certainly higher than the vast majority of AGU Fellows”.
Hi Steve,
I am a newbie since Climategate broke. I was wondering if you could have worked with the data sets and code minus those covered by confidential agreements (if they had been made available to you) in a meaningful way. Would the results be valid?
If there was explicit intent to honor agreements not to share the data, then no scientific papers involving the data could have been published in journals that required data and software to be made available for public scrutiny (review by other scientists for verification purposes); and specific review of the data also would have been denied to IPCC. One would think that this being the case, then a prepared list of sources requiring permission to release the data would have existed long before the substance of this issue surfaced.
Also, if DOE contracts were involved, then DOE as part of the contract, would require detailed annual/quarterly reports including data sets and software programs be submitted to the DOE contracting office. Actually I would suspect that DOE would have some form of joint ownership of the data and analyses that formed the backbone of the research contracts. DOE doesn’t just write blank checks for research grants; it wants something concrete for its money. It might even be possible to find out who were the DOE individuals who served as contract monitors for Jones’s contracts; they could provide information about the ins and outs of such contracts because such contracts would not have been “classified” contracts requiring an element of security.
Steve: DOE were contacted years ago and refused to do anything to require Jones to give the up-to-date data that they had paid for.
If Dept of Energy awarded a grant, there may be a premise that it supports the independent efforts of the university of East Anglia, unless such a grant included a supplemental special condition that obliged the receipient to report information to the public or to DOE.
A basic distinction between a grant and a contract may pertain. A contractual relationship could make UEA more obliged to comply with requirements that govern USDOE, whereas a grant is much more liberating for a recipient. A grantee is generally regarded as independent of the grantor. A grant is akin to a gift, whereas a contract is a stronger obligation and often has more expectations associated with it.
“Mann advised Jones that he would use the inflated value in the nomination form anyway.”
Let’s hope the Penn State Ethics Committee agrees with Dr. Mann’s interpretation of honesty. Otherwise I would think this would be a career limiting move, at least in any other Faculty or University.
Bet Penn State ignores it and then the blogosphere and the op-ed world is going to light them up.
It’s a pity you don’t live in Sweden, which has the oldest and possibly strongest FOIA in the world. That may be useful for any data that may have come into the hands of a Swedish research institution (as long as it is government operated).
The Swedish FOIA was a part of the 1766 Freedom of the Press Act (believed to be the first of its kind in the world). It is called “offentlighetsprincipen” (approx “public principle”) and states that all government documents (also with local authorities etc) shall be open to the public.
You can visit an institution (you may be anonymous; you may also do this over the phone, via E-mail etc) and ask for any doucment. It shall be yours within minutes, unless *specifically* declared secret. If there is no classification, the insitution must have an officer who on the spot shall make a security classification.
If you are denied a document, you can go to an administrative court, which shall rule in the case *with priority*. Ie the court must drop other cases, and decide the FOIA request first.
There are a lot of intricate details to the Swedish FOIA, but the basic idea is that you’ll have a good chance of getting documents from a Swedish institution. (The same applies for Finland, which in 1766 was a part of Sweden, and kept this “offentlighetsprincipen” even under Russian rule and later as independent.)
Scientific data etc is probably not classified at all. “Agreements” to keep data under cover would probably be void (they would have to *prove* damage from handing out the data, and could be tricky).
So, has any info of importance come in the hands of a Swedish institution? (Or a Finnish one.) Get things from them! (The local climate skeptic group Stockholmsinitiativet should be happy to help.)
–Ahrvid
Ps. As I see it, Jones and cohorts just try to hide behind legalese. They even admit to that in the CRU mails. Even under normal scientific principles – data should always be open! It’s only if you know that there’s something fishy with your work you want to keep data under wraps. Handing out data should be no burden. You have it in a directory on a server. You zip it. You E-mail it. Should take less than five minutes.
LOL…Exactly….a few minutes is all it would take….
I have just rx a copy of the entire DEM (1km grid) for Antarctica…it was posted on http://nsidc.org/data/nsidc-0082.html…it is around 47 mB, zipped and ready to go…it contains the results of a rather lengthy and expensive survey…but it is in the public domain….all of the CRU data should have also been available in the same manner…it’s simply a joke
That’s a very good point, really. What CRU are doing is broadly-speaking a form of cryptography. It’s a standard cryptoanalytical technique to look for information which has been transmitted both in the obscured system, and in systems which have been cracked. In this case, the ‘insecure’ channel is anything that has been sent to Sweden or Finland. The analogy holds up further, in that I expect the data obtained that way will contain information that can been fed into the analysis of what CRU are attempting to with-hold, allowing further requests to be made.
Steve.
Another question. Would it have been difficult for CRU or Phil Jones to separate and delete confidential data sets from their work to be able to send the rest of the information to you?
If they had the information in a well documented database, this would be trivial. Unfortunately, I think we’ve seen that their record keeping has not exactly followed “best practices”.
During June/July 2003, Phil asked Osborne to pass data to Tom Crowley:
http://www.eastangliaemails.com/emails.php?eid=340&filename=1057944829.txt
“So I have these data and they are attached in the following files.
western US and western Greenland are in file “mann12prox.dat”. I didn’t have time to extract just these two series from the full file, so the file contains 11 others series too. Please do *not* use the others because I’m not sure whether I am free to distribute them or not.”
[This appears to be partly stimulated by the DeFrietas controversy]
Previously (Nov. 2002), Crowley is repling about other data sent to him by Phil, including intermediate results:
http://www.eastangliaemails.com/emails.php?eid=280&filename=1036182485.txt
“Dear Tom
after a little detective work we have deduced that the data sent to you constitute a version of Northern Hemisphere Land temperatures (april- sept) produced by PCA regression using regional average density chronologies (ie the JGR paper you refereed I believe). It is true that high frequency component is not in my opinion optimal in describing the relative magnitude of extreme inter-annual extremes. This is to do with the unpredictable weighting ascribed to certain areas (tree-density series) in the averaging of the original raw data ( this is boring and I won’t go into it unless you
really want me to)…
…If you would like us to dig
out the composite series or discuss specific aspects of the logic or uncertainties associated with the different large averages let me know.”
Even back in 2000 there was a quid pro quo being established see last ‘eastanglia’ ref below:
http://www.eastangliaemails.com/emails.php?eid=193&filename=969652335.txt
“>Regarding our MBH98 and GRL99 datasets, I’m pretty sure that Scott put those
>on anonymous ftp for you some months ago. So you *should* already have had
access to all the data we used. In fact, it was only a few select series of
Malcolm’s that weren’t made available from the get-go. So data has never
>been an issue for us. I’m happy to hear that it is not an issue for
you/keith/phil and that you are ready to make your density data available…”
Also, in reply:
http://www.eastangliaemails.com/emails.php?eid=195&filename=969912361.txt
Mann to Briffa, apparently before an official collaboration, just the prospect of one:
“So I’m sure we’ll be able to find common ground. Tim will have free access
to our data and codes, and can make the comparisons indicated below. We of
course appreciate your willingness to make available to us the tree ring
density data….
… It is my feeling that the relatively short time Tim has with you ,
>might be best spent getting to grips with the finer details of your “old”
>and “new” approaches, including the details and results of your other work
>that is only partly described in the publications ( seasonal
>runs, different data sets etc.)”
Lastly, for now:
http://www.eastangliaemails.com/emails.php?eid=59&filename=898099393.txt
(mmann to pjones)
“Date: Wed, 17 Jun 1998…
…I remain committed to doing this with
you guys, and to explore applications to synthetic datasets with
manufactured biases/etc remains high priority. It sounds like it
would all fit into the proposal you mention. There may be some
overlap w/proposals we will eventually submit to NSF (renewal
of our present funding), etc. by I don’t see a problem with that
in the least.
Once the collaboration is officially in place, I think that sharing
of codes, data, etc. should not be a problem.”
1. Does PJones get to decided when a collaboration is “official” and then become free of contract restrictions?
2. Was Crowley working on the “same grant” though he’s at DOE, not CRU.
3. Does collaboration on an IPCC chapter free one from local NMS contract restrictions?
With respect to 3., I seem to recall the use of IPCC collaboration to avoid FOI
Another interesting observation is that in the original request for data from Norway, the requester states that they will not redistribute the data, effectively asking to be bound by confidentiality. The Norwegians echoed this back so perhaps it was their desire anyway, but had they failed to say anything about confidentiality in their response the CRU would still be bound by the promise not to redistribute the data.
The CRU might have been unnecessarily promising confidentiality for no other reason than to be able have an excuse to control the data availability while giving the impression it is required by the source when that may not be their desire or requirement.
This could be the genesis of some of the mislaid and verbal agreements; some confidentiality requirements may well exist that need never had.
Steve,
Just curious…using the NERC data as an example, can it be argued that the NERC data was for CRU staff only? What if scientists from other organizations around the world were working on the NERC project with CRU, would they be allowed to see the data? Does CRU contract out portions of a project for others to complete especially if some scientist somewhere other than CRU is a specialist with some piece of the data or analysis? And if working with a CRU sponsored project, does this qualify scientists from places other than CRU to see the data?
I’m not saying this is what happened or is true, just wondering if it is a possibility?
If they we serious about this they would have a price on the data for Steve.
I didn’t mean to get off subject, but when one considers his most profound contribution to science is a deception, as shown in the IEA paper, and a decade follows to conceal that fact, deliberate lying about the H-Index is just standard procedure for this guy.
Don,
Let’s not be unfair to Jones. After all, even fairness on many issues puts him in a bad light. Recall that Jones was warning Mann that the computed H-index was wrong owing to another Phil Jones. It was Mann who apparently went ahead with the wrong data (I wonder if he’s ever done that before?). And the ClimateGate Deep Throat cunningly did not reveal to us Jones’s later email to Mann insisting that he do no such thing…
Rich.
Jones himself could easily put that one right then.
Steve – minor correction “Oak Ridge National Laboratory”, not “Ridges”.
I once stayed at a nearby motel which for reasons unknown had been given a Scottish decorative theme. It was quite amusing to hear the receptioniste, dressed in a psuedo Scots kilt, start talking in a broad Tennessee accent. Maybe they were selling moonshine Scotch whisky without a confidentiality agreement.
Geoff, the Appalachians were settled by Scotch-Irish heavily in the 1700s bringing
with them the music that later became blue-grass and country music. Big Scots influence, thus the kilts.
Geoff:
Southern States were dumping grounds for many of the displaced Highlanders from 1745 Rebellion. There are very strong Scottish/Celtic influences in the Appalachian Mountain regions.
Steve: One branch of my family (grandmother Wallace’s branch) went to North Carolina around that time; they moved to Nova Scotia after what is known here as the “American Revolution” – I understand that people in the present U.S.A. have their own local term for the event.
Even the rocks are the same!
Hey, we locals are mostly Scottish around Oak Ridge, TN. We just make our whiskey out the locally available grain. When corn is what you got that’s what you use. We are pretty clannish down here in Eastern Tennessee, Western North Carolina, and South Western Virginia you know. Even the Cherokee Indians down here have Scottish surnames.
This explains a lot.
“Law being too tenuous to rely upon in [Ulster and the Scottish borderlands], people developed patterns of settling differences by personal fighting and family feuds.” Thomas Sowell in Conquests and Cultures.
In other words, don’t mess with the Scotch-Irish, Dr. Mann
Aren’t we getting a bit OT?
OT obviously, but I am from the Oak Ridge/Knoxville area originally, thus I have fairly noticeable Appalachian accent. The Appalachian accent didn’t just “appear” out of nowhere. It has strong roots in the Scott/Irish settlement of the region. For instance, the pronunciation of words ending “ire” (i.e. fire) are often pronounced “ar”…Even the proper noun “Ireland” might be pronounced “Ar-land” by many. If you listen to modern day Irish people speaking “The Queen’s English”, you’ll hear many of the same vowel slurs. Some people think the accent sound “ignorant” or “hick” but it has an origin that is quite remarkable. You might even note that the common pejorative term “redneck” is an old English slur for Presbyterian Scotts.
Interesting discussion. I typically handle the contractual arrangements for confidentiality, non-disclose and trade secrets. Not everything is worth covering here but here are some of the topics you normally cover. Further, what intrigues me is that the confidentiality agreements in the document to which you linked cover so little.
Another reason it’s worth mentioning is that Britain and Canada fall under the common law – which defines a Master/Servant” relationship for employment – which may or may not conflict with any intended use – so it is best to spell out the rights and obligations.
In some cases you want these agreements to specify the freedom of the information, in some cases you want to specify a restriction. If you don’t the courts can later decide what is customary under all the circumstances – a concept not normally applied (adhered to? , recognized?) in the USA.
So normally you define the following…
Proprietary Data – what is it – paper, records, books, table or whatever – so you know what you are discussing.
Disclosure obligations and rights – who can see it, employees, sub-contractors etc.
Terms of the disclosure rights – do the employees and subcontractors have to sign an agreement – etc.
Identification – a requirement to mark the data so that people know that it’s confidential.
Restricted use – copying, giving lending etc.
Non-Competition – i.e. not re-sell or use the data to compete with the vending/supplying service.
Loss theft or destruction provisions.
Public domain issues – what happens if the data does get released into the public domain – what are the subsequent obligations.
Copyright and ownership of derivative works. Plus moral rights.
Conflicts with other agreements or obligations. Very important in this case.
Term and survival
All the usual legal stuff such as waivers, entire agreement and equitable relief etc.
So all in all I am not sure what would have been permitted to the people who passed along the data – if there were partnership agreements – it might be permissible – since nobody really covered much of anything in their agreements – such as they were/are.
Most confusing. My guess is that they could have sent you the data by declaring you an “auditor”, partner sub contractor — or perhaps pal. (Sorry about the auditor joke.)
Re: sending “loads of data” to Rutherford, the term “loads of data” is too ambiguous to know for certain what was sent.
Do you have any evidence that the station data sent to Rutherford included any of the data covered by confidentiality agreements?
Do you have any evidence that Jones did not receive approval by the countries in question to share with Rutherford any of the data covered by confidentiality agreements ?
I am sure that Steve would rather answer you — but I have a spare $0.02 tonight.
First — you should read the agreement document.
They all had a “non-copy” provision — as I recall. Maybe I’ll re-read it.
Dr. Jones did say that all of the data was covered by some sort of confidentiality agreement. After all — that’s why he could not share with Steve — or anyone else — as much as he would have liked to!
If he had received agreement to share the data I am certain that he would have informed Steve and offered to supply same. He does seem to be a gentleman and all that!
As for “loads of data” — perhaps that query should be addressed to Dr. Phil Jones hisself. But I do know from experience that it’s a lot more than a “whole bunch” — being a computer geek an’ all that. Beyond that you are asking for something none of us could possibly know. It would just be speculation.
Happy new year Susann!
I thought that Jones claimed that only certain countries were covered by confidentiality agreements, so it seems to me that much of the data *could* be released if Jones felt inclined. The “some” meant that he couldn’t release the entire record because some of it was covered by confidentiality agreements. Maybe I’m wrong about this.
If a colleague wanted data from a country not covered by confidentiality agreements to do some kind of specific research, perhaps the confidentiality agreements would not matter?
I tried to see what papers Rutherford had written that might have used the “loads of data” myself so I could get an idea of what data he was sent, but I haven’t finished going through what is available online yet.
My point exactly. 😀
Happy New Year WillR!
snip – the Harry file is OT to this issue
My questions were intended to clarify what is known and what is not known about this and to point out that what has already been claimed is also speculation and is not based on all the necessary information.
If I understand correctly, Steve McIntyre asked for the entire data set, but since some of it was covered by confidentiality agreements, the entire data set could not be released.
Does this mean that a “subset” of it couldn’t be released — the data that they knew was not covered by confidentiality agreements?
I don’t know that and I don’t believe the emails themselves tell the whole story.
Steve: None of it makes “sense”. The Met Office has just released a subset of the data, which apparently was not confidential all along. No apology, no nothing.
shewonk,
CRU claimed they no longer had the confidentiality agreements (the old “dog ate my homework” excuse). I believe CRU was claiming they no longer knew for sure what was covered by the confidentiality agreements, so they were not releasing anything.
Steve,
Am I remembering that correctly?
Steve: None of it makes “sense”. The Met Office has just released a subset of the data, which apparently was not confidential all along. No apology, no nothing.
McIntyre, you were told all along that 98% of the data was available from GISS GHCN Presumably this plus any newly freed up data is what the MO has released. Why should they apologise that you failed to obtain the data from GISS?
No nothing – does this mean everything?
Steve: Puh-leeze. I’ve obviously downloaded GISS and GHCN data versions. There are many posts on the topic. CRU “value-added” data is not necessarily the “same”. And for your information, the Met Office did not take the position that CRU “value added data” was available at GISS/GHCN. They took the position that it was tied up by confidentiality agreements – an entirely different argument.
Really? That simple is it?
By all means, please indicate where we can find the exact list of stations used. Which version is used in a particular case – the adjusted or unadjusted? What other adjustments have been made by CRU as well? Try to be a little more realistic with your simplistic observations.
“Presumably …!” Yeah, make a guess…
A candid reader might have thought that WillR was just coatracking Harry in a comment that was related to the subject of this very thread.
That wasn’t the intent — and if I did I apologize to anyone offended.
However, I do not believe that they did not always know the provenance of their data and that they preferred to keep it all secret rather than breach an agreement unless you were a “friend” of course. ..hence the attempt to connect the information I had.
It appears that all the agreements allow you to create derivative products that can be distributed. Perhaps I should also say that other do not dis-allow it as well. If Steve is thinking of the same data UK Met office set now on my system then I believe the explanation for that data set is simply that the UK Met office can make any policy or decision that they wish to in regards to their data sets where it is a derivative product.
For American posters/readers — note that custom and precedent are far stronger in British and Canadian common law for instance. So I suspect that they got a legal opinion reflecting “custom and precedence” and reading of the agreements such as they are within that particular research community — in other words since you have always shared your product when you wished to, and others including the providers expect you to — go ahead and share — if you wish to.
So to answer Steve’s implied question — it is not surprising. They could have shared their derivative product at anytime — if they had wished to — without fear of running afoul with any of their agreements. I am only speculating that they had never clarified the position and only now bothered to do so via legal advice — and this because of all the attention recently received.
Hopefully that is more clear and on-topic.
Shewonk (Susann) – there is a whole long history to this which is covered in earlier posts at CA. If you use the archives and category drop downs at the left you should find as much of the “whole story” as is known: You are not the only one with more questions than answers!
https://climateaudit.org/category/foia/
There have also been good posts elsewhere – for example at WUWT, the AirVent and Bishop Hill:
http://wattsupwiththat.com/2009/11/24/the-people-vs-the-cru-freedom-of-information-my-okole%e2%80%a6/
http://bishophill.squarespace.com/blog/2009/12/30/the-east-anglia-publication-scheme.html
Susann,
According to UEA Director of Information Services Jonathan Colam-French, restrictions applied to the data subset that was sent to Georgia Tech, and that data was provided without first obtaining the necessary consent:
I have not seen Steve’s e-mail from 2 September, so I do not know for certain that Mr. Colam-French is referring to the same data transmission to Georgia Tech that is the subject of this post.
shewonk: “I thought that Jones claimed that only certain countries were covered by confidentiality agreements, so it seems to me that much of the data *could* be released if Jones felt inclined.”
It’s not about how he felt – it’s about his obligations under the law. It may come as a shock to you that the law doesn’t allow individuals to decide whether it applies to them or not.
It’s not about how he felt – it’s about his obligations under the law. It may come as a shock to you that the law doesn’t allow individuals to decide whether it applies to them or not.
Sarchasm: The giant gulf (chasm) between what is said and the person who doesn’t get it.
In any other profession, for example: the legal profession, actions indicated in the Climategate emails would undoubtedly subject the individuals to expulsion investigations. The equivalent behavior in law would be substantial grounds for disbarment. Yet, I have not heard of a single professional association to which these individuals belong announce any kind of investigation. Does not membership in AGU require ethics in professional conduct for membership?
I think that this is about as lose as you get…
http://www.agu.org/about/mission.shtml
Engineers and accountants definitely have a code of behavior and ethics. The behavior in the emails would be difficult to explain to a disciplinary committee of same.
Professions of course do have ethical committees and boards that would take such indicators of misconduct very seriously. The fact that we see no evidence of any such action by scientific bodies has made me wonder just what is the status of science as a profession. It would be very hard to take scientific professional bodies, in general, seriously if no investigation and hearings by relevant bodies takes place. I hope that disinterested bodies of note question the relevant associations and point out the greater implications. The UN itself has science regulations which possibly have been breached – I recall reading them some time ago. For disinterested science bodies I would think a review of science ethics and procedures as a whole would be a very worthwhile agenda for this year to restore a public faith in scientific bodies and practices.
In science there is no granted license as there is in medicine, engineering, and law. One needs not even have a degree of any kind to publish–and in fact no one has ever asked me to show proof of a Ph.D. when I submit my papers to journals. Thus, no disciplinary bodies. If you commit real fraud–making up data–you might lose your job. But that is about it.
Hi Steve–
I know it’s OT but I’m watching you on Fox News Reports and I wanted to say how much I appreciate all you’ve done and endured in your search for truth amidst all the rhetoric. I don’t know if this is a rerun but if I’m late with my gratitude please forgive me.
God bless you in all you do!
emelks
Also, Steve McIntyre wrote “that CRU had left versions of the confidential station data on their FTP site for many years (see CA, July 2009, A Mole?)”.
I add that it wasn’t just a CRU FTP site (which could have been password protected) but an “anonymous ftp server”, Login: anonymous, no identity verification. Heffernan mentions the phrase “anonymous ftp server” and the same phrase was used in one of the CRU emails I reviewed. It looks like the team was being pretty loose given the seriousness of the proposed confidentiality agreements.
Steve: As I noted last summer, one such version from 1996 ( cruwld2a ) was online for over 10 years and is cited as such in an academic publication (see Google Scholar.)
The key to understanding this is to see that Jones and associates are “First Parties”, and those who they choose to give it to are “Second Parties”. It is only people like Pielke Jr. and Ross and Steve who are “Third Parties” and to whom nothing can be given. I hope this clarifies the meaning…
Legally speaking, a third party is a person or group who is not a party to a legal contract or transaction, but who has an interest or involvement in some way.
Craig please, the lender is always the first party, Jones is the second party. Gotta start the chart from the top.
Beg to differ…
The original source is the party of the first part. 🙂 lol
Does my lack of law training show? I did think about that about five minutes after posting.
I was simply appreciating your sense of humour. 🙂
I’ve added a postscript to this post, citing a Climategate Letter which mentions the Team view on restrictive agreements using characteristic Team rhetoric, a Climategate Letter that arose out of the very first post in the current FOI sequence.
On May 11, 2009, as mentioned in a CA post here, I had noticed that the UK Met Office webpage on CRUTEM stated:
No mention of restrictions due to confidential agreements. I reported that I would inquire about the station data. On May 12, 2009, Peter Thorne of the UK Met Office wrote to Phil Jones ( see Climategate Letter):
The Team has such an elegant rhetorical style.
Thorne was correct that our IJC comment had been rejected, but, at that point, I hadn’t commented on the status of our Santer comment. On May 28, 2009 (CA here), I stated:
It appears that Santer co-author Thorne was a reviewer of our submission, a reviewer who recommended rejection and who then raced to tittle-tattle about the rejection to his confidante, Phil Jones.
I have to be honest. NDA or no NDA, if I was sitting on evidence that the end of all existence was eminent as they claim I would be releasing the data every where I possibly could. The idea that we are going to ask some countries to completely destroy their economies and we are about to unleash a multi-trillion dollar wealth transfer program but we can’t see the data so some one make a couple of hundred or a couple of million dollars off of this data is absurd. The EU is at the forefront of lecturing the entire world on the coming doom and gloom, surely no member country would dare stand in the way of releasing the evidence of this impending parrel. Unless it was of course a load of blarney and they knew it.
Couldn’t agree with you more….this is the bit I just cannot get my head around…”sitting on evidence that the end of all existence was eminent” …you would think that they might let us know..!!!
What H-index value was used to support Jones’ AGU Fellow application?
I note that he was awarded status as an AGU fellow in 2009, but I don’t see any evidence with respect to the H-Index number that was used to support it.
Re: shewonk (Jan 1 23:22), note that while h-indices are normally only used as one argument among many for awarding fellowships and equivalent academic honours, an index around 40 would normally be considered sufficient within the physics community, so the argument as to whether his h-index is 52 or 62 is not particularly important.
But see my comment on the h-index at 1/1, 11:21AM below. Since Jones’ h-index as calculated almost surely counted cites to papers like Santer+16 2008 as a full cite to a full paper, even when they are self-cites by Jones or Santer, instead of 0.059 cites to 0.059 paper, and then only when the cite is not by any of the 17 authors, it was grossly inflated, even aside from the other P Jones.
Re: Hu McCulloch (Jan 2 11:38), true but irrelevant. To first approximation everybody’s h-index is inflated in this way, and the extent on inflation is relatively constant within a field (though it can vary quite sharply between fields).
Incidentally, my current best estimate of Jones’s h-index is 62 and rising fast.
Perhaps even if 62 was wrong at the time, it might be possible that by the time Mann submitted his letter of support and claim of 62, it might have become correct. 😀
Ironic, if so.
The letter is in June of 2008. To prove Mann planned to deceive the committee, they’d have to go back to that data and check what his H-Index value was.
What if Mann ran the numbers himself later and came up with a wholly different number?
The fact is that until we get more evidence, all we have are the emails and they are not sufficient to convict IMO. YMMV.
I know you used a smiley, but this is just a ridiculous post. Mann was told by Jones himself that 62 was wrong. Mann acknowledged that and proceeded to ignore it. Sound about the same as he does with his scientific conclusions. (I know, piling on. But one has to call a spade a garden tool)
I read the first few paragraph and I have to say that my skeptic spidey senses are heightened whenever a purported scientific paper uses words like “unofficial king of climate research” and refers to climate skeptics as “scatterbrained”. It seems to me this is no scientific paper but an opinion piece cloaked in shiny paper to fool the unsuspecting.
Re: Susann (Jan 2 13:41),
There need be no witch hunt. My understanding is this. The accusations have been made publicly regarding the H-Index, also, the accusations against Jones regarding wrongful withholding of data have been made publicly.
Mann can publicly deny his written email and explain what hed did. He should! If he does not, and anyone subsequently uses the information in a complaint against Mann then a judge/arbitrator etc. can (and should) take his words to be true — they are his words after all.
In other words — if you do not defend yourself against such accusations they can reasonably be assumed to be true — in the absence of proof to the contrary.
Don’t take my word — call your favorite lawyer.
I won’t bother with Jones because the arguments are similar — and very public.
True but very relevant. Even within fields, the number of co-authors can vary enormously. Eg within climatology, Santer+16 2008 has 17 co-authors. One of Jones’ probably highly cited CRUTEM background papers, Brohan et al JGR 2006, has 5 co-authors. The MBH papers have only 3, while Loehle (2007) has only 1. It would be enormously misleading, not to mention inequitable, to count these all the same, whether for salary or tenure purposes or for honorifics like AGU or PNAS Fellowship.
Does your count of 62 include self-cites by Jones and co-authors? What is his average number of co-authors? Did you include the other P. Jones as Mann did?
Re: Hu McCulloch (Jan 2 12:46), obviously individual papers have varying numbers of authors: my papers vary from 1 to 11 (I think). However for major authors within a given field the distribution of authorship counts is normally fairly constant. While more complex definitions of the h-index would no doubt have some effect on individual rankings these will normally be fairly mild – I very much doubt that any such change would stop Phil Jones coming out as a highly significant author in his field, which he undoubtedly is – and the original point of h was to have an easily calculated statistic which was not wildly inaccurate, rather than an accurate number.
My 62 count comes by searching on web of science for P D Jones, limiting to papers in physical and interdisciplinary sciences, and selecting for authors at UEA and a few obvious climate science locations. Glancing at the top 20 papers they were all plausibly his. Coauthor numbers varied wildly.
The “AGU Fellows Nomination Guidelines” here do not mention the “H index”. The AGU website lists Dr. Jones as a Fellow here, but the nomination package submitted by Dr. Mann is not available on-line. Susann is correct to the extent that the statement by Dr. Mann in the June 2008 e-mail cannot be independently corroborated at this time. For all we know, Dr. Mann may have made a typographical error when he wrote, “I’ll just go w/ the H=62”. I have e-mailed the AGU Fellows Committee requesting clarification of the circumstances regarding the nomination of Dr. Jones and will post any response I receive here, but I don’t expect to hear back from them until Monday at the earliest.
Re: shewonk (Jan 1 23:22), Whatever the H-index, it is amazing to me that the AGU would accept a number value without asking for copies of the actual published papers. That is the same as Nature and Science not demanding the data and methodology, IMHO.
This is membership in an elite science group, and they wouldn’t require the actual documents??? IF this is the way they work it, color me underwhelmed at their process.
Also, I completely agree with Jeff Albert at 1:18 on Jan 2: It is the INTENT of Mann that is despicable. And if the AGU can be so easily scammed/gamed, shame on them. It doesn’t matter whether Jones was above 40 or if he now is at 62. What was the intent of Mann? Nothing else matters.
Well the Hi- index worked, whether it was the ‘tipping point’ we dont know.
List of AGU Fellows for 2009 – the 0.1% of members
http://www.agu.org/about/honors/fellows/fellows2009.shtml
The dog ate our agreements.
The name Mr Fraudit puzzled me.
Having searched the emails it appears to have been first used by Thomas C. Peterson of NOAA, about Climate Audit. So the Hockey Team can stop complaining about others calling them fraudsters since they started it.
email 1249045162.txt
From:
Date: Wednesday, July 29, 2009 12:07 pm
Subject: Re: This and that
Hi, Phil,
Yes, Friday-Saturday I noticed that ClimateFraudit had renewed their interest in you.
Steve: Thorne’s use of the term was in May 2009. The term was previously used at a well-known blog operated by NASA employee Gavin Schmidt.
Steve, has Gavin (or anyone else) ever actually explained to you what behaviour they consider to be fraudulent?
It’s a pretty serious accusation, and to make it without any evidence is extremely childish (and in some countries might even be considered libelous).
My impression is that you are above board. However I am curious what fraud they are specifically accusing you of.
Steve: I have no idea what their accusation is.
Readers: if you comment much on CA, please try the major new update to the CA Assistant. It greatly enhances the user experience:
– Preview
– Reply links
– Old comment hiding and new comment color controls
– Comment sorting and thread control
Feedback needed on usability, etc.
Re: MrPete (Jan 2 01:51),
After reading the favorable comments on the enhanced version of CA via Firefox/Greasemonkey, I installed the pieces. Unfortunately my extensive collection of bookmarks did not copy from IE to Firefox. Is there some way that this can be accomplished? Also some mention was made of obtaining previews of comments. As I am typing this, I see nothing to “toggle”.
Re: Mike Rankin (Jan 2 11:49),
You should have a box on the top right which says “preview”. When you click it another box saying “hide” will appear. Clicking that will get you back to the original raw text.
Re: MrPete (Jan 2 01:51),
After poking around, I discovered the menu item of import / export. Was able to get the IE bookmarks. Still have not found the preview “button”.
Re: MrPete (Jan 2 01:51),
ORNL = “Oak Ridge National Laboratory”, not “Oak Ridges Nuclear Lab”.
http://cdiac.ornl.gov/
ClimateGate – The Prequel
I am trying to determine the validity of the famous quote Jones is said to have made to Warwick Hughes back in 2005, when refusing to disclose data supporting his thesis:
“We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it.”
So far:
* Steve mentioned it here
* Pat Michaels quotes it in the opening of his book Climate of Extremes
* Hughes has put the contents up here
but no longer has the original email with headers etc
* Hans von Storch has told me that Jones admitted it to him, and that this was reported to the NAS Panel, but also does not still have the email.
I have also asked Ed Wegman, who says he has no such correspondence.
And apparently is it not in the hack. However I have been told he makes another similar comment that IS in the hack – complaining this time that Steve only wants data for that reason – but I have not found it. Can anyone here point me to this?
More generally, does anyone else perhaps have or know of emails, or other evidence?
Thanks.
Steve: Warwick Hughes sent me a copy of the original emails. Von Storch stated at the NAS meeting that he confirmed the quote with Jones and included this statement in his NAS PPT (which doesn’t seem to be online anymore.)
Can’t help with the actual email but poster 10 on http://www.warwickhughes.com/blog/&p=308 specifically refers to it being sent from Jones to Hughes on 21/2/05 and wonders if the following sent on the same day is linked to the Warwick Hughes request, prompting Jones Post Script.
February 21, 2005: email 1109021312
Phil Jones writes to Mike Mann, Ray Badley, and Malcolm Hughes, regarding news reports that Mann will be forced to release his data:
The skeptics seem to be building up a head of steam here!
…
Leave it to you to delete as appropriate!
…
PS I’m getting hassled by a couple of people to release the Climatic Research Unit … temperature data. Don’t any of you three tell anybody that the United Kingdom has a Freedom of Information Act!
Peterk. Not exactly sure what you’re looking for but these two may help. Suggest
http://www.assassinationscience.com/climategate/
August 6, 2004: email 1091798809
Phil Jones replies to a favorable comment by Australian climate scientist, Janice Lough:
Janice,
Most of the data for most of the graphs have just appeared on the Climatic Research Unit web site. Go to “data”, then to “paleoclimate”. We did this to stop getting hassled by the skeptics for the data sets. Mike Mann refuses to talk to these people and I can understand why. They are just trying to find if we’ve done anything wrong.
Feb 2 2005 email 1107454306 Jones to Mann
“I think I’ll delete the file rather than send it to anyone” !
In the whistle blower email files this quote shows up in two emails with the following file names:
1188412866.txt
1256765544.txt
BTW, I beieve it has been pointed out prior that these numbers are Unix time counters that indicate time in seconds since the beginning of the Unix epoch which is 00:00:00 UTC on 1 January 1970, and which I recognized immediately having worked with them nearly that long. By itself that has no significance as they can be assigned ad hoc and as desired to file names. But in my exploration of the emails the file name matches the first “Date: ” header line in the message contained in the file. That is the delivery date to the end point, or what you all know as the recipient. Many messages also include quotes of earlier messages with earlier date headers. Haven’t yet found one that has later dates which would indicate manipulation.
I have not found the phrase in any of the FOI documents that accompany the emails.
In both cases in the identified files the quoted text (“… find something wrong with it”…) was just that – not a smoking gun, that is to say, it is not a first person statement, but a second-hand reference to a possible smoking gun. Not enough proof for me die on this particular hill, yet still believable given the larger context.
For what it’s worth, here are the headers and I’ve munged the email addresses because I’m a messaging professional and it is a professional courtesy:
1188412866.txt
From: Kevin Trenberth
To: Phil Jones
Subject: Re: Something not to pass on
Date: Wed, 29 Aug 2007 14:41:06 -0600
Cc: “Michael E. Mann”
1256765544.txt
From: “Graham F Haughton”
To: “Phil Jones”
Subject: RE: Dr Sonja BOEHMER-CHRISTIANSEN
Date: Wed, 28 Oct 2009 17:32:24 -0000
—– End quote
Something that needs to be known, if it is not, is that these headers are subsets of the full header sets that are created by the mail transport agent (mail serve software). They are also incredibly easy to fake. The really valuable headers are the machine to machine headers as those contain the IP address of the system pairs exchanging email. They are also easy to fake but also easy to examine for validity.
I apologize for this dry content, but as it is what pays my bills I consider it very interesting. Delete it if it contains no contribution.
Another take on the Team providing answers to requests for info. Note the Freudian “global warming proponents” slip!
Acknowledgment to John Costella at http://www.assassinationscience.com/climategate/
February 2, 2004: email 1075750656
Keith Briffa makes an astounding comment to Rashit Hantemirov, regarding a request made of Hantemirov:
Dear Rashit
Thanks for this—these people ask many questions as they try constantly to attack the global warming proponents. I answer sometimes, but it usually means they come back with many more questions. All part of science, I suppose.
John Costella comments:
Indeed!
It is remarkable, firstly, that Briffa describes himself and his colleagues as “global warming proponents”, rather than “researchers”, “investigators”, or even just “scientists”. Surely they are not meant to be “proponents” of a predetermined view? A Freudian slip on Briffa’s part, perhaps?
Secondly, Briffa’s bewilderment that anyone would question them over their work—and that an answer would not simply provide a brush-off, but may well stimulate follow-up questions—is astounding.
Having only read the thread, but no comments, a few points:
1.Confidentiality agreements almost always have a clause allowing information to be passed on to 3rd parties – so long there’s previous express written permission to do so. It is entirely possible that Jones did indeed get such permission in each case.
2. If he had not gotten express written permission, and did indeed wrongly forward the information, then the originator of the information would have had, or would have today, a legitimate claim against the CRU. Are there any court records showing such a legal claim or case against the CRU? To my knowledge, there is not.
I don’t see anything wrong by Jones, so far.
3. Yet, all signs indicate that the claim of existing confidentiality agreements is bogus. If such agreements truly existed, then why would Jones make such a fuss over the FOIA. He could have simply rejected requests by citing confidentiality agreements.
4. So far not a trace of evidence has been presented showing the existence of confidentiality agreements. The pattern descríbed by Steve above seems to confirm non-existence.
Re:work that Steve is doing…recent article http://www.theaustralian.com.au/news/opinion/boffins-may-be-illegal/story-e6frg6zo-1225815349833
he states…”Let us ignore the fact that even a prestigious journal such as Nature is happy to label scientific scepticism as the work of “denialists”, which is good evidence that the CRU disease has spread far and wide into the general science community. Prior to Climategate, there were probably fewer than a dozen FOI demands to CRU. There would have been no need even for those if the information had been made available when it was first requested.”
Further posted at WUWT Tips
Options appear to be:
1. Send them the data
2. Send them a subset removing station data from some of the countries who made us pay in the normals papers of Hulme et al. (1990s) and also any number that David can remember. This should also omit some other countries like (Australia, NZ, Canada, Antarctica). Also could extract some of the sources that Anders added in (31-38 source codes in J&M 2003). Also should remove many of the early stations that we coded up in the 1980s.
3. Send them the raw data as is, by reconstructing it from GHCN. How could this be done? Replace all stations where the WMO ID agrees with what is in GHCN. This would be the raw data, but it would annoy them.
This is from file jones-foiathoughts.doc. Is this relevant to the released data?
I noticed that too. In the list of CRU documents here the date on that file is April 23rd, 2007. The paper referred to is:
Hulme, M., 1994: The cost of climate data: A European experience. Weather 49, 168-176.
…but I haven’t found it yet. Does anyone know where I can get a PDF of this paper?
Is the purpose of the confidentiality agreements to insure that the data sources have an opportunity to discuss different uses and receive compensation? i.e. future use not related to the initial agreed purpose requires negotiating another agreement? If so, this is more an IPR discussion than confidentiality.
Meaning both science and the data owners would have been better served with a full disclosure of meta-data referencing datasets available for purchase elsewhere, and the edits, adjustments, normalization, etc, that were applied to same fully documented and published. Then others would have likely (happily) paid the data owners for their services and been able to proceed independently. A pretty clear win-win for all involved. And damning of those who would deny those (likely poor and struggling countries’ climate researchers) their appropriate compensation.
From the AGU website statement of “Mission, Values, and Goals”
The organization provides a list of “widely shared values”, among them:
•Free exchange of ideas and information. (“Don’t any of you three tell anybody that the United Kingdon has a Freedom of Information Act!”)
(“We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it.)
Diversity of backgrounds, scientific ideas, and approaches (Jones resentment/attempted refusal to release information because McKintyre is not an “academic”)
•Accountability to the public. ( You will have to make your own
judgments about that one… how about “Hide the decline”?)
•Unselfish cooperation in research (enough said).
The liklihood of an honest accounting of the Mann/Jones et.al. debacle is made vanishingly small by virtue of the widespread professional,
financial, political and institutional vested interests on an international scale that have become involved here. Can anyone really see East Anglia, Penn State,IPCC, COP(15-16-17), NASA, and NOAA
actually “fessing up” and admitting that some of the basic data are corrupted? Far too many reputations and agendas would be destroyed.
So JV’s observations above strike home with me. The only way “they” can really “win” is if the “skeptics” concede the day and just go away. I can’t quite see Steve or many of the rest of us doing that.
I have nothing substantive to add to this thread, but I do attempt to put all these efforts of Steve M into perspective – at least for my own edification. I judge what Steve M shows here, by way of evidence and attempts to string the events together in some reasonable form, in effect, presents the prosecutions side of a case that the institutions now doing “independent” investigations might be tempted to overlook.
My contemplation of these developments leaves me wondering about:
The attitude of some climate scientists, that I see exemplified by Jones, seems so untypical of a scientist. As an example, we have Jones’ responses of lost data, not remembering with whom he had confidential agreements and unclear to whom he gave confidential data and under what terms.
The institutions doing the investigations may have more at stake here then an individual scientist would and that may preclude any tendencies that they might have to whitewash the case – and as stated above I wonder how much Steve M’s detailed postings of his accounting of what has transpired will “help” the investigators provide some real answers to what gives with these data.
The GISS analysis of the surface temperature gives essentially the same result as the CRU analysis. Since all of the GISS data and code is available, and has been for many years, does that mean everybody agrees that the conclusions are robust? I don’t necessarily agree with the choices Phil Jones made but where is the science in any of this?
CityTrader, it seems to me that some people here are not as interested in the science per se as the scientists.
It appears to me that they are busy focusing on the scientists perhaps expecting that if/when they fall, the data will fall as well — by association.
Is there anything in the so-called climategate messages and documents that concerns you? I’ve seen you brush off a lot of other poster’s comments in regards to same but I am left wondering about what the unexpected look at the “team” means to you, and not so much what you think of other persons’s view of it. I’m concerned about some policy and process issues and not so concerned about the science impact, but ultimately they are interlinked in a much large way than is obvious if viewed casually. Specifically, the behavior of the “team” has caused lost opportunity to those of us that remain unconvinced, and undeveloped areas of interest that may even suggest alternate solutions to the conflicting data. The scale of this could even involve funding resources being denied on false premise. That is not insignificant.
Sorry if this is out of sequence with your blog threading but that’s the hazard of blogging software.
I think the emails suggest bad behavior on the part of the scientists involved — talking about trying to get around FOI requests, denying access to data and methods etc. — those are just requirements of the job and fundamental to good science.
This behavior occurred in a particular context — a highly-politicized environment and the emails suggest that some of the basic principles of the scientific method were being circumvented in part because of that environment, especially in the case of data sharing, peer review and maintaining a healthy degree of objectivity towards the research – not overstepping the bounds of the work in order to support a public policy agenda.
I don’t excuse this behavior on the basis that the environment has been compromised, but I do understand it through that context. It does look bad, and that appearance will be used to cast doubt on the underlying science — a fact that is clear from any review of the blogosphere. It’s like a bonanza for those who want to deny AGW but don’t want to have to take the trouble of providing actual research.
Susann, Jan 4, 2010 at 1:49 AM
You put the cart before the horse – the bad behaviour is the reason the environment is compromised in the first place.
As regards the “bonanza”, that’s rich, given that virtually all climatology money is tied to the ‘consensus’ view – coming from govenments, who have an obvious and massive vested interest in convincing everyone that AGW is serious.
And the alleged bonanza could be eliminated in an instant if the wrongdoers were being fired from their jobs. That they are not tells is the ‘consensus’ endorses and depends on these practices.
STILL awaiting moderation since Jan 4 ?
Susann (Your comment is once again OT for the thread but I will respond anyway)
This is just part of the “devil made me do it” defence, but it should be looked at in the proper context. The “politicized environment” was self-inflicted, and not forced upon them. They chose to enter that environment and in so doing morphed themselves from scientists into propagandists for the cause with the subsequent loss of objectivity and the ever increasing exaggeration of the results of their work.
It is at that point that many in the blogosphere recognized that this overt exaggeration was becoming rife in the AGW community – since confirmed by the content of the emails – and began actively questioning it. The response to requests for details on which their pronouncements were based was to become secretive with a refusal to release relevant information and actively abusive and obstructive to the work of anyone anyone not blindly accepting their settled science (again evident from the released email communications).
There are two aspects to the “science”, the research itself and the dissemination of the results to the public sector. First, the science has become tainted by the loss of objectivity resulting from their activist position. The scientific research is not cut-and-dried and leaves room for manipulation. It depends heavily on choices made by the researcher, which data to use and which to reject, the methods used to analyze that data and how to summarize and present the results minimizing the components which may create uncertainty on their validity. The purpose of the audits has been to determine the extent of any such manipulation (along with looking for mathematical and statistical errors which are more common in this area than you might think) and the emails underline the need for this process to continue.
The second aspect is the presentation of the research to the public sphere. Here, there has been an active propaganda campaign for the purpose of driving a particular policy agenda to which the team were regular contributors. Such a campaign results in the intentional distortion of results (e.g. Mike’s trick, “significant” warming in the Antarctic, the sea level rise if all the ice in Greenland or Antarctica melted, etc.) which raise the level of alarm in the less informed portion of the public. Rather than have such alarmism go unchallenged, it is necessary to separate fact from fiction to provide a genuinely balanced and realistic view.
So, yes, this is a bonanza, but not for as you put it, to “deny AGW”, but to separate the misinformation from the actual facts of the situation. Nor do I think that your straw man of the need for those who question their results “providing actual research” is appropriate. If you prevent the manipulators from inordinately controlling the climate science process, you might find that there are enough others in the field who can do a more honest assessment of AGW.
OT
Once a scientist accepts money from the petroleum industry to speak against AGW, they have passed from skeptic and into the contrarian/denialist category in my view.
Steve: As with 99% of Susann’s comments, this is OT. Please do not debate this point on this thread.
Seven posts on people’s ancestry, that as far as I can tell have no bearing on any issue raised in your post, are allowed but a few posts that are truly relevant to issues of ethics, which was raised in your post re: Mann and the 62 H-Index number — are not?
” … re: Mann and the 62 H-Index …”
We’re waiting for the unexpurgated result of some “official” investigation here, although we have no idea which “official” investigation we are waiting for
You may remember what happened to that question earlier …
They both use the GHCN station data. Chiefio has shown that the stations have moved toward the equator and dropped in altitude. The results of that should be obvious to anyone.
Steve: It’s not as obvious you may think – as the averages are done on an “anomaly” basis.
This is OT but where is the appropriate place to discuss the land temperature record? Unthreaded? I’ve been looking at USHCN/GHCN station data and adjustments and there are many curious things.
If you are discussing information from countries outside the USA and GB, you should establish the provenance and possibly prior “adjustment” of the data supplied to CRU and eventually to GISS. The match between the latter two can be only as good as the data the collecting country supplies. Then you examine the adjustments (if any) made by CRU and Giss.
It is not scientific to simply look at two data sets, imagine that they look essentially similar, then conclude that they are the same.
The “H-index”, as defined by physicist AE Hirsch in PNAS 11/15/05 (http://www.pnas.org/content/102/46/16569.full.pdf+html) is an interesting index of scientific impact: As Hirsch defines it, a scientist has index H if H or his or her papers has at least H citations, while the others each have less than H citations.
However, a big flaw in this is that as defined, it counts co-authored papers the same as individually authored papers, and self-cites the same as cites by others.
Santer+16 (IJC 2008), for example, is padded with no less than 17 co-authors, including one P.D. Jones. In order for the H-index to be meaningful, this should count as 1/17 = 0.059 paper for each of the co-authors, and any non-self citation to it should likewise count as 0.059 cite for each author.
Hirsch concedes towards the end of his note that ideally citations should be deflated in the case of “subfields with typically large collaborations (eg high-energy experiment),” and that “one would like to eliminate the self-citations.” However, he does not include either of these qualifications in his definition.
In fact these obvious corrections are not so hard to implement. My department, for example, (OSU Economics) every year compiles a spread sheet of ISI cites for each tenure-track faculty member by article, eliminating self-cites by that faculty member. It would be trivial to add a column with the number of co-authors on each article, and then to deflate the cites before adding them up.
In economics, 3 or 4 is a high number of co-authors, while in other social sciences like psychology, a dozen is not unusual. Equity both within departments and between departments (since college tenure committees must compare candidates from different fields) demands that cites be deflated.
In eliminating self-cites, a citation by any of the co-authors should count as a self-cite for each of the co-authors. Thus, if Santer cites Santer+16, that should count as a self-cite for Jones, just as much as Jones himself cited that paper, since it is not an objective cite by a disinterested scholar. (My dept. doesn’t bother with this adjustment, but I have argued that it should.)
East Anglia CRU is one of several top research stations. The withheld data has not been shown to dent the overall picture of increasing temperatures and impacts (such as melting ice). Even if it did, the effect on the other two hundred lines of evidence from other data sources and research stations could only be miniscule.
Scientists tell me that the data streaming in has become scarier every year. In the real world, the hockey stick is intact. So let’s move on to more significant issues, such as how to cope with the unfolding tragedy that is human caused global warming.
“The withheld data has not been shown to dent the overall picture of increasing temperatures…”
If it was withheld who did the checking to see if it did any denting?
1) how do you know the warming is human-caused?
2) the GISS climate history uses similar methods to the CRU for adjusting station histories
3) While GISS released their code, no one has to my knowledge been able to get it to run
4) There is evidence that none of the climate histories factors out the urban heat island effect, meaning that the warming trend may be less than claimed. If the true warming is not 0.8 deg C but perhaps 0.4 deg C per 100 yrs, would people be as alarmed?
Steve: The main issue is not so much getting it to run, but the handling of UHI in these indices. There are important differences between CRU and GISS in different regions e.g. a large difference in the USA trends in the 20th century. There are interesting questions – some of the questions are relevant to the allocation of temperature increase between the first and second halfs – which is relvant for attribution.
Stonewalling is no longer a viable strategy for the simple reason that various official investigations (e.g. via U.S. Congressional committees) will in effect supercede confidentiality agreements, such agreements constructively waived for purposes of completing appropriate investigations, and the same with court authorized trial discovery for various tort actions – which will soon be common in light of the recent EPA endangerment finding.
Steve – Is it the case that you had the Hantemirov raw data all along – since 2004 ?
Steve: Please see discussion in October 2008 on this. As the Climategate Letters show, there are sometimes multiple versions of data from a site. Hantemirov used a version for his reconstruction which yielded different results than the Briffa reconstruction. RCS methodology requires a much larger dataset than the Hantemirov data set, Rob Wilson had told me that Briffa’s Yamal was more highly replicated than Polar Urals – so I presumed that Briffa had used another version. In 2003, Mann claimed that we had used the “wrong” version of MBH data. (Of course, we used the version that Rutherford had directed us to – which was deleted after our 2003 publication and the other set of books were produced.) So I asked Briffa for the version that HE used. This was not provided until Sep 2009 and it then proved to be virtually identical to the version that Hantemirov had sent me a few years earlier. This was very unexpected. It has nothing to do with Briffa’s refusal to send me the data that HE had used.
The discussion was in October 2009.
Does Rob Wilson still hold to this view about Yamal?
Wouldn’t this just be much easier if they went ahead and posted the darned code, data, what have you, for all to see rather than go through this stupidity? I still find it unfathomable that temperature data and the code by which it is massaged is considered sacrosanct.
I notice commenters eg Dave L. Posted Jan 1, 2010 at 7:21 PM
opining that [Also, if DOE contracts were involved, then DOE as part of the contract, would require detailed annual/quarterly reports including data sets and software programs be submitted to the DOE contracting office. Actually I would suspect that DOE would have some form of joint ownership of the data and analyses that formed the backbone of the research contracts. DOE doesn’t just write blank checks for research grants;]
Well, what DoE told me in late 2005 suggests that in this case they did, “..write blank checks for research grants..”
See my article;
What a shock it must have been for Phil Jones to go from acting like the Red Queen to being kicked to the curb as an embarrassment.
City Trader when you say:
“The GISS analysis of the surface temperature gives essentially the same result as the CRU analysis. Since all of the GISS data and code is available, and has been for many years, does that mean everybody agrees that the conclusions are robust? I don’t necessarily agree with the choices Phil Jones made but where is the science in any of this?”
I think we can summarize the situations, at least as I judge some here view it as the following:
1. Robust would indicate that the temperature data can be verified independently, and while that can be the case since satellite data became available in 1979, we have the problem that going back in time past 1979 the data sets used are not necessarily independent. It can also be shown that the two sources (actually there was a third that seems to have fallen out use or citing) of satellite data do not agree completely.
2. If we assume that the raw data is mainly common to all three of the major non-satellite and non-radio sonde sources of temperature data, then we can compare these sources and determine whether there are statistically significant differences between the sources over time segments and regions of the globe.
3. The problem has been that the owners/keepers of these data sets have been hestitant or not forthcoming in revealing exactly what data was used and how it was adjusted.
4. Therefore when we make analysis we have to make assumptions about the data and as Steve M has and is related these stories that can be frustrating and inefficient.
I think you really have to get your hands a little dirty digging into these data sets before you can understand the subtle, and not so subtle, issues involved. And, no, you do not have to be someone who wants to proclaim no warming over recent times to question the raw temperature data, its adjustments and the uncertainties that have been attached.
I certainly would not expect someone like Susann, the policy lass, to understand or appreciate all of the complications and complexities involved.
In 2 above, I wanted to point to testing the significance of the difference in adjustments, if we assume the raw data are the same or nearly the same.
Ken, I’ve renamed my blog in honor of your reference to me as “the policy lass”, ignorant of such complicated complex issues as handling data as I may be. 😀
It seems to me that a considerable amount of time lately, but especially in the wake of the CRU event, is being spent on the scientists, their ethics and motives, rather than on the data or the other lines of evidence. It’s all very National Enquirer in feel, and while I do understand that attraction of speculating on Mann’s ethics, it is not really relevant to the data.
Any given scientist can be an absolute jerk but his or her work can still be valid.
And because someone asked at my blog, the difference in names between Susann and Shewonk was the result of posting at times when I was logged in at WordPress and other times when I was not logged in. I will try to be consistent in the future. Apologies for the confusion.
Susan:”It seems to me that a considerable amount of time lately, but especially in the wake of the CRU event, is being spent on the scientists, their ethics and motives, rather than on the data or the other lines of evidence. It’s all very National Enquirer in feel, and while I do understand that attraction of speculating on Mann’s ethics, it is not really relevant to the data.”
—————-
Firstly, Climate Audit in particular has, over the years, concentrated on the science probably more than any other blog. Steve has tried to discourage discussion of motives and possible wrong-doing in the absence of evidence. Since the release of CRU emails there is ample evidence of motives, unscientific-behaviour, wrong-doing and apparent illegality, much of it directly related to Steve and the material he has been dissecting for years. It would be perverse not to reflect that.
Secondly, it is the scientists concerned who have made their own integrity such an issue. If they had been open about their data and methods from the start, making their work replicable, then their personal integrity would be immaterial. The work would stand or fall on its own merits. By conspiring to hide their working methods they are in effect saying “trust us”.
They cannot then complain if their honesty, behaviour and motives become a matter for debate.
Finally, and leastly, any sympathy I may have once had for any alarmist in this situation dissipated long ago with their appalling behaviour towards “denialists” or anyone who merely dared to question them.
Susann has a post on her blog
here where one may debate her on this subject.
I wouldn’t bother now – posts are edited (censored, moved from one context to the other) as cutely as RC does it
Now that your error about deleting or moving posts has been pointed out over at my blog, why don’t you clarify that in fact no posts were moved or deleted or is that beneath you?
Re: Susann (Jan 3 20:30),
Cross-posting not allowed. Afraid this will be snipped here.
I doubt even they know what data was used and how it was adjusted. According to CRU:
The CRU website is back up and they have a “Data Available from CRU” page here. The CRUTEM3, CRUTEM3v, HadCRUT3, HadCRUT3v, HadSST2 and absolute temperatures for the base period 1961-90 are all available for download but, as has been pointed out many times, this is only the “value-added” data. If I read the above statement correctly, they are saying the original raw data is lost and their method for deriving these gridded data sets is undocumented.
It strikes me, first, that releasing data is not the same as releasing information about which stations were used, which would explicitly NOT be covered under such agreements. So their alibi doesn’t hold water for at least a subset of your requests.
Second, and possibly more importantly, it would seem inappropriate for public policy to be based on private data and analysis. I know this argument has been made in a general way by many people, but this specific instance of “protection of sources” seems to indicate the need for a frontal assault on this point. Data that cannot be subject to public scrutiny is inappropriate for use in such broad public policy measures as are being proposed by the warmists.
I think the argument can be pushed that the IPCC, which is supposed to represent worldwide public interests, should not base its assessments on any data that cannot be publicly verified. In particular, this would eliminate most of the “gridded baseline climatology” of the CRU. What would the IPCC assessments look like without ANY private data or analysis?
I think there is some reason to reevaluate the peer-review system, but it should happen at the editorial and referreeing level: Analysis based on data that is not publicly accessible, or using computational techniques that are not fully disclosed, should not be admissible to the peer-reviewed literature. Such an approach is clearly the most sensible, and would have avoided the mess that has been made of climate science today.
More appropriate for your posting here might be:
Susann, Susann, the policy lass,
who claims to be sorting the climate morass.
But Susann, you have not always seemed to be in a big hurry to discuss the science here and instead you do what you accuse those here of doing to the climate scientists, you appear to be passing judgment on the behavior of posters here – and in much more general terms than those here do for the climate scientists.
I too yearn for those days in 2010 when we will be once again analyzing climate science papers and methods. Will you be joining me?
Cant you just see me frolicking through the meadow, big sunhat on, swirling my skirts? 😀
I’m not a natural scientist. I’m here to read what’s said about the science, yes. But I cross check what is said here over at RC and other AGW blogs.
Perhaps you should think of me as possibly doing a participant observation study or content analysis of CA to see what I can discover about it as a phenomenon. There’s a lot of really interesting material here — is this the start of a new social movement? Non-professional auditing of science by interested laypeople and techies? It’s potentially very interesting research to a social scientist.
Steve:
People dislike the MSM leaving us with a lie. Please don’t join them in that behavior.
Susann said:
“Once a scientist accepts money from the petroleum industry to speak against AGW, they have passed from skeptic and into the contrarian/denialist category in my view.”
100% pure trollism. OT, yes, but please include the definitive rebuttal and then snip any further OT on this subject:
from “Andrew30
I did not notice the Associated Press as being one of the funders of the CRU. Perhaps the CRU get enough from British Petroleum and Shell Oil and other companies interested in demonizing and then taxing CO2 to make the Nuclear, Liquefied Natural Gas and Food to Ethanol businesses more cost competitive. From the CRU web site, look at the bottom of the page. http://www.cru.uea.ac.uk/cru/about/history/ From the Climate Research Units own web site you will find a partial list of companies that fund the CRU. It includes: British Petroleum, “Oil, LNG” Broom’s Barn Sugar Beet Research Centre, “Food to Ethanol” The United States Department of Energy, “Nuclear” Irish Electricity Supply Board. “LNG, Nuclear” UK Nirex Ltd. “Nuclear” Sultanate of Oman, “LNG” Shell Oil, “Oil, LNG” Nuclear Installations Inspectorate, “Nuclear” Tate and Lyle. “Food to Ethanol” KFA Germany, “Nuclear” And the climate scientologists have the audacity to claim that the skeptics get funding from ‘Big Oil’!”
http://www.washingtontimes.com/news/2009/dec/28/biased-reporting-on-climategate/comments/
Susann, used as you are to writing for the audience at RC, please recognize the different intellectual level at CA, and don’t [aaargh] accompany your simple puns with an explanation.
TheLeavener
Not a lie, an opinion. You’ll note I wrote “in my view”.
But it’s a view supported by research. There are many studies on the effects of industry funding on research findings. My area of study was the pharmaceutical industry so I know it best. A simple google scholar search yields many studies that show this.
Re: Susann (Jan 18 19:25),
So in your opinion, does industry money only bias those who are skeptical, while providing a solid basis for the support of the topic? How about government funding? Obviously, politicians have a conflict of interest in that handing out grants in support of popular views earns them more votes.
Once you start down this path, you’re going to end up with vanishingly few honest sources.
Perhaps only Climate Audit 🙂
This is sophomore level social science that you take in your first research methods course.
Do you think there are totally bias free researchers? Of course not, whether they are funded by government granting bodies or industry.
It’s not the scientists but the process that is designed to reduce bias. Bias is a natural state for humans. That’s why the scientific method is so revolutionary over every other form of producing knowledge. The checks and balances of the method, the peer review process, replication are designed to minimize bias arising from individual and social forces, and for the most part, they succeed, but they are never entirely erased.
In the normal course of things, the social/economic /political context doesn’t amount to much pressure to override the checks and balances science has in place. When there are significant economic or political or ideological interests involved, those checks and balances can be diminished such that the research produced and its conclusions are biased.
Do I think CA is bias free source? Not for a moment.
‘Once a scientist accepts money from the petroleum industry…’
… he should be assumed to be biased towards favouring the petroleum industry’s interests.
Similarly, ‘Once a scientist accepts money from the state’
… he should be assumed to be biased towards favouring the state’s interests.
The ‘consensus’ is thus likely to be a reflection of the relative amounts of money the state vs the petroleum industry puts into climatology. Several orders of magnitude, would be my guess.
And what are the state’s interests in global warming? Please do tell…
Just the usual – raising taxes, adding more bureaucracies and social controls, generally expanding itself.
Those are great talking points for good sound bites but show a lot of naiveté regarding the power of both constituencies over — well — the p-word.
Re: Susann (Jan 18 19:14),
Interesting area. Here are two experienced scientists commenting on the way they believe incentives can go wrong with state funding. First, William Lapper interviewed in The Daily Princetonian a year ago:
Second, Matt Davidson asked the panellists this question at The Great Climategate Debate at MIT last month:
Richard Lindzen replied:
He smiled ruefully as he said this last sentence. But we’re not dealing with an abstract problem in the case of climate science. The very sharp increase in funding from the US state during the four years of Bush Senior is a crucial factor to consider, a sea-change that Professor Lindzen can well remember, having been elected to the National Academy of Sciences as a budding atmospheric physicist back in 1977.
Hope that helps.
Apologies to Professor Happer (not Lapper) there!
It seems hypocritical that Nature, which states in its policy, “An inherent principle of publication is that others should be able to replicate and build upon the authors’ published claims”, could defend with such vigor those who went to extreme measures to prevent disclosure. See http://socratesparadox.com/?p=53#more-53
Hello Mr McIntyre, this is slightly of topic but you may want to look at this since you have asked me to tell you if I come across an exposition in more or less engineering terms.
http://www.landshape.org/dokuwiki/doku.php?id=introduction#the_cabauw_measurements
Steve: This has nothing to do with an engineering-quality exposition of mainstream results. An “engineering-quality” study is a different thing than a study that you happen to like.
snip – nothing to do with my request
The data from the Spanish Meteorological Agency are by no way confidential. If you need data from the past that have been collected by the agency the procedure is easy. Everybody can request them, pay for them, and that’s it. Nothing else is needed. The CRU might argue about re-sending the data to someone else but afaik a) the data are free for researchers or research institutions b) a permit or authorization can be obtained in matter of hours if not minutes just by e-mail if the request comes from colleagues(as Steve emphasizes in his text).
In fact the official form to request the data from Spain is here
http://www.aemet.es/es/servicios/tasas/sscc
best
Re: rafa (Jan 3 18:30),
As to the Spanish data…
There is a confidentiality agreement. You can check it at the link Rafa provides. I was a little confused by the English version, so I read the Spanish version. I read it from the context of another Spanish speaking culture — and sometimes that can be important — but I think I understand it.
You cannot transfer the data rights to a third party (yield/ceder) or publish (difusion) the data without their permission. You cannot transmit (copy) the data to a third person/party without explicit permission — unless it is in the original request.
In the obligations (point two) you agree not to use the data make any announcements/publications that create embarrassment to the providers. (Damage their public image is a more direct translation.)
You must get permission for use in any and every publication, and provide a reference to the information source. It seems to apply to raw data and derivative product equally — although not precisely stated.
It is reasonable in most regards — but the data is clearly considered confidential and private — between the two contracting parties.
Also worthy of note — The agreement to provide data can be canceled for any reason at the discretion of the provider — or if agreement with another State (government department — competent authority) precludes supply to you of the same data. In other words — they can cancel if a later deal they like better precludes your use.
The agreement in my opinion is designed to help them defray the cost of collecting the data. I don’t see it as onerous.
The non-embarrassment clause would seem to exclude Steve McIntyre, but not Scott Rutherford. As far as copying is concerned, we don’t know yet what the “original request” was. Or do we?
Re: Norbert (Jan 4 04:08),
Norbert:
I hope that was humour. Read the agreements link above. It does not include the confidentiality agreement though.
Rafa:
I think we agree. The data is confidential — but not secret!
You are right — getting agreement to pass it on could/should be quick and painless — however the processed data sets provided by CRU have many data sets.
I think that they would have been wise (in retrospect) to include a clause that allowed them to provide data sets — original or derivative — to those who wished to check their work.
Since this is a journal requirement it seems it only makes sense.
Where research could impact public policy it seems wise to stipulate that requirement up front! Perhaps people have learned form this experience!
“I don’t see it as onerous”, exactly!. The CRU attitude pretending that the agreement with Spain is a kind of unsurmountable legal barrier sounds like a poor excuse. The agreement is designed to avoid for-profit organizations using the public data, no more than that. I’m pretty sure they receive quite often requests from universities, research teams, etc, and the files with the data are sent immediately without further bureaucracy. Imho the CRU claim is just a distraction, a smoke screen. If the CRU really wanted that Steve or anyone else had access to the files with Spain’s data I think obtaining the permit from Spain would be a simple task and a very easy procedure for the CRU.They not only “hide the decline”.They pretend to hide themselves behind legal excuses.
best
OT: Over at Heresy Corner is a nugget of information of which I wasn’t aware – maybe everyone else was – but it seems astonishing to me:
[The Tony mentioned was someone who claimed to work for the Met Office, commenting on a Daily Mail post.]
“But you’ll have noticed the really amazing sentence in Tony’s defence of the Met Office. “We take the highest 15 readings between November and March and then produce an average.” They ignore, in other words, the lowest readings. This is a method almost certain to produce massive distortion. A warm November would fix a winter as “mild” even if the temperature in the subsequent three months averaged sixty degrees below zero. Which is almost, but not quite, what is likely to happen this winter.”
http://heresycorner.blogspot.com/2010/01/however-cold-it-gets-this-is-officially.html#reply
I think this may be slightly off topic but you heard it here thirdly.
“The UK government has decided that the Met Office should have a new name to demonstrate their role in climate science. From 31st January 2010 the service is to be renamed the Meteorological Science Service. For ease of use this will be shortened to “The MESS”. A government spokesmen said this acronym truly demonstrates the services involvement and capability in climate science and weather forecasting.”
End of de press release.
I did some checking of the new Hadley
CRUTv3.
When I downloaded in June,
Temps for first four months were
0.37 0.354 0.356 0.38
Now they are
0.372 0.362 0.364 0.408
When I look back at 2007/2008/2009.
Changes in 2007 are about 0.001.
But 90% of the changes from my
June version of CRUT3V to version
on web-site have T’s increasing.
Interview with Mann — he mentions the 52/62 controversy.
See the end of the article.
Michaels also flagged an e-mail in which Mann discussed a reference he was writing for Jones. In that, Mann misstated a number called the “h index,” which measures a scientist’s productivity and influence.
Mann said it was just an error in a hastily typed private e-mail, in which he meant to say 52 but instead said 62. He produced a copy of the official letter, in which he got the number correct.
He said he was surprised that Michaels would castigate him over a typographical error. “They’ve sunk to a new low,” Mann said of the e-mails. “This shows they’ve given up legitimate debate.”
Don’t think there is much to say now till AGU pulls out a letter sayed different — with say a date log stamp or something.
Looked like a softball interview — but still.
Re: WillR (Jan 9 16:05),
Forgot link — sorry about that.
http://www.philly.com/philly/news/homepage/81058187.html?viewAll=y
http://bigjournalism.com/pcourrielche/2010/01/12/peer-to-peer-review-part-iii-how-climategate-marks-the-maturing-of-a-new-science-movement/#more-2402
The last part of an investigative report on who got the climategate files and when, Mosher had them before release.
General Information that may be of interest, I didn’t know where else to post the link.
6 Trackbacks
[…] double standards on FOI […]
[…] Jan 1, 2010 at 8:54 PM | Permalink | Reply Re: sending “loads of data” to Rutherford, the term “loads of data” is too […]
[…] And ClimateAudit takes aim at double standards concerning “confidentiality” of temperature records. […]
[…] AGW, CA, proof at 1:39 pm by shewonk Over at Climate Audit, there are a few comments in the “Lots of Station Data” post that get to the heart of the matter when it comes to the CRU […]
[…] “Sent loads of station data to Scott” “Community” climate scientists (e.g. Nature, Realclimate) have been quick to accept the idea that CRU was […] […]
[…] "Sent loads of station data to Scott” […]