An excellent analysis of whether the CRU zipfile was “hacked” or “leaked” here, arguing for a combination of a dossier prepared by the university in relation to potential FOI responsibilities (though not necessarily FOI requests in hand) and discovery of the dossier by someone at the university who released it to the outside world – very much along the lines hypothesized by Charles the Moderator of WUWT, but substantially fleshed out.
This is a detailed analysis, unlike IPCC allegations of attacks by the Russian secret service (their version of “A miracle occurred”).
Climate Audit 
73 Comments
The inability to track recent comments is *severely* limiting the usefulness and impact of this blog.
Steve: Drives me crazy too. The new CA is being prepared as we speak. Say nice things to MrPete, John A and Anthony.
To MrPete, John A, Anthony:
“Nice things”
🙂
Thanks! We’re not done yet, but at least you can view and respond to posts.
“unlike IPCC allegations of attacks by the Russian secret service”
Which will only increase when Putin or his #2 makes a demonstrative NYET at Copenhagen. Which they already suspect will happen, explaining the allegations in the first instance.
Someone took a long time filtering out personal emails, and there is a substantial lack of uninteresting, mundane business emails also. Arranged with care.
nah, actually it was russians funded by george bush and al qaeda
I read this over on WUWT and was surprised he missed the epoch times as file names. It isn’t a show stopper based on the final conclusion but I don’t completely agree with it.
He claimed the ‘simplest’ answer was “that someone at UEA found it and released it to the wild and the release of FOIA2009.zip wasn’t because of some hacker, but because of a leak from UEA by a person with scruples.”
Er… no. While I agree it appears it was from the inside nothing in the analysis can make a claim as to their motivation.
It could have been someone was rather ‘careless’ or that someone was ‘paid’ to do so. Or maybe, the person(s) happened to be a programmer who was tired of Phil Jones’s and others and their excessively large EGO’s.
We all wanted ‘Deep Throat’ to be a person of good conscious too but in the end he was just po’ed that he was passed over for promotion twice to he big chair.
My 2 cents
Take comfort from the fact that anyone with any degree of technical knowledge will find it ludicrous to equate “uploaded to a server physically located in Russia” with “hacked by Russian spies”.
This is a distinction that’s easy to make convincingly to the man in the street and when it’s pointed out, the original contention will appear justifiably risible; it will be another own-goal for anyone to try to make capital on this point.
It will be very interesting to see how the UK Police progress their enquiries. No-one hold their breath for any bold pronouncements. One could see this one tailing off as “inconclusive”. Political inteference in the investigation should not be discounted. The fact Gordon Brown called all sceptics “flat earthers” is cause for serious concern.
This is why i feel it will inevitably be the scientific community itself which finally deals with the problem within pertaining to climate sciences.
The community needs to sort itself out because if that enforcement is dictated by an outside agency it will take a generation or more for the public to regain trust in science.
Denbo: The author updated his analysis making note of the epoch timestamps issue you raised. The updated article with corrections here http://www.smalldeadanimals.com/FOIA_Leaked/
OK, I may have found something interesting – or may not. So I decided to bring it to the attention of you experts.
http://strata-sphere.com/blog/index.php/archives/11774
Hopefully this is of some value to the cause.
I find it intriguing, given the importance of the harry_read_me.txt file, that nobody seems to know much about the author (apparently one Ian “Harry” Harris). From his comments in that file, he appears to be a prime suspect for the leak, but I haven’t heard that the paparazzi are camping on his doorstep. Personally, I’d like to sit him down over a few beers and get him to give me his opinion on the “science” conducted at CRU over the last few years.
Perhaps he’s been locked up in a basement somewhere?
“The fact Gordon Brown called all sceptics “flat earthers” is cause for serious concern.”
The overreactions come from nervousness. One place it can come from is guilty knowledge. In Brown’s case, and the Brit political class, it comes from fear arising from the knowledge how dependent local financing is tied up with carbon derivatives. They are far ahead of the USA on this, and their natives are getting restless. From Friends of the Earth:
http://www.foe.co.uk/resource/press_releases/carbon_trading_05112009.html
As someone who has administered unix systems this explanation is what I have thought was most likely from the start.
My first assumption was that it was likely a system administrator who leaked the file, or it was placed where someone inside had access to it and decided to release it.
I seriously doubt they were hacked from the outside, though it is quite possible.
Steve:
Are you going to analyze the data that was part of the CRU file download or are you going to wait until the data is officially released by UEA CRU?
I suspect there is a lot of distrust and paranoia to be found within the EAU.
As has been noted, the email filenames are UNIX timestamps, e.g.
1258053464.txt
From: “Thorne, Peter (Climate Research)”
To: “Phil Jones”
Date: Thu, 12 Nov 2009 14:17:44 -0000
1258053464 -> 12/11/2009 19:17:44
1255530325.txt
From: Michael Mann
To: Kevin Trenberth
Date: Wed, 14 Oct 2009 10:25:25 -0400
1255530325 -> 14/10/2009 14:25:25
0826209667.txt
From: “Tatiana M. Dedkova”
To: K.Briffa@uea.ac.uk
Date: Thu, 7 Mar 96 09:41:07 +0500
0826209667 -> 7/03/1996 14:41:07
They agree to the second with the Date field set by the sender but the inconsistent hour values may have some bearing on whether all names were generated by the same software at the same time.
Looking at the dates provides some extra info. The FOIA.zip file contains emails dated up to the afternoon of Nov. 12, 2009. The next day, CRU rejected Steve McIntyre’s freedom of information request (which they reference as FOI_99-44). On Nov.12, the FOIA.zip file appeared on a .ru server.
More importantly, Paul Hudson at the BBC claims he got the “same” files on Oct. 12, 2009. Obviously it could not have been the SAME version of FOIA.zip, which contains files dated into November. But he got something very similar.
Therefore–whoever provided FOIA.zip in November probably also had access to the essentially the same information in October. I suppose it COULD have been a hacker–but it sure looks like somebody who had access to the files on an ongoing basis, and who made the decision to “go rogue” after FOI_09-44 was denied.
Steve: please do not use the Paul Hudson thing in your reasoning. It’s almost certainly not the same thing.
Looking at the dates and times in some of the .doc files, I’m beginning to suspect that the moles are the night watchman (or an insomniac post-doc) and the departmental secretary, both working for someone in the +4 time zone. Some of the creation/modification/printing times are a bit odd, as are the total editing times of some of the documents.
“The community needs to sort itself out because if that enforcement is dictated by an outside agency it will take a generation or more for the public to regain trust in science.”
The public should never have had trust in this science in the first place. Trust comes when science provides useful tools. The ‘Follow me off cliff’ promotion of climate science and ‘Stop natural evolution’ green science offers no useful tools.
Quondam – the hours are different due to time zone differences. Mann is 4 timezones from Greenwich in mid-October due to Daylight Savings Time.
As far as Unix timestamps go-
It should be pointed out that Unix Epoch Time is an expression of Greenwich Mean Time. It doesn’t recognize time zones. The local time settings on the machine where the mail client application resides uses the timestamp to determine what to print in the email date/time header.
All 1,073 .txt file names have the same 4 or 5 hour offset (depending on daylight/standard time adjustments) relative to the date/time contained in the email, regardless the time zone of the sender/recipient.
We’ve discussed this anamoly here previously. It’s Interesting, but not germaine to Levsen’s very good analysis.
steve,
true, paul hudson’s chain of emails is not the same thing – cos emails from later than 12 oct are in the uea cache, BUT bbc is vulnerable to pressure because of public funding and they need to release ALL the stuff hudson received, so the public can decide whether or not bbc sat on info critical to what we now call climategate. best wishes
I agree with Mark Barratt, this Harry guy has to deal a mess he has not created. Why would he (on his first day on the job!) use strong language to comment on his own incompetence and the mess he finds? And why does he try to solve these problems on his own?
I think the route into the CRU server – and directly to Jones’s emails is found in the file: 1248862973 – dated July 29 2009.
Here Mann writes to Jones:
“Santer et al paper still didn’t come through in your followup message. Can you post in on ftp where it can be downloaded?”
Jones replies to Mann:
“See below for instructions […]
file is at http://ftp.cru.uea.ac.uk
login anonymously with emails as pw
then go to people/philjones
and you should find santeretal2001.pdf”
(my emphasis)
This email is CCed to
Kevin Trenberth
Jim Salinger
j.renwick
b.mullan
Gavin Schmidt
James Annan
Grant Foster
Foster’s address being a ‘tamino’ Hotmail account. Now ‘Tamino’ isn’t the most appealing character on the block for “loathesome” AGW sceptics – and much of what made him so depended upon his hiding behind a pseudonym that many people would have liked to officially ‘out’ him from. Apparently, Hotmail email accounts are fairly easy to crack – with instructions on how to do so openly available via a quick search on the internet. Anyone succeeding in such a crack would have found Jones’ ftp instructions in ‘Tamino’s’ in box.
If Jones did indeed have a folder named “emails” on http://ftp.cru.uea.ac.uk (as his password suggests), and if that folder was intended to be a password-protected repository for the “loads of emails” Jones claimed to have deleted “2 months ago” (on December 3rd 2008) in response to FOI requests (1228330629), then anyone cracking into Tamino’s Hotmail email account would have come up trumps.
Ok, us sceptics keep getting accused of being conspiracy theorists, so here’s one ( Tabloids and visitors from realclimate please note – this is NOT a serious suggestion, although it is as credible as some of the stuff we keep getting thrown at us)
This file was obviously prepared in response to an FOI request, which they finally couldn’t refuse. But, rather than honouring the request and accepting inevitable fallout over the contents (especially from poor Harry’s work) they made a strategic decision to create a “leak” where they could detract from the contents by shouting about the “illegal hack” – hell, if it backfired they could always blame the Russians or something! They then, obviously, timed the leak so that it would run for a bit then be overshadowed by Copenhagen.
Can I have a scripting job on the next Bruce Willis film now?*
* or as a spokesman for the IPCC 😛
Praise be! Recent comments! 🙂
😀
At least the Russian secret service would plausibly have the resources to collate the information in the file. So they are getting better at their cover stories. It is so obviously a leak it is funny that we even discuss any other possibility. Who ever did it has a pretty good idea of the difference between a climate arse and a climate elbow.
Over and above the questions marks over the basis of AGW, I’m not a huge conspiracy theory fan.
But given Steve’s “Augean Stable” comment previously, whether a deliberate and subtle tactic or not, releasing such a huge volume of information could act like electronic chaff.
The thing that struck me was that Steve Mc. precipitated this sequence of events by relentlessly plugging away at the data, rather than being subject to the distractions of witless name-calling and deconstructing motive.
You can’t snap at every minnow in the shoal.
It would be interesting to organize the people and organizations along the lines of:
http://www.muckety.com/Albert-A-Gore-Jr/21.muckety
Interesting but I think he’s wrong in his analysis.
The most recent email, is To: Phil Jones. And at the bottom it says the attachment has been converted and is stored in c:\eudora\attach\….
This means that as recently as 12th November Phil Jones was still using eudora as his mail client. And the leaked emails are copies from what had been downloaded to his desktop PC. These emails are emails after they have been downloaded and processed by an email client – in this case Eudora. All the headers have been thrown away except the ones that get displayed. And attachments have been converted from base64 encoding. Also, I think, Eudora has some facility for spotting URLs in a message and turning them into references, and these are marked in the txt files.
Now I think eudora stores emails in large mailbox files. But these have basically the same format as mailboxes on a unix server. And as they are just great big text files, there is no real problem writing some small script to work through the mailbox file and spit out separate .txt files for each message.
A lot of the emails are either To: From or CC: to Phil Jones. and presumably come from the mailboxes on his desktop PC. However, some are not. So this would imply that the emails have been culled from more than one person’s mailboxes on more than one desktop PC.
One obvious scenario, is that someone was sent to go round each PC and look for data that related to a FOI request. However, it is also quite possible that copies of every person’s desktop PC end up back on a university server. It might be that there is a backup service, so that stuff on laptops or desktop PCs gets uploaded to a server and then backed up to tape. Or, it’s possible that the university runs a thin client setup. In which case, the data doesn’t really reside in the workstations but agan on some central server.
What I think is clear, is that a great deal of very intelligent selection has taken place. There are no office gossip emails, no spam, and presumably sometimes these guys send each other emails which are just about science and not about news management. There seems to be a lot of signal and very little noise. which thinking about it, implies the work was not done by a climate scientist.
Whoever did the selection must really know the ins-and-outs of the climate debate. If this were really done by a hacker, I’d say thay would have to be fluent in english, and they must have followed the debate on a place like Climate Audit – for several years. And if they ripped off entire mailboxes and then sorted through them for the nuggets – they must have been at it for months or there must be a fairly large team of them.
On the other hand, if an insider was tasked with putting this information together I’m slightly unsure as to why some things were chosen. The emails seem about right. I can understand that all of these emails are ones that might be relevant. You might want to put these through the electronic equivalent of the shredder. But the mix of documents is slightly strange. There are various funding applications/grant applications. There is the delightful Harry readme. The nastiest (speaking as a british taxpayer) is the government financed document explaining to government agencies how they should go about changing people’s perceptions of climate change. Basically how to manage a propaganda campaign. But I fail to see how these would be part of an FOI request. The communicating_cc.pdf is labelled as Crown Copyright don’t see that a university would have any requirement to release it.
Personally, I think the hacker argument is unlikely. I think this is a combination of leak possibly after some internal process to gather stuff for a possible FOI request. And I’m quite happy that whoever did it should remain anonymous. They’ve performed a public service but I doubt if they would benefit from being outed. It would probably blight their chances of ever getting another job.
Can my previous comment pass moderation? Or do I have to repost it without active links???
@Third Party
There’s a graphical analysis of email exchanges arranged as a network of nodes connected by lines of various thickness depending on volume and frequency.
Not exactly the same as the link you provided (and I can’t put my finger on it at the moment) but similar. I’ll grep my history fir the link.
I think the route into the CRU server – and directly to Jones’s emails is found in the file: 1248862973 – dated July 29 2009.
Here Mann writes to Jones:
“Santer et al paper still didn’t come through in your followup message. Can you post in on ftp where it can be downloaded?”
Jones replies to Mann:
“See below for instructions […]
file is at ftp . cru . uea . xx . xx
login anonymously with emails as pw
then go to people/philjones
and you should find santeretal2001.pdf”
(my emphasis)
This email is CCed to
Kevin Trenberth
Jim Salinger
j.renwick
b.mullan
Gavin Schmidt
James Annan
Grant Foster
Foster’s address being a ‘tamino’ Hotmail account. Now ‘Tamino’ isn’t the most appealing character on the block for “loathesome” AGW sceptics – and much of what made him so depended upon his hiding behind a pseudonym that many people would have liked to officially ‘out’ him from. Apparently, Hotmail email accounts are fairly easy to crack – with instructions on how to do so openly available via a quick search on the internet. Anyone succeeding in such a crack would have found Jones’ ftp instructions in ‘Tamino’s’ in box.
If Jones did indeed have a folder named “emails” on ftp . cru . uea . xx . xx (as his password suggests), and if that folder was intended to be a password-protected repository for the “loads of emails” Jones claimed to have deleted “2 months ago” (on December 3rd 2008) in response to FOI requests (1228330629), then anyone cracking into Tamino’s Hotmail email account would have come up trumps.
“emails” in this case is simply the normal anonymous ftp login instruction: for anonymous ftp, you’re requested to login using your email address.
Nothing to see here; move along 🙂
I like my explanation for the mysterious CRU e-mail mystery….. It’s the fault of the Large Hadron Collider.
OK. OK. I know you’re saying “What?” but bear with me.
Many predicted that if CERN tried to restart the Large Hadron Collider, either the world was either going to be swallowed up by micro black holes (hasn’t happened so far), or because of the quantum nature of the higgs-boson particle, that it can travel through time, it doesn’t want to be discovered. It is often called the “God Particle” after all. Thus the very act of trying to discover the particle would stop the LHC from ever firing – note how the actions of one bird caused a severe delay the reboot process. That was one-heck-of-a-strategically-placed bread crumb! But there is a third possibility. Maybe the effects of the higgs boson do travel through time, but they don’t prevent its discovery…. but, however, they do strange things to the world in a quantum way. Obviously, the LHC has succeeded sometime in the near future of creating the “God Particle”, and since the effects of such are quantum and not anchored in time, well, the effects are being felt now! And one of the effect is… drum roll please…. to cause the CRU e-mails and code to go all quantum on us and shift from a secured server at the University of East Anglia to one in Vlad’s house that is quite open to the web! It’s so easy to see…. And it’s quantum to boot!
Who can argue with that!
Nick Moon wrote:
“A lot of the emails are either To: From or CC: to Phil Jones. and presumably come from the mailboxes on his desktop PC. However, some are not. So this would imply that the emails have been culled from more than one person’s mailboxes on more than one desktop PC.”
Every email has one thing in common. *(@) uea.ac.uk appears in the From:, To: or Cc: field(s).
This would support Levsen’s theory that they all resided on the same mail server. It would be an easy task to copy them if one had root directory access to that server.
Three puzzles:
1. How much email does a senior academic send and receive per day? Let’s guess that they send 10 and receive 40 on Monday to Friday (as a former academic, I reckon that’s a very low estimate). That’s 250/week/academic. This email corpus concerns about 10 core academics over a 10 year period. That’s a total of around 1.3M email messages — let’s say 1.0M to allow for duplicates. The leaker provided us just over a thousand. Most, perhaps all, are highly pertinent to understanding what has been going on at CRU. That’s one hell of an editorial task.
2. We can infer from the absence of forgery claims from the participants that none of the emails had material added to them. But what about redactions? Academic email is often copied to research students, secretaries, admin assistants, and system support staff if a message contains something that is relevant to them (e.g., so a secretary can set up a meeting). Were the names of such innocent bystanders redacted from the emails? An ethical leaker would probably have worried about blasting their email addresses all over the known universe. If he or she was a junior member of the unit then many of these bystanders would have been the leaker’s friends and coffee room companions.
Did the leaker redact other content — e.g., personal matters such as an enquiry after a child’s health — that might have been appended to some messages?
3. FOIA requests are very specific. Faced with a set of FOIA requests, why would one dump all the relevant material into a single bundle? The rational thing to do would be to create a set of directories, one for each FOIA request, and sort the relevant material into the relevant directory. There’s no trace of such a structure in FOIA.zip.
With regard to public access to data, which would obviate the need for hackers or leakers, there is an interesting post on UK government policy at
http://blogs.telegraph.co.uk/technology/iandouglas/100004326/government-information-gropes-towards-freedom/
In particular, around halfway down the article, this:
“Opening Met Office Public Weather Service data to include: releasing significant underlying data for weather forecasts for free download and reuse by April 2010, and working to further expand the release of weather data, while recognising all public safety considerations; releasing a free iPhone application to access weather data by April 2010; releasing a widget that enables other websites to deploy Met Office supplied weather information by April 2010; and making available more information on Met Office scientists, their work and scientific papers, free of charge”
A problem I have with his analysis is his claim:
“The hacker would have to crack an Administrative file server to get to the emails and crack numerous workstations, desktops, and servers to get the documents”
This is not entirely true if you’re using an LDAP or directory based authentication infrastructure.
It’s possible UEA are using OpenLdap or the like.
Social engineering is easier with prepackaged requests for sure, a luxury that did not seem to have enjoyed those who allegedly tried to break into the University of Victoria recently: http://www.nationalpost.com/story.html?id=2300282
I like SonicFrog’s theory.
I rather like this cartoon from our oh so caring and green NZ Herald too…
http://www.nzherald.co.nz/news-cartoons/news/headlines.cfm?c_id=500814
I think the most important thing now is to make sure these emails aren’t pushed under the table. Too many politicians and IPCC officials have publicly dismissed them. Everything must be re-evaluated. The proxies have to be fully tested against satellite measurements. It doesn’t seem as if this has ever been done. You have every “scientist” simply choosing an ad-hoc value for the error in the proxies. Phil Jones goes around telling people that global avg. temp will go up .2C per decade. How can he use a level of accuracy close to or less than the margin for proxies measurement error? The name recognition, prestige and the weight given to his opinion has gone right to his head.
Lets not forget dear old Micheal Mann now saying the tree ring proxies are unreliable. Of course then why was it ok that Phil Jones showed everyone in 2007 the tree ring temp graph. Mann is fine with using middle ages proxies only when it doesn’t show the middle ages warming.
What does everyone have to say about te proxies accuracy and whether the proxies have been compared o satellite data over the last 30-40 years?
“bender permalink
The inability to track recent comments is *severely* limiting the usefulness and impact of this blog.
Steve: Drives me crazy too. The new CA is being prepared as we speak. Say nice things to MrPete, John A and Anthony.”
Steve, are you in need of another upgrade?? Let us know if need be…I agree, I liked the “old” site better….Keep it goin!!!
I am a librarian, not a climate scientist or computer programer. Thus, my reading of these documents is with an archivist eye rather than that of a scientist.
It appears to me that whoever put these together was working from a set of parameters. Almost like generating a subject list in a library catalog. However, I don’t really have a clue as to the vocabulary or indexing system. If we knew that then datamining the documents would easier because we’d know exactly what the author/editor/provider wanted us to find rather than having to work through the paers. It is obvious that they are carefully selected, by someone who knew the EAU-CRU computer system and file structure. There is almost no way a hacker did this. Maybe a mole but not a hacker. Whoever did this knew how to access the system. I’d like to have their indexing notes.
They do not appear redacted. If they were then the materials would not read as smoothly as it does unless there was extensive editing. If there was editing then I think the principles involved would have used that editing as a defense.
Thanks for the notification that there was a node analysis out there.
Here’s the one I found: http://seadragon.com/view/h0i
Quite a few more points of contact than I’d have guessed.
From: http://computationallegalstudies.com/2009/11/27/visualizing-the-east-anglia-climate-research-unit-leaked-email-network/
“Hubs and Authorities:
In addition to the visual, we provide hub and authority scores for the nodes in the network. We provide names for these nodes but do not provide their email address.
Authority
1. Phil Jones: 1.0
2. Keith Briffa: 0.86
3. Tim Osborn: 0.80
4. Jonathan Overpeck: 0.57
5. Tom Wigley: 0.54
6. Gavin Schmidt: 0.54
7. Raymond Bradley: 0.52
8. Kevin Trenberth: 0.49
9. Benjamin Santer: 0.49
10. Michael Mann: 0.46
Hubs returns nearly identical ranks with slightly perturbed orders with the notable exception that the UK Met Office IPCC Working Group has the highest hub score.
Thus, so far as these emails are a reasonable “proxy” for the true structure of this communication network, these are some of the most important individuals in the network.”
I’m not familiar with all the FOI requests they had, but I wonder if it would be possible to narrow it down to which one it was based on the content? If there were an ‘all emails relating to x, y, and z’ type list it should be possible to get close to a match with just pattern matching.
Also if all the FOI requests required say mail that spoke about say Steve McIntyre, and Steve has email he sent and received from one of them that isn’t in the list of emails, one might get closer to knowing if this was content not to include, rather than content required to include. Given what some of it says, that might be less surprising (although that would be even worse than FOI evasion imo — speculation of course).
Hey Jeff,
how did you get that onto youtube?
I would like to try to do the same thing with the Wegman segment on CNN.
The link seems to just be some kind of java thing.
TIA
TL
Wow, I think Peter S may be on to something (S for Sherlock I assume : )… That really does seem to be plausible. Imagine someone hacked someone’s email account a while back, and was just snooping. That email also does seem to suggest there were files missing, and the files was in a zip file, and placed on a server. The ’emails’ pw is a double clue – it suggests emails (with some files as well as that is what is being sought), and it is a password which is potentially how the break in happened.
That could explain the weird Paul Hudson chain of emails too. That sounded more like he was sent a single thread of one email chain, and it did sound like it was from someone ‘outside’. I suspect he was sent the FOIA\mail\1255558867.txt series (ok, that seems sure).
Paul says:
“I was forwarded the chain of e-mails on the 12th October, which are comments from some of the worlds leading climate scientists written as a direct result of my article ‘whatever happened to global warming’. The e-mails released on the internet as a result of CRU being hacked into are identical to the ones I was forwarded and read at the time and so, as far as l can see, they are authentic.”
So potentially this hacker guy was evesdropping, found this and thought it was a clear smoking gun — looking again at Tom Wigley’s mail in there, you can see why:
> > Mike,
> >
> > The Figure you sent is very deceptive. As an example, historical
> > runs with PCM look as though they match observations — but the
> > match is a fluke. PCM has no indirect aerosol forcing and a low
> > climate sensitivity — compensating errors. In my (perhaps too
> > harsh)
> > view, there have been a number of dishonest presentations of model
> > results by individual authors and by IPCC. This is why I still use
> > results from MAGICC to compare with observed temperatures. At least
> > here I can assess how sensitive matches are to sensitivity and
> > forcing assumptions/uncertainties.
> >
> > Tom.
That goes to thePaul Hudson at the bbc, and nothing.
Why not make it public then? Maybe the hacker feels that if that is forwarded elsewhere, everyone involved knows they are compromised, and changes passwords etc, so the guy waits. Then he gets the stuff Peter S mentions (foia2009.zip I assume) and uploads it to servers and blogs.
If (yes big if ; ) that is so, then the only four emails that are common to both chains are:
Mann
Jones
Kevin Trenberth
Gavin Schmidt
You would think that would have to mean Jones, as he is the only one at UEA, however all the email we see is from the zip file. One way to test this would be to see the email chain that Paul Hudson received. If that is from the ‘perspective’ of one of those four, then that is probably who was compromised.
If he means identical as in server stamps, then it would indicate Phil Jones. Given he’s taken the fall, maybe that is the case.
Sorry if this is too much wild speculation of a post, I understand the comment policy here tries to keep it on topic and on facts, I understand if this is over the line, feel free to snip…
It’s no proof but it indeed sounds convincing. The file (and/or directories) could have been prepared or set too free access rights and a grad student or whoever had an account on the same machine could have posted it. A minimum amount of miracles is needed.
Not everyone is allowed to read that SysAdmin’s Perspective. When trying to enter the address indicated in Steve’s post, http://www.smalldeadanimals.com/FOIA_Leaked/, I read:
”
217.66.146.92
Opera/9.63 (Windows NT 5.1; U; ru) Presto/2.1.1
FORBIDDEN
If you’ve reached this page, it’s probably due to one of the following conditions:
a) Your isp is blocked because it originates from a country that welcomes spammers. (.ru, .br).
b) Your isp is blocked due to abusive behavior by someone else, and you’ve been caught in the net. Sorry about that – email me at kate [at] katewerk [dot] com and I’ll see what I can do for you.
c) You’re poking around where you’re not supposed to. Stop it.
d) Your isp is blocked due to your abusive behavior. If you think that’s unfair, then email me privately to discuss it.
e) You’re trying to circumvent the block using an anonymizing proxy. See d). Nice try, luser.
”
Well, I am indeed from Russia.
Different take: The emails were not aggregated for an FOIA response, the were being deleted / purged in order to provide and incomplete response. Consider:
They are the residual emails of a batch which had already been *sanitized* from the CRU systems, in order to illegally prepare an incomplete response for a future FOIA request. The emails in question were *not* going to be provided under a FOIA request.
These are deleted emails from a sanitized batch which were foolishly or purposely archived and/or discovered by an insider or whistleblower (perhaps the sanitizer himself). The insider then had pangs of conscience or an axe to grind and released them surreptitiously.
Also: The leaker may enjoy protection under the UK’s Public Interest Disclosure Act of 1998, which was enacted to protect whistleblowers
Long time lurker, first time poster.
Regarding the ’emails’ as password part in that one email is just normal way to log on to a public ftp server…
“login anonymously with emails as pw” as the instruction stated means that you should use ‘anonymous” as username, and any email address as password, eg. foobar@matrix.com (there is no validation other than possibly checking for @ sign).
//K
Is there any consistent searchable word in all the emails? I work at a large bank and whenever we get any sort of legal inquiry on a particular topic, an email goes around asking all of us to forward any possibly responsive emails/documents to compliance.
I use X1 to search all my emails for anything that may be responsive then someone comes to my desk and collects the presumably-relevant emails. I suppose they filter out housekeeping emails that may mention the search word but not be responsive.
Could the CRU emails have been collected in this manner? Is there any reader here who worked at UEA when they were served with a supoena or FOI request so you could compare collection methodology?
Nice!
Are you actually British? Your sense of humor is delightfully dry. I caught part of the CNN interview last night, and now can put a voice (and facial expressions) to your posts. What fun!
Steve: Toronto. 6 generations Canadian Scots.
As Steve requested.
To John A, Mr. Pete, and Anthony.
“Nice Things”
Joe, I like it!
And just because you made it up out of nothing doesn’t mean you cannot use it. You, too, can have a lucrative future in climate science, with good pay, relaxing trips to warm places, and free e-mail accounts!
Anastassia Makarieva,
Here is a link that should work. WUWT
Bob Koss — many thanks.
It is sad that you accept at face value that “analysis”.
It is not a technical analysis, just a description full of technical jargon so that most visitors will take at face value.
The “analysis” does not manage to show that the e-mails were leaked.
There is a big credibility issue with this website and the author.
Maybe I missed it, but is there a reason to believe that all the data files were kept on local drives and not on a central server? It seems to me that it’s well within possibility that data was kept on a central server somewhere, in which an external hack would be feasible. What did I miss?
John P
“The “analysis” does not manage to show that the e-mails were leaked.
“There is a big credibility issue with this website and the author.”
You need to watch your writing. What do you mean by “show”? Prove? Of course nobody can prove something without access to the actual computers. But he/she does show a good set of reasons to suspect strongly that the e-mails were leaked rather than hacked.
And what are you referring to by “this website”? Small Dead Animals or Climate Audit? And what is the credibility issue?
To me your screed just seems to be a drive-by.
JohnP,
I think it is a pretty good argument for why the files were collected internally. As far as how that internally collected zip was released is up in the air though, I agree if that is what you mean.
If there is any technical part in there you doubt or would like to hear in plainer terms I’d be happy to go over those parts with you… (not implying you aren’t right or aren’t comfortable with the tech (are you?), I may have missed something you caught, or there may be alternate explanations for some things I don’t see..
Robin, Dave,
As someone who’s done a bit of digital forensics professionally, this analysis is worse than useless. It goes into great depth about irrelevant points. It makes unstated assumptions, and comes to conclusions not backed by the analysis. A few points about it’s awfulness:
1) The author goes to great lengths to outline the inferred mail infrastructure of the University of East Anglia, all to intuit where precisely the emails were might have been archived. Totally ignoring the fact that many emails in the archive have distinctive traces, via the “Attachment Converted: ” footer, which indicates they have been processed by the Eudora End User Agent. Which indicates that the emails have been recovered from Eudora email archives (stored in standard MBOX format, not some mysterious “binary format”. Which makes all the speculation about department level archiving and postfix configured BCC addresses nothing but fairy tales.
2) The author fails to understand the significance what’s been pointed out to him, that the file names of the email files are unix epoch times. Not only that, but they correspond to the timestamps of the emails, if you ignore the time zone indicated in the email, and instead interpret the time as EST/EDT (i.e., 4 or 5 hours behind UTC). Any hypothesis explaining the origins of these files needs to account for this anomaly to be credible. Perhaps the University of East Anglia outsources it’s FOI operations to someone in Canada. Perhaps the CRU is so in love with the UN that they set the clocks of all their computers to New York Time. Both explanations are more credible than the persiflage in this “analysis”.
It goes on and on. It’s exactly what I’d expect if you had someone who knew something about information technology, but nothing about digital forensics do digital forensics: It’s crap digital forensics. I’d like to conduct a thorough, professional review of the data to come to proper forensic conclusions, but that takes time and effort which I haven’t had available to spare to date. But that doesn’t mean this sort of drivel should be accepted in the absence of decent forensic reviews.
J Petry –
You are mistaken about those eudora attachment lines. Those text lines are added to the body when the sender used certain older versions of Eudora (before standardization of MIME). They do not mean the message was retrieved from an end-user machine, that line would be in the body on the server before it has ever been viewed.
Observer,
No, I’m not mistaken. The most thorough way to verify is to download a Version of Eudora, connect to a pop server and verify the contents of the generated MBOX files. But here’s a less involved way that might convince you. Look at the message in 1059664704.txt. It’s from Michael Mann to Tim Osborne. And the converted attachments are stored in Tim Osbourne’s My Documents directory. That line had to have been generated by Tim Osbourne’s email client. There are lots of other similar examples in the archive.
I’m not just guessing here, I’ve actually reviewed the evidence. I didn’t come at it with any preconception, either, if the evidence actually suggested that these files came from a central mail archive, as this “analysis” suggests, I’d say so. But it doesn’t.
The question of the origins of the zip archive is a largely independent question, and one I’m trying to reserve judgment on. Attribution of cyber events is hard, and generally requires quite a bit more evidence than we have here to establish with any certainty. But to claim that it’s “almost certain” these files were leaked by an insider is just plain not supported by the evidence, period. If you can’t accept that, all you’re demonstrating is your confirmation bias.
J Petry-
Good observation. Does this preclude any possiblity that the emails in question were stored on a server-side folder based on a “last modified” event? Would opening an attachment through a local Eurdora mail client generate such an event detectable by the server?
It’s possible in the sense that anything’s possible, in the same way that it’s possible that the emails were dictated by by psychics using remote viewing, I suppose, but it’s not really plausible or reasonable.
The preponderance of evidence suggests that the most likely scenario is that these files were extracted from Eudora mailbox archives, and that this extraction was done on a unix or unix-like OS configured to use the Eastern Time Zone. I can’t see how you can reconcile these conclusions with the hypothesis that these files were prepared as part of a UEA internal process to respond to an FOI request, and unintentionally made public.
UEA uses UNIX IMAP or Exchange for it’s mail service:
http://www.uea.ac.uk/is/ithelp/itfaqs/email
Virtually all staff uses UNIX IMAP with a 1 GIG storage limit on a central serve-store. It would seem that Levson, the author, should have perused the above webpage before posting his analysis.
I wasn’t arguing for insider vs. hacked at all, just pointing out what I thought was the behavior of the Eudora clients. I accept that you’ve confirmed it.
OT a little bit – but this is the closet post I could find to ask this. I was looking on http://www.di2.nu/foia/ last night, and tehn about midnight Eastern Time I could not get to in again. I have tried all day today, and still cannot. Did nslookup, used IP in the address bar, still can’t get to it. The DNS Auth record look up times out.
1) Anybody body else see the same problem?
2) Is there another site with either all the files available (not just the emails) or b) a download of the zip? Googlee is only showing the di2.nu site that has all the files.
Kevin
Never mind about FOIA2009.zip found a download for it here:
http://www.megaupload.com/?d=75J4XO4T
(Google did not come up with anything for FOIA2009 that was useful) but…
http://www.anenglishmanscastle.com/archives/007984.html
Still curious about http://www.di2.nu/foia/