Server performance at http://www.climateaudit.org has become impossible. I’ve set up a mirror on wordpress. And will try to figure out how to coordinate things.
-
Tip Jar
(The Tip Jar is working again, via a temporary location) -
Pages
-
Categories
-
Articles
-
Blogroll
- Accuweather Blogs
- Andrew Revkin
- Anthony Watts
- Bishop Hill
- Bob Tisdale
- Dan Hughes
- David Stockwell
- Icecap
- Idsos
- James Annan
- Jeff Id
- Josh Halpern
- Judith Curry
- Keith Kloor
- Klimazweibel
- Lubos Motl
- Lucia's Blackboard
- Matt Briggs
- NASA GISS
- Nature Blogs
- RealClimate
- Roger Pielke Jr
- Roger Pielke Sr
- Roman M
- Science of Doom
- Tamino
- Warwick Hughes
- Watts Up With That
- William Connolley
- WordPress.com
- World Climate Report
-
Favorite posts
-
Links
-
Weblogs and resources
-
Archives
Climate Audit 
22 Comments
Are the files from the FOI request and have they been released by someone revolted by the actions of the scientists?
Deleting emails after an FOI request is a criminal offence in GB.
Steve,
I always thought you should have a blog. I’ll put a link up to it since I still am having an impossible time getting to the old one.
You have chosen the right topic. Laws are for everyone, even for climate scientists.
Is there evidence of collusion between french and Jones to subvert your FOI requests and if so what are the possible consequences of such action.
“Housekeeping emails are absent from the file. ”
I thought this was pretty interesting too. I’ve since done a bit of thinking about it, and I’ve come to the conclusion that it’s not as interesting as I originally had thought.
The data was stolen off of an e-mail server. I’ve looked through my own archival e-mail from years past (I have no idea why I’ve kept it). The same kind of pattern emerges – there are basically no e-mails from others inviting me to baby showers or automated out-of-office messages, or anything time-sensitive like appointments, etc. The only things I’ve kept were relevant information (things I might want to read again) and e-mails from others which I kept for CYA reasons (in case anyone were to ask later whose bad decision it was to do something and the blame might end up being mine). All of the inter-office nonsense which didn’t actually impact my job have been deleted.
E-mails stored on a server (archival e-mail) seem to be fairly self-selecting – many servers have storage limits imposed on the users. Additionally, it makes sense to clean up e-mail before archival since more junk can make it difficult to find something pertinent. This is not to say that the hackers didn’t pare the list down a bit before-hand, but it doesn’t seem like they’d have had much time to do so in the time between the hack and the files appearing everywhere. I just think that it’s probably fairly sensible to assume that others who have a similar penchant for storing large numbers of archival messages would behave in a similar fashion to me.
Hopefully CA gets back to somewhat normal traffic levels again. đ
Housekeeping emails are absent from the file.
I don’t understand what you mean by this. What is a “housekeeping email”?
They are trivial emails such as
“Hey Babe, I’ll be late from work tonight, can you put my dinner in the oven please?
That sort of thing…
Steve, you have taken on a thankless task over the last many years.
Well, ‘thankless’ in the eyes of the Mainstream Media and the Team (Hansen, Gore, Mann et al).
You know how thankful so many of us continue to be for your efforts.
Please be safe, and most importantly, take care of your health.
The ‘Server Performance’ you refer to just shows how many of your aficionados are trying to access your main site.
A site that stands for integrity, and truth, and that stands in stark contrast, especially now, after recent revelations, to the sites of the Team.
3 Cheers!
Steve, you might want to put some rules up on this mirror site. Although us lurkers currently lack access to your site for cross referencing, it is becomming fairly easy to put real names to academic posters using the time stamp. You worked hard to help to preserve that anonimity, I think you should continue to uphold that policy.
Steve,
Although the emails provide the juicy stuff, what of the actual datasets and code, which ultimately is the “science” they all rely on.
Have you had an opportunity to check if any of the files are of the data you were looking for all these years?
Hi Stephen,
I just sent an email to you at the address posted at Climate Audit. I am writing a letter to the editor of our newspaper and since I refer to your recent work in the letter (Yamal), I’m wondering if you could do a quick review and provide comments i.e. that I’ve accurately represented your recent work.
I can imagine that you are swamped as the events of the past few days have unfolded and I greatly appreciate any time you can afford.
Thanks,
Vern
I noted on WUWT that the FOIA.zip dump appeared to be compiled in response to a FOI request. I am more convinced after reviewing Steve’s chronology here. I am also more certain that the whistleblower is at UEA.
It’s possible that as this story develops and grows, someone will come forward and offer to host the real Climate Audit on a larger bandwidth server. I have seen this sort of thing happen in the past.
Regarding server performance for Climate Audit.
I’m sure you’re on top of the problem, but thought I’d mention this just in case.
I know you have been getting huge amounts of traffic in the last 36 hrs or so, but wonder if speed problems might be because you have exceeded an agreed server bandwidth allowance for the week/month.
With many host providers, they throttle you speeds automatically once this limit is reached(until you upgrade/pay more). An e-mail to them could confirm.
This could also explain why it is consistently slow across 24 hrs, even when traffic might be less intense.
Just a thought.
OT: Does anyone know if this paper under discussion was published somewhere:
http://www.anelegantchaos.org/cru/emails.php?eid=321&filename=1054756929.txt
“I would like to play with it in an effort to refute their claims.
If published as is, this paper could really do some damage. It is also an ugly paper to review because it is rather mathematical, with a lot of Box-Jenkins stuff in it. It won’t be easy to dismiss out of hand as the math appears to be correct theoretically”
Hah!
ChrisS: CA is not slow around these hours … the same yesterday (NA is sleeping). It will be slow again in 2-3 hours as the east coast starts waking up.
Housekeeping emails involve things being swept under the carpet or put through the laundry. Nothing like what’s been published then.
I think we should take up another collection to help defray the costs of upgrading Steve’s servers and systems.
Steve,
As a retired IT person from an educational institution, I find this whole email question interesting. I saw the articles on the so called hack of the CRU and decided to look into it. I found it interesting that the emails in question were text files with a UNIX email server’s time date stamp tracking as file names. When I opened the email text files I also found it quite interesting that someone had taken the time to delete the full email tracking headers from the file. What was the person trying to cover up from the deletion of the headers? Well I think it is quite apparent from this is that he or she was covering up the fact that it was a blind carbon copy of the email automatically send to the email administrator for archiving purposes. This is pretty much standard operating procedure for UNIX server administrators. The FOIA folder was most likely created by an server administrator for a freedom of information request.
Although I am not familiar with the UK’s IT email archiving practices, I can tell you that the people in question are royally screwed if they think that any of the emails that they think they deleted were to a person or copied to a person at a US institution.
A 2006 US Supreme Court ruling on the “Federal Rules of Civil Procedure” forces any publicly funded K-12 educational institution, college, University, government organization, and federal grant receiver to archive all email. So any email after 2006 that they think they have deleted from or to a person at one of these institutions is still floating around on a server. Even worse for the people at CRU telling people to delete their emails is that most of the major universities were archiving email for many years even decades before 2006 because of the university’s or organization’s lawyers requiring it. So in effect they were just deleting the email off of their personal computer and the active email database not the server’s email database archives. US businesses have also been archiving emails for decades because of legal concerns.
Next I decided that would look at the documents in the documents folder to see if I could determine anything from them. Well after a quick look at the doc, rtf, and pdf files it seems apparent that they are the real thing. Their properties and change tracking show that they were created and edited by the people that they claim to have come from without any additional third party editing.
People are talking about the emails being smoking guns but I find the remarks in the code and the code more of a smoking gun. The code is so hacked around to give predetermined results that it shows the bias of the coder. In other words make the code ignore inconvenient data to show what I want it to show. The code after a quick scan is quite a mess. Anyone with any pride would be to ashamed of to let it out public viewing. As examples bias take a look at the following remarks from the MANN code files:
function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$
datathresh=datathresh
;
; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES
; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate
; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE
; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE
; USUAL correlate FUNCTION ON THE RESIDUALS.
;
pro maps12,yrstart,doinfill=doinfill
;
; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.
;
;
; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD
; reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.
;
This site is missing a tip jar!
Ok, Steve, et al;
Get ready for another tidal wave of web traffic.
The GB TV Show is going to cover the CRU doc-bomb, within the hour.
Maybe you’ve already seen it from the various radio talkers.
BTW, you might enjoy ‘No Highway in the Sky’ , from the late 1940’s.
Jimmy Stewart plays the pesky scientist who eventually triumphs over the entrenched interests.
VHS only, but worth the trouble.
(remarkably prescient of the ‘Comet’ failures of the subsequent decade).
TL
Steve,
Regarding your server problems and the WordPress mirror site, here’s what I’d recommend:
Use WordPress as your publishing platform only, i.e. download the WordPress publishing platform file, but STILL USE YOUR REGULAR OLD climate audit URL. The url with the “wordpress” in it is unnecessary.
Then you get your blog hosted by a third party. I have mine hosted by GoDaddy, but there are many. It costs me about $50/year for the hosting.
Just a suggestion.
By the way, here’s a post I put up today that might make you chuckle (or worse) about a CRU staffer that has “data manipulation” in their job description!
http://algorelied.com/?p=3237
I note that tonight with the last few CA threads closed to comments and it being past most people’s bedtimes, the original site is pretty peppy. I don’t know what that means in terms of the server, however.
At least i was able to use the Google Search and check to see when CA became aware of the problem with the stripbark growth form (as opposed to CO2 fertilization) and it doesn’t seem to be earlier than with the Team (though there may have been a couple of discussions earlier that I’ve missed.
Well, I’m surprised it took them this long to decide to just hammer the CA site.
Or, perhaps it is just having a very busy day…
TL