New climategate dossier here. [Update – try here]
See notice from “FOIA” here and subsequent comments as well as discussion at Jeff Id here.
I’ll comment later after I have a chance to look at things.
[Update /Jean S : Searchable database here. h/t phil]
Climate Audit 
362 Comments
It’s like welcome home FOIA. And thanks.
Lubos Motl posted a link to all emails as one big 65MB text file. BUt it does not have individual file numbers.
Here we go again. Popcorn time!
Unbelievable. Never in all my years as a scientist have I seen such blatant dishonesty so openly displayed. How on earth do they get away with this stuff.
But… is that science?
This turkey is thankful.
========
I’m sure these will add an extra interesting dimension to the discussions at Durban next week. Immaculate timing (if verifiably genuine)
This one perhaps worth picking out….
“I thought I’d play around with some randomly generated time-series and see if I
could ‘reconstruct’ northern hemisphere temperatures.
[…] The reconstructions clearly show a ‘hockey-stick’ trend. I guess this is
precisely the phenomenon that Macintyre has been going on about.
Bradley: “
Lost it’s format – txt 3373
Also…
Bradley:
I’m sure you agree–the Mann/Jones GRL paper was truly pathetic and should never have been published. I don’t want to be associated with that 2000 year “reconstruction”.
Cook:
I am afraid that Mike is defending something that increasingly cannot be defended. He is investing too much personal stuff in this and not letting the science move ahead.
Barnett:
[IPCC AR5 models] clearly, some tuning or very good luck involved. I doubt the modeling world will be able to get away with this much longer
Jones:
Basic problem is that all models are wrong – not got enough middle and low level clouds.
Jones:
I’ve been told that IPCC is above national FOI Acts. One way to cover yourself and all those working in AR5 would be to delete all emails at the end of the process
#3148 is a bit interesting in terms of the team feeling for this blog:
Hi Phil,
Looking forward to the new TS 3.0(!!!!) and the new scPDSI calculated from it. How will
Gerard determine where it is rubbish? From my own perspective, I would prefer to determine
that myself. Will I send the paper to Climate Audit? Have I gone senile or insane? Since
not yet on both accounts, nah! I agree that Venus is better than Mars for those bastards.
It is more hell-like by all accounts. The audit folks might want to debate the existence of
greenhouse warming there as well. Not even Bush would recommend sending astronauts to
Venus. Mars after being bathed in cosmic rays for months to get there is okay by his logic.
Cheers,
Ed
😉 😉 😉
Mann:
the important thing is to make sure they’re loosing the PR battle. That’s what
the site [Real Climate] is about.
and here is Gavin, doing his job.
http://www.realclimate.org/index.php/archives/2011/11/two-year-old-turkey/
BTW, are all of the emails in this cache different from the ones in the last batch?
They’re going to need a lot more whitewash. The only possible conclusion is that the CRU was created up to bring down the CAGW myth.
But who will whitewash the whitewashers from last time?
Here’s a short script that I wrote to organize the files into an R-list so that they are quickly searchable.
K=5349 alias=c("000","00","0","") numbertochar=function(j) { n=nchar(j); paste(alias[n],j,sep="") } cru=rep(list(NA),K) for(i in 1074:K) { handle=try(unz("d:/foi/FOIA2011.zip", paste( "FOIA/mail/",numbertochar(i),".txt",sep=""),"r")) if(! class(handle)=="try-error") {cru[[i]]=scan(handle,sep="\n",what=""); close(handle)} } # save(cru,file="d:/climate/data/cru/cru2.tab")Here is a search function
##disinformation search=function(word,A=cru) { Data=NULL for(i in 1:K) { index=grep(word,A[[i]]) N=length(index) if(N>0) {dummy=data.frame(num=rep(i,N),line=index,comment= substr(A[[i]][index],1,80)) Data=rbind(Data,dummy)} } return(Data) }I did a quick search of my own name and found 941 mentions (of which quite a few are repeats in email threads.
Re: Steve McIntyre (Nov 22 09:26),
also 9 MacIntyres.
And a couple of ‘the Macs’.
Steve, please let us know if you can confirm any of these emails. Do any include forwarded replies from skeptics in the thread so we know they are real?
Jo
these are from the same dossier.
The original dossier typically had vignettes from ongoing correspondence. It was like the curtain being lifted to give a glimpse, but the curtain would then close. The present dossier looks like a much more complete version.
The new dossier is not in calendar order either. I’ve spent time this morning trying to place the emails in order.
That’s great to hear. Sorry for making a link with Durban in my excitement earlier, by the way. I should have let foia’s stated motivation stand as it is. Whatever they’re playing at there’s style and substance. Respect.
Thank you Steve!
Now the converstaions with switfhack make more sense
how so?
it seemed to me that swifthack and RC had talked.. and swifthack was askinghim “what do you want
”
as if there was a deal on the table.
pure speculation
Can we presume CRUtape Vol 2 is on the way?
no. Im busy on other stuff.
unless the press gets it horribly wrong ( which was my first motivation) I am not inclined
to dumpster dive
One of those McIntyre references is #1017 from Tom Wigley to Keith Briffa:
“Keith,
See the last item. Why don’t you just give these people the raw data?
Are you hiding something — your apparent refusal to be forthcoming sure
makes it look as though you are.
Tom.”
The “last item” is
(10) AND FINALLY: SCIENCE SHENANGIGANS GO ON
Steve McIntyre, 9 May 2006
presumably referring to this post.
‘foia’ also left the link on the talkshop at 09.28GMT today, soon after posting here on CA. Making sure it wasn’t missed this time. 🙂
What’s ‘the talkshop’?
My blog. tallbloke.wordpress.com
So far as I know, ‘foia’ left the link at five sites: CA, Wattsup, the talkshop, jo-nova’s and Jeff ID’s.
Why ‘foia’ chose my site in the UK raher than one of the higher profile, more ‘politics’ orientated sites like Bishop Hill, I don’t know.
Maybe (s)he’s a regular lurker. 🙂
I will always be grateful that you rudely interrupted the flow at Judith Curry’s to alert me to FOIA2001 in the morning London time, shortly after it had ‘gone live’ Roger. (I ended up as downloader number 22 on the original .ru site.) Getting the link and spotting it will remain a great feather in the talkshop’s hat – though I decided to focus here on Climate Audit as we tried to figure out what exactly it contained. Of all places, this seems the spiritual home for ‘foia’. As I suggested the whistleblower might be saying on The Mosher Timeline on 15 Jan 10:
It turned out this was the next time – and we’re very grateful.
Warming to see that they managed to duplicate YOUR results, Steve 🙂
” Crowley:
I am not convinced that the “truth” is always worth reaching if it is at the
cost of damaged personal relationships”.”
That’s where you and I differ, bub.
I can only assume that Steig is referring to Steve here:
No, he’s referring to David Bromwich. 2095.txt
Does this make Mitchell a denier?
As I noted at Bishophill, A lot of the “MetO” comments seem level-headed and genuinely sceptical (by comparison with the rest of the hysterical bunch) – shows them in a relatively better light. A clue perhaps….
Except that they won’t say any of those things in public, so no, it doesn’t show them in a better light.
Boy, am I looking forward to Steve’s next article! [Must stop hitting ‘refresh’ and let the man think..]
He is still posting the link. I saw the link on Joanne Nova’s site pop up about an hour ago.
2509.txt is an interesting continuance of the “Travesty” thread.
Synopsis:
Wigley: Mike, the Figure you sent is very deceptive…
Mann: Well, ok…[snip]…but you should talk to Gavin, it’s his plot, after all.
Schmidt: The kind of things we are hearing “no model showed a cooling”, the “data is outside the range of the models” need to be addressed directly.
Heh.
Steve:
Do a search on the word “Oxburgh”.
FYI, a simple text search turned up the following 7 files with “Oxburgh” in them – none of which was after Nov 19, 2009:
0619.txt
0652.txt
1085.txt
1191.txt
2593.txt
5118.txt
5349.txt
A reasonable idea, but nothing of importance.
Steve, no, there wouldn’t have been anything after that date. The significance lies in the term “undependent inquiry”. Thanks for the list, though, as I’ve been unable to set up a search facility on my own…. yet.
Having found them, you should look at the content of those messages.
They don’t seem to be either to him or from him or even concerning him in any way other than he may be speaking somewhere.
If you find something like “I’ve spoken to our buddy Oxburgh and he agrees”. It may be worth looking at.
I don’t think that is the case. This is a non event.
I saw that. That is why I said “This is a non event.”
My download at 10.22am EST was #965.
Checked just now and I get an error “can’t connect to the remote server.” Either it is overloaded or has been taken off line.
The first link seems to be broken 😦
(the “here” in “New climategate dossier here”).
Are these authentic?
Those defending the IPCC team seem to be taking the stance that these emails are no big deal, and nothing illegal. Here is why I think it matters:
1) Actively subverting FOIA intent
2) Admitting a) Hockey stick flawed & Steve is right, b) hide decline was dishonest, c) climate models are pretty bad, and d) cherry picking results like Japan hurricanes to emphasize a pre-ordained message
3) Trying to manipulate (and probably succeeding) who gets to be IPCC author
4) Trying to manage the message (PR concern)
5) Viewing science results as helping or hurting “the cause” — Mann especially
All the above subverts the official messages of “overwhelming consencus” and “science is settled”, world’s best scientists just doing their science, and that it would be “absurd” to see a conspiracy.
I’m not sure I believe that these are real yet. How do we verify that they haven’t been faked?
@ BarryW: Mann seems to think so
http://tomnelson.blogspot.com/2011/11/fresh-round-of-hacked-climate-science.html
Thanks. I hope Leo Hickman/The Guardian or someone gets a few more confirmations. Eventually, UEA should confirm or deny.
The last time this happened, it impressed me that Mosher first made an effort to confirm that the emails were authentic.
That’s probably the right approach for the second batch of emails.
Confirm the authenticity of the emails.
Mann has already done that, while claiming that they contain nothing of importance.
Do you know how much work would have to go into faking 5000 emails? Cui bono?
Recall what I said the other day
Yes, I remember. A little birdie showed up at your window?
premonition
My favorite one so far is where they admit tuning the models to 20th Century climate, which in debates (e.g., one I had in person with Michael Schlesinger and others at Judith’s) is strenuously denied.
Someone calling themselves ‘Joe Public’ has put a torrent link on my blog.
1527.txt
“There has been criticism by Macintyre of Mann’s sole reliance on RE, and I am now starting to believe the accusations”
Always right before climate talks…the timing of these e-mails is rather suspicious.
As though the barrage of alarmist releases isn’t? A little biased are you?
From my comment at Bishop Hill:
=====
As we coast into COP17 on 28th November, BBC R4’s “The Infinite Monkey Cage” will be looking a “balance” with Brian Cox and a special guest appearance from Paul Nurse on that same day.
According to the trailer at the end of this weeks episode
http://www.bbc.co.uk/iplayer/episode/b017cb0w/The_Infinite_Monkey_Cage_Series_5_Whats_the_North_Ever_Done_for_Us/
“…is it only fair to give everyone a platform however wrong they are?”
(at 27:30 onwards)
I expect the fact that these two events are happening at the same time is the purest coincidence, and we won’t be hearing phrases involving the words “denier”, “consensus”, “robust” etc. etc. etc.
=====
Also a marked increase in tweets from various team members.
What ‘alarmist releases’ (specifically those with some connection to working scientists, rather than random journalists)?
I won’t deny that people exaggerate issues on both sides, I was just mentioning the timing. You can pretty much guarantee some sort of nonsense will come out before big climate events- the swindle video, climategate, etc.
So, Chris, you think climategate was “nonsense”? I seem to remember that it clarified the “hide the decline” history, as only one example of what it showed.
Yes, I thought the post-gate “spin” on much the e-mails was “nonsense.” It showed in some cases irresponsibility but nothing more. And everyone knows that “hide the decline” referred to a well-known divergence issue that was justified in the literature- the ClimateAudit twist has no weight and was indefensible.
Hand -> face
LOL pax LOL
No, Chris, what the divergence in each case has been written in the literature is that it is assumed to be a problem that can be ignored, not shown to be justified in removal and/or replacement of different schemes, not shown that it can be ignored.
It might be an assumption, and the cause of the phenomenon may not well be understood. That is irrelevant. The authors made a choice to take out data that is known to be wrong, and that choice was documented. You don’t need to like the choice, but the conspiracies and accusations don’t at all follow, and it’s precisely the reason why everyone outside of two or three blogs does not take CA’s generous host seriously. Yet everyone wonders…
Wow, Chris. “data known to be wrong”–and how does anyone know this? The tree ring data in some series, but not others, goes down not up in recent decades. I explained in
Loehle, C. 2009. A Mathematical Analysis of the Divergence Problem in Dendroclimatology. Climatic Change 94:233-245
that this implies that the tree ring data make a dubious proxy. There is no “proof” that there is something “wrong” with the post-1960 data. No justification for dropping it from graphs. They just dropped it because it did not give an answer they liked. Do your homework.
Craig, I really have no desire to respond to you, but everyone who studies this subject knows the data is wrong (anyone can look at the instrumental record and see this trivially). The problem is also known to apply primarily to only a particular subset of tree rings, and trees in different regions are known to respond differently to various factors. This is not my field, but to be blunt I’d trust a dendroclimatologist to tell me more about the limitations and current understanding than anyone here.
Of course, I didn’t say that the pre-divergence data is “right,” but again, that is irrelevant to the accusations related to climategate. Personally, I agree there is a lot of progress to be made in tree ring science, and whether tree rings are dubious or not, and what they may have the potential to tell us, is largely irrelevant to what I focus on. It’s also of tiny importance to the whole issue of AGW, and almost entirely irrelevant to the physics, sensitivity, or attribution issues.
Well Chris, I actually AM a forester, so trees are my thing. If trees respond sometimes + and sometimes – to temperature (see Wilmking’s work, for example), then they are not very good treemometers.
As far as “it doesn’t matter”–remember the key place of the hockey stick graph in AR3 and the spagetti graph in AR4, as well as the attribution studies (attribution including climate sensitivity) based on Moberg or Mann’s work for a pre-instrumental test of climate models (and used to dismiss the importance of solar activity). So, not so unimportant to the debate.
Well, we clearly disagree regarding its importance and its relevance to future climate change.
Chris, it warms my heart to see you throw out IPCC AR4 with your response to me and Craig. It just makes me wonder where your support for stating it is “”almost entirely irrelevant to the physics, sensitivity, or attribution issues”” which is not what IPCC AR4 stated in chapters 9 and 10.
Though it does tickle me to see you disagree with AR4.
Keep up the good work.
Instead of trying to point score, try to be serious. Certainly, some people see the value in using the last millennia as a good target for climate sensitivity estimates, but it is only a very weak constraint (see e.g., Knutti and Hegerl, 2008 for a review) and while there a few papers on it, they aren’t highly influential; other paleoclimates such as the LGM are usually held in higher regard, because they are well approximated as an equilibrium state, and with a much larger forcing/temperature signal.
Clearly, the causes of climate change over the last millennium have very little to do with attribution of modern warming, or for future prediction. Attribution is very much like playing detective, and you can’t say someone is innocent of arson just because fires have happened naturally due to lightning in the past. This post seems too obvious to require justification, and I don’t see how the IPCC says otherwise.
This point**
“The problem is also known to apply primarily to only a particular subset of tree rings”
Is that known? Really? One more reason to discard recent work of Michael Mann who shows that the problem extends to even non-dendro proxies:
Unless you are referring to sloppy measurement, data can not be wrong. This data was not discarded because it was wrong, but because the researchers either did not understand or were not willing to accept what it implied.
This is where you are wrong Chris – the data is not demonstrably “wrong”. Can you point to where the data is wrong? No, you can only suggest that by using the data as a proxy for temperature, the resultant proxy record does not match the instrumental record. This does not mean the data is wrong, it means your interpretation of it as a proxy for temperature is wrong. Further, since no-one appears to be able to show convincingly exactly why and how this divergence occurs, this taints the entire record – at least in terms of it’s use as a proxy for temperature.
Craig –
For mush less academic reasons, I agree that tree rings are not good proxies for temperature, since tree rings are also used as proxies for precipitation. I can’t see how they can be proxies for both, especially in the pre-1900 (pre-thermometer) period.
In reading your referenced paper, this stood out:
It seems terribly unlikely that the various positive responder tree species would all have their divergence threshold within the same narrow temperature/anomaly range.
Also, this:
How is this not cheating or fudging? To pick only those sites? Doesn’t that give an artificially high – and misleading – responding rate?
In addition, I agree with this:
But this means there should be less divergence. And the fact that a couple of more recent studies show less divergence should not sway the discussion unless those same studies both go back to the early 1900 period, plus show some convincing argument is made as to why previous studies have wrong methodology.
Chris, the data isn’t “wrong”. That’s a Wonderland-worthy inversion of the meanings of the words.
The data could be wrong if it was gathered improperly…if, for instance, the rings were mismeasured. But no one has ever claimed that.
The data is what the data is. What can be WRONG is the theory that tree-ring data accurately and predictably shows what the world temp is.
“The problem is also known to apply primarily to only a particular subset of tree rings”…if true, it means NONE of the data from that this set of tree-rings should have been on the graph in the first place.
Understand that if you are accurately portraying their determinations, then everything below follows logically from BELIEVING them. You don’t have to believe me; this is what follows if that set of tree-rings is known to be an exception to the theory.
The graph was intended to show that multiple proxies all showed the same story. So, if this subset of tree-rings is known to be an exception to the theory, then the deceit was including the portion that was known to have ACCIDENTALLY told the same story, as if it were further evidence of the agreement of the proxies.
From the climategate letters, I think this is actually a pretty accurate description of what actually happened. Trenberth wanted this proxy on the graph, but it was problematic for the story…so they “hid” the decline. But, if what you’re saying is true, it was deceitful to show any portion of that data on the graph.
Counsel has no further questions for the witness. Further impeachment of his credibility isn’t necessary.
Oh, really.
So the Copenhagen Diagnosis wasn’t timed to set up the Copenhagen conference?
No Colose is NOT a little biased; He is VERY biased. A frequent contributer to RC.
which of course makes me “biased!”
Given what Mann says about the purpose of RC, that certainly is a logical conclusion.
I have no affiliation with RC other than a commenter (and guest contributed a few times), and on several occasions have expressed disagreement with an RC author. I’ve done the same at various other blogs, including Judith Curry’s (where I’ve also done a guest post).
Part of the good thing of an atmospheric science education is that you can actually judge other blog comments on issues you have been trained in without being spoon fed information on stuff you aren’t in a position to judge. I say that without comment or implications about this site….
Why not release all your emails to RC?
Chris you make a claim about your ability to judge without being spoonfed information and yet do not acknowledge what was stated in AR4 9 and 10 with respect to models and paleo reconstructions, ignoring about 25 pages of AR4 and how it was attributed. Your detective story is just that a story. It is not what is stated in the chapters. I am not scoring points except as that you will not read, or will not comprehend. The AR4 is quite specific in how the agreement of paleo and models with and without CO2 forcing raised their confidence in their conclusions. In fact the ability to do centenial and even multidecadal model runs depend on the efficacy of the paleo work per AR4. Your statement is nonsense wrt AR4. The emails already put out on the net wrt to models are already showing that this was discussed in their emails as part of AR4.
No, the content of your comments such as this one do.
Why suspicious? It’s totally obvious the timing is deliberate. Why shouldn’t it be?
Precisely. This was intended to have an impact. And water is still wet.
maybe so, maybe not. there’s usually some sort of climate event on the horizon. Why wouldnt it have been released before Cancun?
Tough. Afraid it might slow down the funding pipeline?
No. I don’t think NSF or people interested in science really care much about the conspiracies. I’m quote sure my funding over the next two years will be just fine, regardless of what a few scientists have said in their personal e-mails.
Then why are you bringing up the timing? Is exposure of gaming the system supposed to only happen when no climate conferences are scheduled? What is the proper timing to show the system is being gamed? Glad your funding is sound by the way. I am sure you will keep working “for the cause”.
Your funding won’t be safe, post 2012 election. Sorry.
We can hope.
It’s a lot harder to earn a living when you have to produce more than a consensus.
4241.txt
“The [[synthetic]] reconstructions show a ‘hockey-stick’ trend. I guess this is precisely the phenomenon that MacIntyre has been going on about. ….. It is certainly worrying…”
1680.txt
date: Wed, 29 Aug 2007 12:03:05 -0400
from: “Michael E. Mann”
subject: Re: Something not to pass on
to: Phil Jones
Phil,
I would not respond to this. They will misrepresent and take out of
context anything you give them. This is a set up. They will certainly
publish this, and will ignore any evidence to the contrary that you
provide. s They are going after Wei-Chyung because he’s U.S. and there
is a higher threshold for establishing libel. Nonetheless, he should
consider filing a defamation lawsuit, perhaps you too.
I have been talking w/ folks in the states about finding an
investigative journalist to investigate and expose McIntyre, and his
thusfar unexplored connections with fossil fuel interests.Perhaps the
same needs to be done w/ this Keenan guy.
I believe that the only way to stop these people is by exposing them and
discrediting them.
Do you mind if I send this on to Gavin Schmidt (w/ a request to respect
the confidentiality with which you have provided it) for his additional
advice/thoughts? He usually has thoughtful insights wiith respect to
such matters,
mike
C’mon Steve, save them the trouble – which gas brand do you fill up with?
Looks like the person claiming to be “RC” (who posted in August) was right.
Indeed, there was no deal made.
4868.txt
“I think this is an abuse of McIntyre’s position as an IPCC reviewer…”
“Phil has agreed to forward these messages to Susan Solomon and Michael Manning…”
WRT attempting to review the papers he’s been asked to review by means of seeing supporting data and methods.
From what I read, all Steve M was doing was suggesting that JGR make d’Arrigo comply with the journal’s requirements. He was not arguing that the paper not be published, only that the authors be required to follow the rules.
This, as opposed to the Team wanting to redefine peer review itself (Climategate I).
Actually, it DOES look like Steve may have overstepped, but it was done simply in the interests of getting data archived in a way that those wishing to follow up on the work could find the data.
It always amazes me how many times Steve is able to point to specific sites and work out which are the correct/applicable ones being used. Certainly, most of the people following up on d’Arrigo’s work would have difficulty replicating.
Steve Mosher,
Why the drive-by obscurantism? Do you have a point to make?
Mosher did post something and I remember it and have waited for it. He even mentioned the other guy when he did.
That’s just the way of the Mosh, may the Schwartz be with him.
Matt,
I read a comment. It was obscure. I put 2 and 2 together and figured that there was another release coming. Call it speculation. call it esp. call it luck. I’ll never know. Im ok with that.
Steven –
Yeah, I also knew there was another shoe to drop. I just thought that whoever did the original release would have held some back in reserve. I’ve been wondering when it would come. At the 2nd anniversary, why not?
Very curious timing.
At the same time FOIA was releasing this shoe, I was hitting a major milestone towards releasing mine.
Mine is still under NDA, and will take a lot longer to land, but it’s a lot bigger. The events were roughly equivalent.
“I have been talking w/ folks in the states about finding an
investigative journalist to investigate and expose McIntyre, and his
thusfar unexplored connections with fossil fuel interests.Perhaps the
same needs to be done w/ this Keenan guy.
I believe that the only way to stop these people is by exposing them and
discrediting them.”
Wonder how Steve will react to the fact Mann looked into hiring people to destroy him?
Mann has to pay his own legal bills first. He is challenging the Virginia FOIA.
You can bet that his costs are being covered.
My guess is that more than one journalist has tried to do that for free.
Email 3021:
…..
I’m sure you are aware of the drivel posted on climateaudit –
…..
I have posted there under the name of thefordprefect. For a year or so.
…..
Who wrote that one? I always wonder who TFP was.
The same person who made an FOI request for Paul Dennis’s emails.
Is this he, as such a request was made and is on Google:
Dear Mr Tuppen,
Please find attached a letter acknowledging your appeal received 24 August 2011 of our refusal notice also sent on 23 August 2011.
The letter also contains further information regarding the handling of this request under the Freedom of Information Act 2000.
We will be in contact with you further in due course.
Kind regards,
Bob Heath
(On behalf of Dave Palmer)
Yup. Mike Tuppen is thefordprefect.
For some reason this revelation made me laugh more than all the others.
To get a better insight into what “FOIA” searched for, read, and thought was interesting enough to redact personal chat from, search for:
[[[redacted:
or just
[[[
Interesting e-mails in that little collection. Briffa isn’t going to like 0058.txt
In the context shown, there does not seem to be any reason or need to delete that email. The thread is pretty innocuous.
date: Mon, 10 Aug 2009 14:18:38 +0100
from: Olive Heffernan
subject: Re: Piece for CRU web site
to: “P.Jones@uea.ac.uk”
Hi Phil,
Ok, thanks for clarifying that. When will the url to the agreements go online?
Best
Olive
On 09/08/2009, P.Jones@uea.ac.uk wrote:
> Olive,
> I did send some of the data to a person working
> with Peter Webster at Georgia Tech. The email wasn’t to
> PW, but he was in the CC list. I don’t know how
> McIntyre found out, but I thought this was a personal
> email. This was one of the first times I’d sent
> some data to a fellow scientist who wasn’t at the
> Hadley Centre. As I said I have taken pity on African
> and Asian PhD students who wanted some temperature and precipitation
> data for their country. The email has only gotten me grief,
> so this is another reason for being much less helpful to
> people emailing CRU. This goes against my nature, but
> I’ve been driven to it. You’d better not say this, otherwise
> McIntyre will request the emails where to prove I’ve been
> unhelpful!
>
> I should have just said to the GA person – use GHCN, like I do to
> everyone else.
>
> I also don’t see why I should help people, I don’t want to work
> with and who spend most of their time critisising me.
>
> Years ago I did send much paleo data to McIntyre but have also
> had nothing but criticism on his blog ever since. As I said,
> this criticism on blog sites is not the way to do science.
> If they want to engage, they have to converse in civil tones,
> and if people don’t want to work with them, they have to respect
> that and live with it.
>
> Cheers
> Phil
>
> > Hi Phil,
> >
> > Thanks so much for that. I have one other question: McIntyre claims
> > that you sent data to Peter Webstre at Georgia Tech, but that you
> > would not supply him with the same data. Is that true, and if so, what
> > was the reasoning?
> >
> > Best,
> > Olive
> >
> >
> >
> > On 07/08/2009, Phil Jones wrote:
> >> Here it is. There will be a link to the agreements – which are
> >> attached.
> >> Files won’t go up with these names. You should recognize one of these
> >> – the data agreement between the Met Office and NERC.
> >>
> >> Met Services appear want to be able to sell data commercially.
> >> As I said some has more value than other types of data. Most
> >> put in clauses separating academic from commercial use.
> >>
> >> Cheers
> >>
> >> Phil
> >>
> >>
> >> Prof. Phil Jones
> >> Climatic Research Unit Telephone +44 (0) 1603 592090
> >> School of Environmental Sciences Fax +44 (0) 1603 507784
> >> University of East Anglia
> >> Norwich Email p.jones@uea.ac.uk
> >> NR4 7TJ
> >> UK
> >> —————————————————————————-
> >>
> >>
> >>
> >
>
>
>
Good question, Olive! She never did make it to our meetings to discuss climate openness in London – but I felt her interest was genuine. It seems so.
> >> Met Services appear want to be able to sell data commercially.
> >> As I said some has more value than other types of data. Most
> >> put in clauses separating academic from commercial use.
It appears that MetOff was providing public forecasts that differed from their commercial forecasts. Selling “more real” forecasts to private parties smells like conflict of interest, doesn’t it?
Wilson:
I thought I’d play around with some randomly generated time-series and see if I
could ‘reconstruct’ northern hemisphere temperatures.
[…] The reconstructions clearly show a ‘hockey-stick’ trend. I guess this is
precisely the phenomenon that Macintyre has been going on about.
Link no workee, can’t download, just gives a “404” error.
w.
Alternate link can be found here.
http://www.megaupload.com/?d=ROCGBR37
Thanks, Bob, that works.
I thought I would leave a message of support with our Gav and his friends. They seem to have a problem so I thought I would post it here?
You guys are very quiet at the moment but don’t worry it is obvious that your remarks in the newly released emails have been taken out of context.
Now what type of person normally says their words have been taken out of context?
Of course someone who is completely honest like for instance politicians.
The middle game was Climategate 1 which your mates managed to help you out with we are now into the end game and unfortunately you have no credible pieces on the board.
Three cheers for the “cause”.
Here’s an interesting one:
1680.txt
date: Wed, 29 Aug 2007 12:03:05 -0400
from: “Michael E. Mann”
subject: Re: Something not to pass on
to: Phil Jones
Phil,
I would not respond to this. They will misrepresent and take out of
context anything you give them. This is a set up. They will certainly
publish this, and will ignore any evidence to the contrary that you
provide. s They are going after Wei-Chyung because he’s U.S. and there
is a higher threshold for establishing libel. Nonetheless, he should
consider filing a defamation lawsuit, perhaps you too.
I have been talking w/ folks in the states about finding an
investigative journalist to investigate and expose McIntyre, and his
thusfar unexplored connections with fossil fuel interests.Perhaps the
same needs to be done w/ this Keenan guy.
I believe that the only way to stop these people is by exposing them and
discrediting them.
Do you mind if I send this on to Gavin Schmidt (w/ a request to respect
the confidentiality with which you have provided it) for his additional
advice/thoughts? He usually has thoughtful insights wiith respect to
such matters,
mike
Hahaha – Mann is big on threats of libel suits.
Search “paleoloonies”
http://files.sinwt.ru/download.php?file=25FOIA2011.zip
Error 404 – Not Found!
No longer available?
See Bobs posting above.
w.
I am happy as a clam.
I’ve been wondering if and when the other shoe would drop. Not surprising that it came almost exactly on the anniversary of the original release. Also, just before Durban, similar to the pre-Copenhagen timing of the first release.
Seeing as what is in these, Mann et all must have been quaking in their boots. wondering when this would happen. Even if merely more of the same (I have no idea at this time…), it just bolsters the arguments Craig Loehle makes at 11:33 AM above. Mann quaking in his boots is almost certainly the reason he is fighting tooth and nail the UVA email release.
Imagine the thought of one year ago when nothing new appeared, the Team must have been thinking they had got away with it.
The 2 year gap was a great idea, long enough for the Team to feel less under pressure but close enough that the fire was restoked easily.
Gavin has a new thread up at RC on this.
http://www.realclimate.org/index.php/archives/2011/11/two-year-old-turkey/
It is hard to interpret Gavin’s post as anything but … denial.
Its rather desperately trying to avoid the question of why he was seen as the ‘gone to man ‘ when it came to smearing others . Oddly his actual got a defense , its clear when he cannot control the debate through the use of sink holes or edited, has he does on RC, is actual rubbish at it .
Ed Cook 4509 after publication of our first article:
I have said many times that the reason that the AGW supporters never criticize another AGW supporter’s work is that they might (gasp) return the favor. Ed Cook lays out my claim very clearly
This ranks up there with Phil Jones’ statement to Warwick for the “I Don’t Understand Science” prize. Don’t criticize another scientist’s work, they might actually find fault with your work, and the whole edifice would be in danger.
The mind boggles …
w.
#4230. Dave Palmer
Dave is the man who is supposed to be on the side of the FOI request, but is working as hard as he can to figure out a way to ignore the FOI request. Note that Dave tells Phil to be careful about what he puts in his correspondence.
Phil’s response to being told to be careful about his emails is to delete them.
Typical.
w.
The undoing of Arthur Anderson was that in response to a hint of trouble one of their offices went on a clean-up spree. That undid their credibility and their client list unraveled in weeks. Rats don’t abandon sinking ships as fast. And the deletion of emails in response to FOIA is actually illegal (as if it were enforced).
Email 3791.txt from Tim Osborn seems to be the “wink wink” reply on how to “legally” delete emails:
3791.txt
date: Mon, 8 Dec 2008 19:49:18 -0000 (GMT)
from: “Tim Osborn”
subject: RE: FW: FOI_08-50 ; EIR_08-01
to: “Jones Philip Prof”
Hi Phil!
re. your email to Dave Palmer [which he copied in his response to you and
cc’d to me, Keith & Michael McGarvie, and which has hence already been
multiply copied within the UEA system, and therefore will probably exist
for a number of months and possibly years, and could be released under FOI
if a request is made for it during that time!]… I assume that you didn’t
delete any emails that David Holland has requested (because that would be
illegal) but that instead his request merely prompted you to do a spring
clean of various other emails that hadn’t been requested, as part of your
regular routine of deleting old emails. If that is what you meant, then
it might be a good idea to clarify your previous email to Dave Palmer, to
avoid it being misunderstood. 🙂
The way things seem to be going, I think it best if we discuss all FOI,
EIR, Data Protection requests in person wherever possible, rather than via
email. It’s such a shame that the skeptics’ vexatious use of this
legislation may prevent us from using such an efficient modern technology
as email, but it seems that if we want to have confidential discussions
then we may need to avoid it.
I shall delete this email and those related to it as part of my regular
routine of deleting old emails!
Cheers
Tim
I cannot imagine a company like say Solyandra getting away with this type of tactic. It just reeks of cover-up. Surely a formal written data retention and deletion policy needs to be in place and in force? Moreover, if the emails are stored centrally wouldn’t these exist in perpetuity?
wow,
so when I speculated about this and people criticized me.. ahh what does the addition context
show?
doh! worse in context
Forgive me for asking, but isn’t deleting email subject to FOI against the law here in the UK?
Phil still hasn’t figured out that deleted emails aren’t. This is hilarious.
Oh, goodness. This is the same Phil Jones who told the “inquiries” that no emails had been deleted? Or do I recall that wrong? But didn’t these people know that deleting emails off one’s PC does not delete them from the server?
Which brings up the question if these and the Climategate I emails were taken from the server… Is there any way to tell?
3333. 2003-11-21
______________________________________________________
date: Fri, 21 Nov 2003 12:00:27 +0000
from: Ian Harris
subject: Mann’s Proxies
to: Keith Briffa
Hi Keith,
Phil asked me to access Mann’s MBH98 predictors from the ftp server in Virginia.
Apparently there are 159. However, with no guidance I’m unable to identify a set of 159 data series from the thousands of files in the ‘MBH98’ directory structure.
Furthermore, attempts to download the entire directory hierarchy for examination (it’s only about 20Mb in total) have been fraught with failures – 3 hours on Wednesday afternoon and all this morning, and still incomplete.
If Steve McIntyre’s having the same difficulties, no wonder he’s pissed off!
Anyway I should have a complete file listing this afternoon; we can
look over it then and hopefully identify the predictors.
By the way – what does this have to do with HOLSMEER? *grin*
Cheers
Harry
Yeh Steve, but you had already guessed that. They just didn’t think you were clever enough to decipher their ‘filing system’ he he . well done, well done. Boy did you piss them off too.
Steve –
Based on the fact that this “Harry” here is looking for data files, I am wondering if this is THE Harry of HARRY_READ_ME fame. From the cheerful, open tone, I’d guess it is.
Re: Steve Garcia (Nov 22 20:57),
Ian Harris is indeed ‘Harry Read Me’
http://www.cru.uea.ac.uk/cru/people/
pissed off:
4101, “If Mike said that my calibration procedure is “flawed”, I will be extremely pissed off.”
3890 is quit funny
#3890
date: Thu, 25 Jan 2007 09:51:05 -0700
Ammann to Wigley and Trenberth, cc Jerry Meehl, Jonathan Overpeck, Phil Jones
“… Trying to interpret a priority list from my personal feeling of this guy. I do this based on a google-cash because his site is down or something. So here is my hunch/speculation. I’ll see if I can get the document somewhere but I’m not very optimistic about this. By the way,
I’m also going to forward this to Susan an few others that might have heard the rumor. Before I do send it to Susan, you might chip in on
this list for “internal and IPCC use”:
(…)
– sneaking in papers into AR4 that were past deadline”
Ah yes, I remember those late nights “strategically” bringing Climate Audit down…just to annoy Realclimateers.
I keep saying it sorry steve but, I can’t believe what I’m reading in the emails. We all had our suspicions after climategate but they were easily batted away by whitewash but these ar something else again. And what about the remaining “220.000”. That is an unbelievable amount of traffic.
This is all down to you Steve and also Watts. Thanks.
That’s exactly what I keep thinking – I just can’t believe this. I’m waiting to see how my warmist friends mange to spin/ignore these new emails.
Interesting bit of history in Tim Osborn comments on Mann’s response to MM03 (3702.txt) – he seems to have spotted a number of the issues too.
“(b) The mention of ftp sites and excel files is contradicted by their email record on their website, which shows no mention of excel files (they say an ASCII file was sent) and also no record that they knew the ftp address.
This doesn’t matter really, since the reason for them using a corrupted data file is not relevant – the relevant thing is that it was corrupt and had you been involved in reviewing the paper then it could have been found prior to publication. But they will use the email record if the ftp sites and excel files are mentioned.”
“The time series of PC1 you sent is certainly different from your standard one – but on the other hand I’d hardly say you “get a similar result” to them, the time series look very different (see their fig 6d). So the dismal RE applies only to your calculation, not to their reconstruction.”
Wow, they had “hide the decline” t-shirts already 11 years ago 😉 (see 3880)
1897. Dec 2008.
Steve:
Presumably “an ongoing record management retention schedule” would be formerly documented and an actual schedule exist. If so, can the deletion of emails not directly relevant to an FOI request but more generally associated with the same subject matter and deleted to thwart any subsequent FOI requests be seen as an offense?
bernie: “Presumably “an ongoing record management retention schedule” would be formerly documented and an actual schedule exist.”
Yes, but in this case we don’t even have to go there. Jones is quite clear in the emails that he is deleting stuff *in response to* the FOI requests, which is a violation. Hence, Dave’s warning above.
Most universities do not have a formal retention/destruction schedule for profs’ emails, research, or reports. They are independent entities.
Did anybody try the “All.7z” compressed file. It contains 220247 files which are password protected. Is the password known? Do we have hackers who can figure it out?
I have a few computers running a brute-force attack on it right now. A hopeless task but I’ll give them a month or two and see if anything happens.
Let me know if you find the key.
You must be feeling lucky.
🙂
code and data please. haha
Yeah, time to fire up the botnet.
Mosh, have you made the same “code and data” request to Richard Muller, and if so, what was the reply?
Thanks,
w.
Hmm, maybe pw = h1d3th3d3cl1n3
h4 h4! 🙂
try “FOIA”!
Of course. The answer was essentially similar to the one anthony gave when I asked him for his data prior to acceptance and publication of his paper.
I play no favorites.
The release of code and data for this is actually ahead of the kind of release that others make before acceptance and publication, but as with Anthony I await the final publication. That after all is when the paper actually has a scientific claim.
Thanks, Mosh. As far as I know, Muller has still not released the code for three of the four papers, and he has not released the raw data for any of them. As a result, it’s not clear what you mean when you say:
Since the papers were released to great fanfare and a full-court PR press complete with media interviews, and despite that most of the code and the raw data have not been released, what do you mean by “actually ahead”? In my world that’s “actually behind”, to “pre-publish” the papers like that but conceal the data and code.
w.
What are you using to generate and test passwords?
Let’s get some coder to write a distributed brute force attack and write a nice screen saver. It could show flying toasters popping out index cards saying: Cracking Climategate 3.0. I bet a LOT of us with download the screen saver to crack it.
1111
Why would the DOE enter into such an agreement with Jones in 2007 after issues had been raised?
It is probably a good-ol-boy wink-wink with a buddy of Jones in DOE–good luck finding any paper trail.
Isn’t US DOE is subject to FOI requests? Maybe they still have the correspondence from Jones.
3755.txt
“It is really important that you don’t just copy or reproduce any bits because it is not my proposal and it would be a court case in theory if a similarity was noticed.”
“but for GODS SAKE please respect the sensitivity here and destroy the file immediately when finished and please do not tell ANYBODY I sent this. Cheers Keith”
Oh dear. Oops!
A brief note on authenticity: 4003.txt looks esentially identical to a copy of an email I was sent by Dave Palmer in response to a data protection request (my original copy was partially redacted and formatted, not plain text).
David Appell had interesting take – see here.
Can’t say I agree with his analysis; The red are certainly red but the rest seem to be dependent on the baggage you are carrying.
1939.txt,
Thorne to Jones, Feb 2, 2005, referring to the AR4 draft on tropospheric trends:
Jones to Thorne, Feb 4, 2005:
AR4 Chapter 3 Executive Summary as published:
The following from regarding dendrochronology is amusing, in particular the concluding statement: “It would be a major step forward if dendrochronology could embrace the scientific method.”
date: Wed, 13 Nov 2002 09:55:08 -0500
from: Ed Cook
subject: Re: Fwd: History and trees
to: ITRDBFOR@LISTSERV.ARIZONA.EDU
Rod’s comments are remarkably ignorant and insulting. I suggest that
he stick to what he knows best and not claim that he understands
dendrochronology and its methods. That way he would not sound so
stupid. To suggest that dendrochronology does not embrace the
scientific method and is as biased as he claims verges on libel. Of
course, Rod has the right to his opinion. It is just a shame that he
chooses to expose his ignorance of dendrochronology in such a
negative way.
>To the Editor, New York Times
>
>Further to the message below, I want to assure you that not everyone agrees
>with the representations by David Lawrence. As a tree physiologist who has
>devoted his career to understanding how trees make wood, I have made
>sufficient observations on tree rings and cambial growth to know that
>dendrochronology is not at all an exact science. Indeed, its activities
>include subjective interpretations of what does and what does not
>constitute an annual ring, statistical manipulation of data to fulfill
>subjective expectations, and discarding of perfectly good data sets when
>they contradict other data sets that have already been accepted. Such
>massaging of data cannot by any stretch of the imagination be considered
>science; it merely demonstrates a total lack of rigor attending so-called
>dendrochronology “research”.
>
>I would add that it is the exceptionally rare dendrochronologist who has
>ever shown any inclination to understand the fundamental biology of wood
>formation, either as regulated intrinsically or influenced by extrinsic
>factors. The science of tree physiology will readily admit that our
>understanding of how trees make wood remains at quite a rudimentary state
>(despite several centuries of research). On the other hand, there are many
>hundreds, if not thousands, of publications by dendrochronologists
>implicitly claiming that they do understand the biology of wood formation,
>as they have used their data to imagine when past regimes of water,
>temperature, pollutants, CO2, soil nutrients, and so forth existed. Note
>that all of the counts and measurements on tree rings in the world cannot
>substantiate anything unequivocally; they are merely observations. It
>would be a major step forward if dendrochronology could embrace the
>scientific method.
>
>sincerely,
>RA Savidge, PhD
>Professor, Tree Physiology/Biochemistry
>Forestry & Environmental Management
>University of New Brunswick
>Fredericton, NB E3B 6C2
I can’t speak to the dendrochronology, but can say with confidence that dendrothermometry does not follow the scientific method. Dendrothermometry is pure correlation = causation. It’s a total crock.
What I find comical is the way the IPCC left hand doesn’t know what the right is doing. You wouldn’t think they can simultaneously claim that higher growth comes from higher temperature in the dendro sections and then argue in the impacts sections that high temperature is automatically bad for agriculture. But they manage to do exactly that…
This is the same Ed Cook who admitted in the first dossier that we know “f*** all” about climate variability >100 years based on dendro. He sounds a bit conflicted.
Whoops, the dendro comment is from 3219.txt
From Bishop Hill:
“I cracked the passcode for the rest of the emails:
“4x00y312dym!231@4″
Enjoy!
Nov 22, 2011 at 9:49 PM | Unregistered Commenterh4x0r”
Please post whether or not it works.
Tks.
nope
The 7zip archiver in which this is stored uses 256 bit AES encryption.
“FOIA” chose this most likely because there are no effective tools for 7zip, while there seem to be many for standard .zip and .RAR files.
From their website:
“7-Zip also supports encryption with AES-256 algorithm. This algorithm uses cipher key with length of 256 bits. To create that key 7-Zip uses derivation function based on SHA-256 hash algorithm. A key derivation function produces a derived key from text password defined by user. For increasing the cost of exhaustive search for passwords 7-Zip uses big number of iterations to produce cipher key from text password.”
The password can be 2047 or 8191 characters long, depending on your operating system.
Doubtful this password will be cracked anytime soon, maybe DoD could do it.
“FOIA” is holding this in reserve, making it known that it is there, ready to pull the firing pin. I expect we’ll see it sooner than later as the reaction so far from RC and the Team is continued arrogance.
“The first key-recovery attacks on full AES due to Andrey Bogdanov, Dmitry Khovratovich, and Christian Rechberger were published in 2011.[22] The attack is based on bicliques and is faster than brute force by a factor of about four. The key is recovered from AES-128 in 2^126.1 operations. For AES-192 and AES-256, 2^189.7 and 2^254.4 operations are needed, respectively.”
Probably not in my lifetime.
“I am reminded of the story of the great French Marshal Lyautey, who once asked his gardener to plant a tree. The gardener objected that the tree was slow-growing and would not reach maturity for a hundred years. The Marshal replied, “In that case, there is no time to lose, plant it this afternoon.”
– John F. Kennedy
Three of the 5000+ emails are by me. (I of course forgot to write down the numbers.) These three are genuine.
Re: Richard S.J. Tol (Nov 22 18:18),
You’re also mentioned in an email asking that you not be a referee for a grant proposal.
Loook at this gem
2440> Jones:
I’ve been told that IPCC is above national FOI Acts. One way to cover yourself
and all those working in AR5 would be to delete all emails at the end of the
process.
Mann:
the important thing is to make sure they’re loosing the PR battle. That’s what
the site [Real Climate] is about.
Steve already called Gavin Schmidt a “NASA blogger”. I think he could call his comrade Mann “an IPCC PR manager”. 🙂
Drudgereport has link to Guardian whitewash. Just out.
Steve
Richard Blacks spin and propaganda can be viewed here.
http://www.bbc.co.uk/news/science-environment-15846886#dna-comments
Absolutely disgraceful but no surprise there.
In his evidence to the second ClimateGate enquiry by the HoC Select Committee on Science and Technology Professor Acton gave the following reply:
Q94 Graham Stringer: Right. I shall look at that. Professor Acton, are you satisfied that these questions weren’t asked, that people in your university were sending out e-mails suggesting that e-mails should be deleted and that hasn’t been investigated?
Professor Edward Acton: It has been investigated. I have asked them and they have assured me that they have never knowingly deleted e-mails subject to a request.
Thanks to FOIA2011 we now have access to 2368
Dave,
Do I understand it correctly – if he doesn’t pay the £10 we don’t have to respond?
With the earlier FOI requests re David Holland, I wasted a part of a day deleting numerous emails and exchanges with almost all the skeptics. So I have virtually nothing. I even deleted the email that I inadvertently sent. There might be some bits of pieces of paper, but I’m not wasting my time
going through these.
Cheers
Phil
This seems to imply that someone is being economical with the truth
http://climategate2011.blogspot.com/2011/11/1897txt.html
Aye, happen it’s a tough life in Norwich deleting email and making up temperature trends, but someone has to save the planet.
http://climategate2011.blogspot.com/2011/11/3021txt.html
Ford, I think that this is called ‘leading the witness…!
Not being a computer wiz would this work on unzipping the last lot? http://file.org/extension/7z It says its a free to try file. I wonder if the person who released these wasn’t really concerned about conferences but just released them when they wanted to. The first batch opened the door a crack and is hoping this second batch will fully open the door.
I read the originals so long ago I can’t recall if these are repeats…
#76
“Of course, Trenberth will go nuts because his name is on the publications associated with the data.”
#37
“Advocating a piece of work of known error is not in the best interest of climate research community in general and RealClimate in particular.” Shaopeng Huang
2713.txt:
David Parker to Phil Jones, regarding Steve’s query about how they ascertained that there was no residual autocorrelation in their trend calculations for the IPCC Report, April 2007
Nice to see the world’s top experts at work. The DW statistic is invalid when applied to a model that has had an AR1 correction applied to it. You need to use an alternative test score since the DW statistic is strongly biased towards 2 (i.e. indicating no AR is present). This is in any textbook treatment of the subject.
The textbooks of the future will have exercises full of this stuff.
Only if they written by Ross I suspect.
Not long ago I wrote ( http://rankexploits.com/musings/2011/response-to-keenan/#comment-85060 ):
I don’t know how this was calculated, but it seems to be a claim that “AR1+ linear trend” is valid. This is at odds with the Beran’s example discussed in another thread. Thus, it would interesting to verify this result.
Beran’s textbook: http://www.amazon.com/Statistics-Long-Memory-Processes-Monographs-Probability/dp/0412049015
UC, I have Beran’s text, and find it to be extremely good. For some comments related to that, see this post:
http://www.bishop-hill.net/blog/2011/6/6/koutsoyiannis-2011.html
Please note that fractional Gaussian noise is called “Hurst-Kolmogorov” there.
(About RankExploits, I gave up trying to find a way to communicate, and so prefer to not comment there.)
This is something simpler than the discussion at Lucia’s blog (pure statistics 🙂 ),
2713
vs.
Beran
The data set is not exactly the same, but there is a clear disagreement. Who is right?
Beran is wrong: he is using seasonal data (in this case, monthly), and not accounting for that. It would be the same if we were using non-fractional ARIMA; we would have to either aggregate the data (into annual values) or use SARIMA.
David Parker, and Phil Jones, and Ross McKitrick, are not doing things properly either. The DW statistic is based on assumptions that virtually never truly hold with real data. It is a relic from an earlier era, and deserves no place in modern statistical analysis. Use AICc etc.
Presume you mean AIC or Akaike Information Criterion, without the trailing c. I remember the hedge fund that followed your advice. Those were good years.
AICc:
http://en.wikipedia.org/wiki/Akaike_information_criterion#AICc
Also note the “etc.”.
Cool, thanks. No doubt I implemented the correction in C++ in 95 but that didn’t make me aware of it! I did at once think AIC and BIC when you first spoke. 24 characters to go?
Gents
Max Beran has been commenting on another thread. How about asking him directly?
to make it clear beran has been commenting on the “Phil stumped” thread
Do you mean that the anomaly method used by Jones makes the conclusion invalid?
On page 11 Beran shows that AIC doesn’t work well for the Nile River data. For GMT we’ll have to wait and see.
No, seasonality is essentially unrelated to the anomaly method.
Your interpretation of Beran’s page 11 is incorrect. For more on AIC, the standard reference is Model Selection and Multimodel Inference by Burnham & Anderson (2002).
Can you briefly explain? I’m here to learn.
BTW, Thomson, “Jackknifing Multitaper Spectrum Estimates”, IEEE Signal Processing Magazine 2007 about the Nile data:
I need to read that as well. To get back to the topic, Thomson seems to mention you (+ Exxon and oil and coal 🙂 ), in the mails (3474.txt).
Doug. its called a comments section. you go to lucias blog and use the comments section
at the bottom of the page. there is a box.
you enter your words.
you hit post
With comments like that, you will not achieve your objectives.
Doug/Steven Mosher,
Examining the database for my blog, “Douglas J. Keenan” has posted 3 times. The comments are on a post from 2009:
http://rankexploits.com/musings/2009/petr-chylek/#comment-27591 which discusses /Wei-Chyung Wang: Jones et al. [Nature, 1990] and Wang et al. [GRL, 1990].’
The other two are
http://rankexploits.com/musings/2011/climograms-hurst-no-hurst-trend-no-trend/#comment-76808
http://rankexploits.com/musings/2011/climograms-hurst-no-hurst-trend-no-trend/#comment-76921
Which appear on a post about Hurst phenomena.
Unless he has posted other comments under a different name, I think those are the only comments by Keenan at my blog.
I leave it to others to read his comments and to judge the cause of Doug’s communication difficulties and also to decide whether Doug has tried very hard to discuss his ideas at my blog.
As for me, it’s long been my impression that Doug would rather send me long winded private emails. My impression is that he wishes to spend a lot of time privately conveying his thoughts while pretty much ignoring questions, points or any substantive criticism what so ever. The communication goes nowhere. But that’s just my view.
To spare myself the time sink involved in these private emails, I have made it clear that I am willing to discuss whatever points he is willing to make in public. Having switched to my policy of responding to him in public rather than private, I responded to the email he sent to me privately, and which, on my prompting, he posted in comments at TAV. My response is here:
http://rankexploits.com/musings/2011/response-to-keenan/
I should also note that while in comments at TAV, Doug decreed Judy and Tamino’s criticisms of his points invalid. I requested post his evaluation these criticism in public. As far as I can tell, he has declined to engage their criticisms in public either.
Steve Mc–Sorry for the OT. But Doug’s complaint is bleeding over to my blog.
Look like I did achieve my objective.
snip –
I thought you guys might be interested in this one…..#5267
Cheers! 🙂
Wow! I remember after Steve revealed the “mole” going through some of those files and folders on that server, and then they were gone the next day. Now we know who ordered their erasure.
I think its fair to say Phil is serial abuser of ‘delete ‘ the e-mail approach , its been clear for long time he does not like the FOI and considers he can ignore it at will , remember two things , one he planned to avoid FOI requests before he got any ,two his claimed his ‘unaware’ of FOI requirements despite the fact he done FOI training and the university has FOI policy which he signed up to.
very juicy
especially all the NGO addresses, so much for impartiality
1566
I dont recall forwarding such criticism. Not the sort of thing that I normally do. The second point is interesting. A few weeks later (after MM2003) was published, Mann deleted the version of data that I had been given access to and a new version materialized – with Mann saying that it had been there all along and I should have located it. (Though as readers may recall, I had asked him about this topic before.) It is my very strong belief that the data that was made public in November 2003 was in a non-public directory in October and that mann’s statements on this point at the time – and, more recent;y, to the Penn State Investigation Committee – were untrue.
Re Jean S and UC’s discussion of the Mannkovitch bodge: – 1283 –
The paper certainly did not ‘say clearly” that the data had to be bodged to get a “skillfully cross-validated” reconstruction. See the Mannkovitch post.
Steve McIntyre (Nov 22 20:00),
Interesting, he is essentially admitting mining for good verification statistics. As we now know, it took at least three attempts to find such a “correction”. IMO, that is clearly “manipulating data to serve a given predetermined outcome“.
Mining for good verification statistics – that’s real expertise for you.
I believe someone would say that is a “foolish” thing to do and he would never do it (blank stare as to what you are saying)
Steve McIntyre (Nov 22 20:00),
this was already in ClimateGate 1.0 files, but I had not noticed it: – 3584 –
Reasonably trusting a trend that is substantially different for every “fix” he tried? I’d sure like to know more about those “uncertainties in”!
Someone should teach these folks how to spell “lose”. And explain the difference with “loose”. Lose is what they do with data they don’t want anyone to see. Loose is what they play with the facts and their quality control.
Every time I read this basic error, I feel like ‘loosing’ my lunch. And that would be a real loss.
You’re such a looser.
1666. the start of the fabricated story about the “Excel” request (which even CRU recognized as a concoction):
Look at 4742.txt. It’s written 48 hours after MM2003 came out and contains a surreal re-telling of the excel spreadsheet tale.
As we’ve pointed out many times, we never asked for an excel spreadsheet and we were never given one. I read the text file into a spreadsheet so I could colour code the columns. The data file we used was called pcproxy.txt. I wonder if Rutherford actually told Mann he prepared the excel spreadsheet, and why he’d have said that?
I have to say, the whole Excel spreadsheet issue is one of the most dumbfounding things I’ve seen. I cannot imagine any way Michael Mann would have genuinely believed what he says about what you guys supposedly did, nor that Scott Rutherford would have confirmed anything like it, yet 4742 clearly indicates both. The only explanation that makes sense to me is Mann is simply lying in an extremely blatant manner, but that is hard to believe.
It seems like he truly believes what he says, but I don’t see how that would be possible.
Although the Penn State “investigation” committee didn’t actually investigate the Climategate issues, the “wrong” data was asked about and, amazingly, Mann gave the same false answer to the Investigation COmmitee. (Needless to say, Penn State didn’t bother checking whether the answer was true.)
I remember being shocked when I saw that. I couldn’t believe Michael Mann would repeat that story, much less that anyone would accept it. It’s flabbergasting to think “investigations” like that are enough to “clear” Mann.
I have to wonder, is there anything he could say which would actually provoke outrage?
It takes a lot to provoke outrage at Penn State
I wonder if Rutherford was in a panic because he’d used pcproxy.txt for another paper already and he told Mann he prepared it specially for us to cover things up, and Mann ran with that claim. Yet why would he still be repeating it 7 years later, to the Penn State investigators?
And why would Penn State investigators accept such an obviously false answer and rush to exonerate one of their faculty members? Well, never mind.
Could be. I’m pretty sure that pcproxy was used in the version of Rutherford et al 2005 that they were working on at the time. I saw a reference to this file in a contemporary graphic – see an early chronology on this.
Brandon,
Lawyers see it all the time. Parties to a lawsuit start with the notion that they know themselves to be good people with proper morals and motives. They really deserve to win. As they become aware of the facts and law relevant to their case, they review and replay their memories and the key parts of their testimony. If the case takes years, by the time they testify at trial they are far more certain of what happened than they were earlier. And their testimony can be far more favorable to their own cause. And they believe every word of it to be true.
My purely speculative take based on his emails, letters to the editor, and his quotes in stories is that Mann’s starting point is that he ‘knows’ he is right and good and moral. And is convinced that he is on a quest to save the world and engaged in a mighty struggle against evil people. It would not surprise me a bit that he truly believes everything he says, even if some of it were to contradict other things he says or clearly established fact.
“No reasonability” eh. Sounds about right.
Some of these guys are pretty illiterate, that’s for sure. Worrying, really!
421. After telling everyone that Rutherford had prepared an Excel file at our request, we wrote back to Mann telling him that the file which we had downloaded from the ftp location to which we had been directed was dated long before our request. Mann asks Rutherford to explain. (I wonder what the answer was.)
Wigley 2003:
When it is under FOIA, it does.
But the data itself wasn’t/isn’t CRU’s anyway. They themselves didn’t earn it the hard way – they got it from others.
Pathetic hubris.
At the same time, it was Wigley who did bring some moments of sanity to the Team, so he isn’t a complete jerk. He did try to get them to change the way they were doing some things.
Maybe someone can check back on what East Anglia said about Boulton not havinghad contact with UEA for nearly 2 decades. here’s an email from Phil Jones to Boulton in 2004. I’ve placed a pdf with all Boulton references online here. http://www.climateaudit.info/correspondence/climategate2/boulton.pdf. (This was produced in a few lines). I’ve placed an R version of the emails online in the same directory as cru.tab.
Also
Steve,
This posting:
quotes Boulton
‘Prof Boulton said he had been open about having worked at the School of Environmental Sciences at UEA between 1968 and 1986. “Since then, I have had no professional contact with the University of East Anglia or the Climatic Research Unit,” he said. ‘
You have a link there to the Scotsman for that quote, but the link doesn’t work anymore.
Steve,
Further, your posting of 2010/02/15 explores the connection in some detail.
I can’t find a record yet of UAE commenting on their relationship with Boulton. I’ll look some more.
Guess that’s true. Boulton has had no professional contact with UEA/CRU, only unprofessional contact.
40. 2000-04-28
______________________________________________________
date: Fri, 28 Apr 2000 08:56:29 +0100
from: Trevor Davies
subject: Re: R Director
to: Mike Hulme
Mike,
It would be worth consulting Peter (in US at the moment) & Martin. I would
also suggest consulting Tim O’R who knows many in big business & Andy
Watson who is well-connected. Characteristics are a good starting point. We
may have to consider special arrangements , but I would regard based in
London as a real downer – OK for say 2 days per week.
Names – Gell-man is how it’s spelt, I think. Grubb – I am very anti (but I
wonder if the RCs have him in mind? Hunt – I am less anti after the TSUNAMI
meeting – he is making a genuine attempt to ‘open out’. Additional name –
Bob Watson. Another long-shot (& I find myself surprised that I am
suggesting this) – Geoff Boulton (I’ll update on what he’s been up to).
I’ll sound out Martin. Tim etc in time for your return on Tuesday.
Trevor
A 2006 meeting between Boulton, UEA reps and the Met Office regarding briefing Stern:
A snapshot among other things of how non-Western governments like that of India are made to ‘buy in’ to the consensus, giving rise to the astonishment of Richard Black yesterday that our friend FOIA dared to intimate in his README that the trillions diverted by climate politicking may not be ideal for the poorest. Indeed, you can see the concern for those without – or at least those who feel they do not yet have enough – oozing out of every sentence here. Leading to what the outsider Nigel Lawson generously described in the Wall Street Journal in October 2009 as:
‘Geoff’ showing his gift for impartiality throughout, as he did four years later on being chosen by Muir Russell to investigate UEA after Climategate 1, without seeing fit to mention such prior involvement seeking to steer a major report on climate change commissioned by the highest level of the British government, working with UEA experts. Presumably it was Clare Goodess (not Goddess) of CRU and Andrew Watkinson of the Tyndall Centre around the table with the great man on this occasion. Slingo herself later being wheeled out as an independent expert to say all was well with CRU and consensus science to the Commons select committee investigating Climategate 1 in March 2010, with no visible grasp of detailed concerns raised by the emails, familiar to CA readers, about CRU paleo temperature reconstructions and their use by the IPCC. Tiny world.
Sari Kovats has only just completed her PhD. I wonder why she was chosen to brief Stern in 2006.
Because she had the right opinion Bish. Expertise is no substitute for that.
The apparently random order is driving me crazy.
I have renamed all the released emails with date stamps. (One is a duplicate of another: 4698 == 5053.) Anybody else want a copy?
When the initial release came out, Bender, myself, and others thought it was a hoax based on the ‘hide the decline’ e-mail.
Noone has given a reaction like that, but I am wondering if this were the original release, would any of these e-mails have led you to believe it’s a hoax?
I like email “451”….
from: Stephen H Schneider
subject: Re: [Fwd: data request]
to: santer1@xxx.govv
“Thanks” Ben for this, hi all and happy new year. I had a similar experience–but not FOIA since we at Climatic Change are a private institution–with Stephen McIntyre demanding that I have the Mann et al cohort publish all their computer codes for papers published in Climatic Change. I put the question to the editorial board who debated it for weeks. The vast majority opinion was that scientists should give enough information on their data sources and methods so others who are scientifically capable can do their own brand of replication work, but that this does not extend to personal computer codes with all their undocumented sub routines etc. It would be odious requirement to have scientists document every line of code so outsiders could then just apply them instantly. Not only is this an intellectual property issue, but it would dramatically reduce our productivity since we are not in the business of producing software products for general consumption and have no resources to do so. The NSF, which funded the studies I published, concurred–so that ended that issue with Climatic Change at the time a few years ago.
Read the rest — it gets better…
On “The Consensus”…
from: “Malcolm Hughes”
subject: Re: Your letter to Science
to: Edward Cook , “Michael E. Mann”
Dear Ed and Mike and others,
All of our attempts, so far, to estimate hemisphere-scale
temperatures for the period around 1000 years ago are
based on far fewer data than any of us would like. None
of the datasets used so far has anything like the
geographical distribution that experience with recent
centuries indicates we need, and no-one has yet found a
convincing way of validating the lower-frequency
components of them against independent data. As Ed
wrote, in the tree-ring records that form the backbone of
most of the published estimates, the problem of poor
replication near the beginnings of records is particularly
acute, and ubiquitous. I would suggest that this problem
probably cuts in closer to 1600 than 1400 in the several
published series. Therefore, I accept that everything we
are doing is preliminary, and should be treated with
considerable caution. I differ from Ed, and his co-authors,
in believing that these problems have a special
significance for the particular implementation of RCS
they used, in the light of one of their conclusions that
depends heavily on that implementation.
As I understand what Ed, Keith and Hal Fritts have
written at various times about RCS, and from my own
limited experience with the method, it is extremely
important to have strong replication, and I don’t see 50-70
samples probably from 25-35 trees as a big sample. For
reference, most chronologies used in dendroclimatology
are based on 10-40 trees, that is 20-80 samples at 2 cores
per tree for a single “site”, usually a few hectares.
etc.
Must be read in full to be appreciated…
It gets better, really…
That was “0209.txt” sorry about that.
I’ve just had a look in my local newsagents to get an idea of how the London papers have treated this. Only The Independent has a link on the front page – prominent at the top centre
with a picture of parched ground and a link to page 7, which is devoted to the story. Nicely laid out and not a bad selection of pictures of errant climate researchers – Jones, Thorne, Jenkins, Wigley, Houghton, Hansen – and their errant (or honest) emails centre stage. But the normal stuff in the report and a Comment which says “These leaks do nothing to undermine the case for man-made global warming”. No doubt you can find the words on the web but the dead tree version somehow seemed important to me this morning. Quaint, I know.
Mike’s Nature Trick and Hide The Decline explained by Phil Jones, Tue, 16 Nov 1999:
http://foia2011.org/index.php?id=8693
cc: REDACTED,REDACTED
date: Tue, 16 Nov 1999 13:31:15 +0000
from: Phil Jones
subject: Diagram for WMO Statement
to: ray bradley ,REDACTED, REDACTED
Dear Ray, Mike and Malcolm,
Once Tim’s got a diagram here we’ll send that either later today or
first thing tomorrow.
I’ve just completed Mike’s Nature trick of adding in the real temps
to each series for the last 20 years (ie from 1981 onwards) amd from
1961 for Keith’s to hide the decline. Mike’s series got the annual
land and marine values while the other two got April-Sept for NH land
N of 20N. The latter two are real for 1999, while the estimate for 1999
for NH combined is +0.44C wrt 61-90. The Global estimate for 1999 with
data through Oct is +0.35C cf. 0.57 for 1998.
Thanks for the comments, Ray.
Cheers
Phil
phil (Nov 23 05:48),
thanks for the link! The e-mail is the original trick-email. Has anyone cross-checked how much overlap there is between these two releases?
I wonder if the “Redacted” is to hide a potential mole..?
Obviously whoever is redacted can be researched by the original holders of those emails…and they can do what they want with that information to inquire as to why whoever released these redacted those names.
No, this is just how the database search was setup by the programmer, as a courtesy to those named in the emails.
http://foia2011.org/
Jean, hat tip should go to WUWT blog, saw the link there.
Can someone elxplain what’s with all the redactions? Were there redactions in the original batch of emails?
Sorry — please ignore my comment. The question was already answered.
This would be interesting to Mosher since it was in response to his request:
http://assassinationscience.com/climategate/2/FOIA/mail/2274.txt
From: Hume Matthew Mr (ACAD)
Sent: Wednesday, August 05, 2009 3:16 PM
To: Palmer Dave Mr (LIB)
Cc: Walker Alan Dr (ACAD)
Subject: RE: FOI/EIR request [FOI_09-69]
Hi Dave,
We all just had a very good laugh at Phil’s comment “We do sometime ignore the
Registry advice”… If this is going to have the kind of publicity that you
suggest, I would prefer if you do not quote ANY of his answer to question 1.
The UEA actually has a very strict policy on entering into confidentiality
agreements, however as Phil so blithely admits, a handful of academics take it
upon themselves to foul things up!
As you will note from points 1 & 2 of our policy; no UEA employee, except
members of our office, has the right to sign anything on behalf of the
university – the problem is that funders/other parties can be sneaky by
sending the agreement in the name of the academic.
Our policy is:-
Someone from the Commercialisation & Enterprise Team should approve and sign
all Confidentiality Agreements:
only our staff have the legal authority to sign agreements on behalf of the
University
all agreements should be between the University of East Anglia and the party
requesting the agreement (not an individual academic or school)
we will negotiate with the other party on any issues within the document that
may be contentious
by doing this we will ensure you the best protection of your IP rights
(In special circumstances, authorisation may be obtained from the
Commercialisation & Enterprise Team allowing you to sign the agreement
yourself. Such authorisation must always be obtained in advance, will only be
valid for a specific instance, and the standard university agreement must be
used without amendment – unless we have authorised an amendment)
In all cases, a copy of the fully signed confidentiality agreement must be
retained in our office.
FYI – we are currently finishing off the final touches to our new intranet
pages – there will be a page on CDA’s with this info on it.
Also, I am away on holiday next week (10th -14th), so if you do any more info
on our policy regarding agreements etc, please contact Anne Donaldson, one of
our Commercialisation Managers ([3]a.donaldson@uea.ac.uk).
Thanks
Matt.
To date despite requests they still mostly failed to produce any documents that prove these Confidentiality Agreements even existed . Odd that given they claimed these stopped them fro realizing date , another issues the ‘reviews ‘ passed by.
Forgive me it this isn’t new.
The email files are numbered 001-5349, but I can only un-zip 5292, leaving 57 numbers or files missing. Has anyone else managed to get all 5349?
And has anyone produced a chronological set yet?
There are only 5292 files. For some reason, not all numbers are in the archive. There is no 0009.txt, for example. Maybe these were purely personal emails (witness how FOIA redacted family info, health issues, etc out of the published emails. Very classy, IMO.
http://foia2011.org/index.php?id=4182
date: Fri, 31 Jul 2009 14:18:02 +0100
from: Phil Jones
subject: Tomorrow’s EDP
to: REDACTED
FYI – might be worth looking at tomorrow’s EDP
Ian McEwan’s next novel will be on climate change deniers
Phil
From: “Ogden Annie Ms (MAC)”
To: “Jones Philip Prof (ENV)”
Date: Fri, 31 Jul 2009 12:51:13 +0100
Subject: Today’s EDP
Thread-Topic: Today’s EDP
Thread-Index: AcoR1TRLi82IJwxgT0GHSJYUs5Da1g==
Accept-Language: en-US, en-GB
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US, en-GB
X-OriginalArrivalTime: 31 Jul 2009 11:51:REDACTEDUTC) FILETIME=[35253EA0:01CA11D5]
On a lighter note, nice interview with Ian McEwan in today’s EDP which you might find
interesting. His latest book, Solar, is about climate change. Here’s the taster:
[1]http://www.edp24.co.uk/content/edp24/news/story.aspx?brand=EDPOnline&category=News&tB
rand=EDPOnline&tCategory=xDefault&itemid=NOED30%20JulREDACTED%3A16%3A23%3A913
Best, Annie
REDACTEDREDACTEDREDACTED
Annie Ogden, Head of Communications,
University of East Anglia,
Norwich, NR4 7TJ.
Tel:REDACTED592764
[2]www.uea.ac.uk/comm
REDACTEDREDACTEDREDACTEDREDACTED.
Prof. Phil Jones
Climatic Research Unit TelephoneREDACTED3 592090
School of Environmental Sciences FaxREDACTED3 507784
University of East Anglia
NorwichREDACTEDREDACTED Email REDACTED
NR4 7TJ
UK
REDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTED——-
Why was Phil Jones sending out this email about carbon trading in 2000? See who got the email.
http://foia2011.org/index.php?id=65
date: Mon, 03 Jul 2000 08:12:39 +0100
from: Phil Jones
subject: The Carbon Trader – 12th Edition – “GORE’S US$79Billion GREEN POLICY”
to: REDACTED
>Reply-To: “Alistair R G Paton”
>From: “Alistair R G Paton”
>To:
>Cc: “Gary R Stewart” ,
>REDACTEDMartin G Green” ,
>REDACTEDMax Kaplan” ,
>REDACTEDGary Wilson” ,
>REDACTEDPublic Relations Division” ,
>REDACTEDRisk Management Division” ,
>REDACTEDLegal Services Division” ,
>REDACTEDFinancial Services Division”
,
>REDACTEDEvaluations Division” ,
>REDACTEDAuditing and Verification Division”
>Subject: The Carbon Trader – 12th Edition – “GORE’S US$79Billion GREEN POLICY”
>Date: Sun, 2 Jul 2000 16:58:26 +1000
>Organization: thecarbontrader.com
>X-MSMail-Priority: Normal
>X-MimeOLE: Produced By Microsoft MimeOLE VREDACTED0
>
>REDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTED
>REDACTEDREDACTEDth Edition of The Carbon Trader – weekly “the
>worlds leading reporter, library and commercial services intermediary
>within the emerging carbon trading market” was published this morning 3
>JulyREDACTEDth Edition free of charge via:www.thecarbontrader.com
>edition we report on: Al Gore’s US$79b Green Policy, World Bank’s Green
>loans to Poland ($93m) and India ($130m), Concerns over Chinese Aluminium
>markets, Nuclear Power to be phased out by 2020, Norway’s Energy Minister
>announces green agency, Senator Robert Hill press releases and many more
>must read articles. contact: REDACTED contact:
> REDACTED contact: REDACTED I hope
>you all enjoy the 12th Edition of http://www.thecarbontrader.com and I again look
>forward to all your responses. Sincerely, Alistair R G Paton
>Chairman
>The Carbon Trader
>Level 4, 379 Pitt Street
>Sydney NSW Australia 2000
>e-mail: REDACTED
>REDACTED089 fax:REDACTED066 Attachment Converted:
>”c:eudoraattachAlistair R G Paton.vcf”
Prof. Phil Jones
Climatic Research Unit TelephoneREDACTEDREDACTED
School of Environmental Sciences FaxREDACTEDREDACTED
University of East AngliaREDACTEDREDACTED
NorwichREDACTEDREDACTED Email REDACTED
NR4 7TJ
UK
REDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTED—-
http://foia2011.org/index.php?id=136
to: Phil Jones , ray bradley
from: “Michael E. Mann”
To refresh your memory, we used the ’61-90 base period for the absolute anomaly scale, but we aligned the series based on an earlier interval of the
instrumental record, which pre-dates (largely) the recent decline in the
Briffa et al series. I think this leads to a similar picture, but if you
think there are any significant discrepancies w/ what Tim is preparing, we
should discuss.
They could not use 1961-1990 to align because it looked bad. Using the earlier period after chopping Briffa’s data is a phony way of making agreement look much better. Key part of “hide the decline”.
I know.
It’s Mike’s Nature trick described by Mann himslef this time 😉
God forbid they let the data speak for itself. That would be. . . what IS the word? Lemme see… Oh yeah, SCIENCE.
If this is copied from the UEA server, why are there 315 emails that do not contain a UEA email address? Even if these were recieved by UEA personell and saved down onto their server, there still ought to be forwarding information in the header. I can’t see a common theme for them, they just seem to be background so far.
That’s interesting. The first email without a UAE address is 0011.txt, and that one has no headers or anything, just some text signed ‘Joel’. No date, To/From information, etc.
The next one, 0036.txt, has headers but not any UEA emails. Curious.
I’ve added these emails (and docs) to my original foia archive. You can now search both together at http://di2.nu/foia/foia.pl
This was interesting.
————————————–
date: Tue Nov 16 08:57:47 1999
from: Tim Osborn
subject: time series for WMO diagram
to: p.jones@uea
The age-banded density Briffa et al. series can be got from:
/cru/u2/f055/tree6/NHtemp_agebandbriffa.dat
It is ready calibrated in deg C wrt. 1961-90, against the average Apr-Sep land temperature north of 20N. It goes from 1402 to 1994 – but you really ought to replace the values from 1961 onwards with observed temperatures due to the decline.
Rather than give you a new file of your reconstruction (Jones et al.) that is re-calibrated, I thought it was easier to just give you the coefficients. Your original normalised file should be multiplied by 0.3856, and then subtract 0.1112 to give the calibrated time series, i.e.:
CAL = (X*0.3856) – 0.1112
Cheers
Tim
“you really ought to replace the values from 1961 onwards with observed temperatures due to the decline.”
This is just good scientific practice. How are people ever going to be able to understand what public policies to adopt, if the scientists confuse them with data that trends the wrong way?
So, they normalize the data, then they delete the inconvenient trends, and THEN they calibrate them. Fudge, fudge, and fudge.
The CAL = (X*0.3856) – 0.1112 is a simple linear-regressed trend line with a slope of 0.3856.
But can you apply a trend line BEFORE you insert the data into whatever process you are analyzing? Is this part of the homogenization of proxies? That might seem reasonable, but I don’t know. If so, though, why are they excluding Briffa’s proxies? Apparently because the trend line is inconvenient, having the decline and all. Best to just pretend it doesn’t exist.
And that is the real problem for them with “hide the decline” when it first was outed, the fact that everybody understood from the first reading: They were simply ignoring facts that they didn’t like. They were cheating, in other words.
Yet, at the same time the emails repeatedly show them discussing among themselves about the divergence problem. And their approach was never – and still is not – to research WHY, but to bury it. Best not look into it too deeply, because it may then become something they can’t hide.
‘Hide the decline’ is really ‘hide the evidence’ – and how low IS that, anyway?
It is certainly something every man on the street can comprehend (regardless of Richard Black and his “Stupid people on the street, what would they understand, anyway?” attitude). Thanks be to Mr/Ms FOIA, for having outed this two years ago, and for presently providing much more background and evidence of tampering.
Yes, this is particularly interesting. Look at the timings.
1645.txt: date: Tue Nov 16 08:57:47 1999
Osborne to Jones: “you really ought to replace the values from 1961 onwards with observed temperatures due to the decline.”
Then less than 5 hours later,
3451.txt: date: Tue, 16 Nov 1999 13:31:15 +0000
The famous “hide the decline” email
[Note that Osborn is a lead author for the AR5 paleo chapter]
Some discoveries:
1) I am not mentioned much, but did find that for my Divergence paper in Climatic Change, it was initially sent to Briffa and he declined for reasons of “conflict of interest”. That is one point for the Team, but why would he have a conflict of interest? Just because I have cooties?
2) there is detailed (gory gory details) evidence of how they waged campaigns to get rid of editors they didn’t like and to recruit people they did like to IPCC.
I don’t post on RC very often at all because the moderation is more like gatekeeping but today I had to say well something. Note Gavin’s response.
“Gavin, the RC welcome page specifically states “Thus we will not get involved in political or economic issues that arise when discussing climate change.”
But here you are with an entire thread dedicated to discussing the release of emails where little if any actual science is discussed.
So if I take your actions here and put them together with Michael Mann’s email that specifically refers to the idea that Real Climate is in fact waging a “PR battle”. I come to the conclusion that you are indeed discussing political issues and Real Climate is in fact exactly what Michael Mann says this website is.
A PR machine for climate scientists. Which means that Real Climate is NOT about just the science at all – now is it?”
[Response: Would you like a pretzel with that logic? – gavin]
So now I am left thinking that my formulation failed. Or is it something else I said?
that’s about as (dis)respectful reply as one might expect. Were you hoping for some kind of acknowledgement?
Carrington in The Guardian : ” The police inquiry, led by Norfolk Constabulary, has spent nothing on the investigation since March this year, and only £6,000 in the six months before that, according to Freedom of Information responses. That is not good enough. “
Excerpt from 0237, Tim Osborn, March 2006:
“I have co-authored a paper in Nature on the reduced response to
warming as seen in tree-ring densitometric data at high-latitude
sites around the Northern Hemisphere, increasingly apparent in the
last 30 years or so.
First, it is important to note that the phenomena is complicated
because it is not clearly identifiable as a ubiquitous problem.
Rather it is a mix of possible regionally distinct indications, a
possible mix of phenomena that is almost certainly in part due to the
methodological aspects of the way tree-ring series are produced. This
applies to my own work, but also very likely to other work.
The implications at this stage for the ‘hockey stick’ and other
reconstructions are not great. That is because virtually all long
tree-ring reconstructions that contribute to the various
reconstructions, are NOT affected by this. Most show good coherence
with temperature at local levels in recent decades. This is not true
for one series (based on the density data). As these are our data, I
am able to say that initial unpublished work will show that the
“problem” can be mitigated with the use of new, and again
unpublished, chronology construction methods.”
If I am interpreting this correctly, Osborn is saying that the “divergence problem” with the hockey stick arose primarily from “methodological aspects,” and had little or nothing to do with actual tree rings. I, and many others I’m sure, have suspected this ever since MM03.
I’m intrigued that Mike Mann (via e-mail to AP), UAE/CRU (via press release), and Gavin
Schmidt (through Real Climate) have all joined in a chorus of:
“These aren’t the droids you’re looking for…”with the refrain, “Move along, move along.”
However, none of the e-mails circulated exclusively among the North American side of the “CAUSE”
would show up in the DOIA’s Climategate 2.0 or 7zip files.
Since Mike Mann has a penchant for flailing out with snarky insults at anyone who questions
anything he says, writes, or does, the really juicy and libelous stuff he shared with the
rest of the Team trioika (i.e. Handen and Bradley) is probably hiding among those University
of Virginia e-mails he’s so desparate to claim as his very private and very personal correspondence.
“This will likely not be possible with the current data set. It may be
crucial to add in to your proposal an update section as well. I understand that it is not
possible to update all 400 sites, but from the current data that are available, it is
possible to identify those sites with the strongest climate signal. Surely the project
could be greatly strengthened if some of these keys sites could be updated to present.”
ROb Wilson to Keith Briffa. #3719
#1025, Melvin to Hantemirov, the Yamal chronology WAS based on a selected subset of available cores. Was this known 2 years ago?
3123 has Juckes and Moberg with a lengthy discussion of MM2003, how their critique doesn’t match up with what is written in MM. Not clear what paper they are referring to.
711
Steve McIntyre had his comment to Nature reduced in word count and finally rejected, but we see that Wahl, Ammann, and Ritson had a request from Science that their comment critiquing Van Storch be lengthened to 1000 words.
Damian Carrington:
So now we know: PC Plod’s the culprit. Go back to your workstations, it was a false alarm.
Yes, hiding data and declines are only false scandals. It is the revealing such activities that is a scandal.
1553
Hantemirov reporting Yamal work to Briffa in October 1998.
The only scandal about this is that “climategate” is allowed to be presented as news.
Out of thousands of ILLEGALLY obtained emails, all that has been found if a few sentences that are then taken entirely out of context.
There are three problems with the argument that the latest batch of emails are being taken “out of context”:
1. Nobody says this when other kinds of whistleblower expose wrongdoing, often through the evidence of a one email. For example the email obtained by the Financial Times in Sep 05 written by an executive of chemical company Bayer misusing his position on the board of Roll Back Malaria to argue for further EU restrictions on DDT, in which he said “DDT use is for us a commercial threat.” Whatever context we don’t have (and we always lack context of some sort), that was not considered good of the image of Bayer and rightly so. There’s plenty of such material lying on the surface of the 5292 emails of Climategate 2.
2. Steve McIntyre has been providing context for Climategate 1 emails for the last two years and, as he says, they normally make the motives of the authors in question seem a lot worse. If people were really concerned about lack of context they’d take a keen interest in such discussions. Instead, before and after context is provided, all they say is “Nothing to see here, please move on.” Why?
3. The second batch of emails provides an enormous amount of context for the first batch. Isn’t that worthy of the attention of the world’s press? Why do you not point this out, given your legitimate concerns about context?
Solution : provide the context then.
A false assumption of this type of defense is that since lots of the emails show nothing, it is all nothing. That is absurd. It is the subset of emails that show the removal of editors, hide the decline, etc that are important. The entire batch is often necessary to provide “context” as well as background info outside the emails. The real problem with the defenders of the Team is that they don’t see these things as wrong, and shame on them.
Its funny how people criticised the news of the world for infringing on peoples privacy by tapping phone line and reading letters obtained from the garbage bin.
Then illegal (yes it is illegal to remove/use computer data unless given permission – whistle blowing is NOT relevant according to high court judges) emails are published and all of a sudden it is OK to peruse PREIVATE communications from many people. Remember FOI in UK occurred in 2005 it is wrong to expect University lecturers to have prescience and realise their university funded career suddenly become public property 7 years later.
It’s funny how apologists for corrupt science try and compare prying into private lives, with getting publicly funded employees to account for what they done with public’s money, especially when it involves trying to dupe the public.
It was always public property. You don’t seem to realize that.
jjthoms – to correct what you said: In England, it is not illegal for someone with authorised access to a computer system to make public information on the system. (It may well be a disciplinary matter with their employer but that’s a different matter.)
No doubt at UEA there were(in 2009) a good number of IT staff with root logon to their servers.
jjthoms, you seem to be under the illusion that the people working for UEA have some kind of ‘privacy rights’ regarding their email. Nothing could be further from the truth. Like most university employees and indeed most employees worldwide, what they produce on company time (including their emails) belongs to the university.
In addition, since the University is publicly funded, all of their bases are belong to the public, and thus subject to FOI. So they are not what you curiously refer to as “PREIVATE” communications. They are the public communications of public employees doing business on the taxpayer’s dime.
Finally, if they had simply answered the FOIs, instead of flat out lying about them, their emails might never have been released. They spent months lying to me about the data, when as David Palmer points out in the emails, it was simply in too screwed up a shape to release. As Dave said, “we don’t know which data belongs to which station, correct?”
And that makes it kinda hard to release, correct?
But being unwilling to admit that ugly reality, instead they went on a long trail of conceal, deny, deceive, avoid the FOI by any means foul or fair, and meanwhile lie to Willis about what’s really happening.
Given that string of actions, your passion about not using “illegal emails” strikes me as both highly self-serving and awfully convenient. I’m curious … did you make the same claim about the Wikileaks emails?
w.
Harvard-Smithsonian, when presented with a FOI for Willie Soon’s emails, told Soon that the “documents requested by Greenpeace are the property of the SMithsonian Institute, not the employee’s personal property” and told Soon that they planned to turn the documents over to Greenpeace (which they did):
To the best of my knowledge, no academic society objected to Willie Soon’s emails being turned over to Greenpeace.
Here is an interesting one about you Steve, some of your original emails are in this one.
http://di2.nu/foia/foia2011/mail/1737.txt
I would immediately delete anything you receive from this fraud.
You’ve probably seen now the paper by Wahl and Ammann which independently exposes
McIntyre and McKitrick for what it is–pure crap. Of course, we’ve already done this on
“RealClimate”, but Wahl and Ammann is peer-reviewed and independent of us. I’ve attached
it in case you haven’t seen (please don’t pass it along to others yet). It should be in
press shortly. Meanwhile, I would NOT RESPOND to this guy. As you know, only bad things
can come of that. The last thing this guy cares about is honest debate–he is funded by
the same people as Singer, Michaels, etc…
Other than this distraction, I hope you’re enjoying the holidays too…
talk to you soon,
mike
This one is Mike Mann admitting to deleting Steve’s posts on RealClimate
date: Mon, 28 Sep 2009 22:33:06 -0400
from: Michael Mann
subject: attacks against Keith
to: Phil Jones , Tim Osborn
Phil, Tim,
…..
Meanwhile, I suspect you’ve both seen the latest attack against his Yamal work by McIntyre.
Gavin and I (having consulted also w/ Malcolm) are wondering what to make of this, and
what sort of response—if any—is necessary and appropriate. So far, we’ve simply deleted
all of the attempts by McIntyre and his minions to draw attention to this at RealClimate.
Love the alliteration – in fact, love everything about this phrase. But how does one know that one has become a minion. I mean, really made the grade?
You get a decoder ring and a secret handshake.
Yes, but only after you have retired. All minions are retired and have therefore infinite time. #4986 (Phil Jones –> Bruce Tofield):
A funny one, seems that Steve is high on their list of subjects.
http://di2.nu/foia/foia2011/mail/3079.txt
Hey Phil,
Please see the below–any update on the Wengen paper? It appears that McIntyre is trying to
scoop us, must have somehow learned that we’ve tracked this down.
It would be nice for the paper to be officially ‘accepted’ before he figures the story out,
mike
——– Original Message ——–
Subject: Re: [Fwd: Blog Swap with Solve Climate]
Date: 09 May 2008 19:01:30 -0400
From: Gavin Schmidt [1]
To: Michael E. Mann [2]
CC: Stefan Rahmstorf [3]
References: [4]
[5]
I saw… but I think we’ll weather the storm. the followup post needs to
be soon though.
On another subject, McIntyre has worked out where IPCC 1990 fig 7.2 has
came from (almost). We are being scooped! (More to the point, what is
the status of the Wengen paper and can we do the origin post at some
point? Not too soon obviously!)
Gavin
UEA”s ”strategic alliance” with Goldman-Sachs:
http://www.ecowho.com/foia.php?file=4092.txt&search=Goldman-Sachs
date: Mon, 18 May 1998 10:00:38 +010 ???
from: Trevor Davies ???@uea.ac.uk
subject: goldman-sachs
to: ???@uea,???@uea,???@uea
Jean,
We (Mike H) have done a modest amount of work on degree-days for G-S. They
now want to extend this. They are involved in dealing in the developing
energy futures market.
G-S is the sort of company that we might be looking for a ”strategic
alliance” with. I suggest the four of us meet with ?? (forgotten his name)
for an hour on the afternoon of Friday 12 June (best guess for Phil & Jean
– he needs a date from us). Thanks.
Trevor
++++++++++++++++++++++++++
Professor Trevor D. Davies
Climatic Research Unit
University of East Anglia
Norwich NR4 7TJ
United Kingdom
Tel. +44 ???
Fax. +44 ???
++++++++++++++++++++++++++
So when they say it’s not about the Benjamins, it really is.
I think a new thread should be started about scientists contacting media. One example:
From: Michael E. Mann ???@meteo.psu.edu
> Date: Thu, 25 Jan 2007 08:17:33 -0500
> To: ???@zetnet.co.uk
> Subject: info about upcoming documentary
>
> Dear Mr. Monbiot,
>
> My previous attempts to reach you (through the Guardian and your monbiot.com email) have failed, so I’m hoping this email address (courtesy of George Marshall) does make it through.
>
> This has to do with a denialist-leaning documentary being filmed by Martin Durkin for Channel 4 TV in Britain. I saw that you had written about Durkin before in the Guardian, and was hoping that you might potentially have some interest in exposing this latest disinformation effort.
>
> I am forwarding messages from Durkin and his assistant, which I’m forwarding separately.
>
> I hope to hear back from you.
>
> best regards,
>
> Mike Mann
I second the idea of a thread for contacts with the media.
Many of you over at the Bish’s are making a rock solid case for the corruption of the BBC.
============
“Contacting” the media indeed–they think they own them.
5053 is interesting. See Phil asking Amman and Wahl to change the received date on a paper. I’m assuming this is the infamous Jesus paper of 2007?
Isn’t this a long-sought indemnifying statement from Jones?
Re: py (Nov 25 08:33),
It seems like this second deluge of emails reveals more “cc” addressees. Its as if the provider of the emails is pulling back the curtain at a deliberate, unhurried pace.
‘fraid so … for the next installment we’ll just have to wait for the next big UN/IPCC alarmism rah-rah-rah. Probably another two years ….
Does anyone think anything will happen this time?
Yep. A corrupt BBC in a cold Great Britain is too much to ignore. Also, the corruption of the science and peer review is now demonstrated as so egregious that the mass of honest scientists will gradually come to their senses.
It’s like a glacier melting, but melting it is. The sad thing is that were the globe warming, the hoax would persist.
==============
I hope so, in my field we would not be allowed to get away with something like that we would reprimanded or fired.
Here is what they think Steve’s agenda and attack will be.
http://di2.nu/foia/foia2011/mail/3890.txt
Well, if I recall right the most recent ‘venting’ on climateaudit did go
towards the hurricanes and sea level. Currently their site is not
responding, and it might just be strategic… I would assume that
they are going to launch a multi-pronged approach trying to undermine a
string of arguments that are currently either in the chain to link to
GHG forcing and/or that have one of the biggest impacts in the popular
perception. Its going to be in their usual fashion: Stir up a lot of dust
and move to the next thing before anybody can answer. In the end there is
little left…
Trying to interpret a priority list from my personal feeling of this
guy. I do this based on a google-cash because his site is down or
something. So here is my hunch/speculation. I’ll see if I can get the
document somewhere but I’m not very optimistic about this. By the way,
I’m also going to forward this to Susan an few others that might have
heard the rumor. Before I do send it to Susan, you might chip in on
this list for “internal and IPCC use”:
http://di2.nu/foia/foia2011/mail/4142.txt
date: Fri, 19 Jan 2007 08:34:16 -0500
from: “Michael E. Mann”
subject: [Fwd: Re: British documentary about global warming]
to: Gavin Schmidt , Caspar Ammann , Phil Jones
as I suspected, this is being done by a right wing hack, his name is
Martin Durkin. He appears to be the British version of our John Stossel.
Monbiot’s all over him:
http://www.monbiot.com/archives/2000/03/16/modified-truth/
http://www.monbiot.com/archives/1997/12/18/the-revolution-has-been-televised/
http://www.gmwatch.org/profile1.asp?PrId=39
I think it would be good to contact some folks like Geoerge Monbiot to
let them no that this charlatan is up to it again.
anyone have contact info for Monbiot?
thanks,
mike
Part of an interesting email thread, all of it needs to be read.
http://di2.nu/foia/foia2011/mail/4341.txt
At 13:43 16/03/2007, Alan Kendall wrote:
Phil, I’m sorry you take exception to my contacting David Palmer about this matter. I
felt that, since the University was being “slagged off” on a much read website and
people were advancing possible legal ways in which the information they request could be
obtained from the university, it was important to warn David Palmer about this
development. I assumed that you would be aware of what might be being said on that
website, whereas David Palmer would not be expected to be so informed. In fact he was
not, and thanked me for the information
I’m not exactly sure what you are complaining about and if the same situation arose
again I think I would act in exactly the same way again.
What you do with your data is not my concern, nor was it ever. I would not presume to
interfere in this area, nor have I done so. I am, however, concerned that UEA will be
beset by possible legal challanges and I consider it only prudent to warn those
involved. Here ends any further envolvement.
Also please don’t you presume to lecture me about what particular website I should or
should not be consulting. I’ll make my own mind up. In actual fact , I read both
ClimateAudit and RealClimate.
AlanK
The genesis of realclimate.org
http://di2.nu/foia/foia2011/mail/4349.txt
Gavin Schmidt
on behalf of the RealClimate.org team:
– Gavin Schmidt
– Mike Mann
– Eric Steig
– William Connolley
– Stefan Rahmstorf
– Ray Bradley
– Amy Clement
– Rasmus Benestad
– William Connolley
– Caspar Ammann
Connelley gets two mentions.
http://di2.nu/foia/foia2011/mail/4739.txt
date: Thu, 29 Mar 2007 09:54:02 +0100
from: Rob Wilson
subject: Re: [ITRDBFOR] should we, as a discipline, respond to Climate Audit
to: ITRDBFOR@LISTSERV.ARIZONA.EDU
Dear All,
I have read your replies to my post with interest and have also received quite a few e-mails privately.
Firstly, I do not think we need to get into any debate out how science affects policy and vise verse. This is simply a discussion about addressing issues raised on CA. Some of the comments are valid (e.g. data archiving) and we should take them to heart. Some are simply wrong. It is these latter set of comments that we need worry about.
The overall consensus view is that we should ignore CA. I can understand why people would choose this option but it could be dangerous to leave Steve McIntyre unchecked. After all, his auditing work had great influence in dragging Mann, Bradley and Hughes to the US Senate.
Other options proposed to me were:
1. A one off ‘guest’ thread to RealClimate, carefully describing the basics dendroclimatology (e.g. see Andy Baker’s spiel on speleothems).
2. Create our own Dendro Blog similar to RealClimate.
3. A new dendro FAQ, but addressing issues raised in CA.
4. A wiki style webpage that is continually updated by individuals within the community.
Personally I think that a one off thread on RealClimate would not really gain us, as a discipline, anything. Such a posting would soon get lost and we would be back to the present status quo. Developing our own Blog would be a huge amount of work and would only open up yet another avenue for active criticism and may in fact undermine what we actually set out to do. I think it would be better to address comments directly on CA.
The Blog approach also seems to me to be a bit too proactive and I would rather spend my time doing science and ensuring that it is done as robustly as possible. Perhaps the answer is a more passive, but continually updated web presence. 10 years ago, Rob Argent and I edited the Tree-Ring FAQ which was superseded by Henri’s far superior web pages. A new FAQ could address quite easily specific issues raised in CA.
However, I actually quite like the idea of the wiki style editing approach mentioned by Peter Brown. This could be continually updated and edited when specific issues are raised, but would really focus on the dendro basics – i.e. basic theory, sampling strategies, data processing methods and commonly used statistical methods used for reconstruction etc etc – with reference to specific papers for further reading.
So – the $1,000,000 question – what do we do now? We are all busy, and I do not think anyone wants to take on a huge burden of extra work which may be arguably a wasted effort. Whatever is decided – if anything – will only be successful if the work is spread out amongst many individuals.
Whatever we do, this should not be seen as a personal attack on Steve McIntyre. I met him last December at the AGU and we had a very civil chat over lunch. On the whole, I think he is well informed and his primary motivation is to see that palaeo science is done in an honest and open way. Many of the misinformed comments on CA are from participants, not from him.
So we need to think in terms of outreach and providing basic information to those who are generally interested in find out more about our discipline. We cannot hope to persuade those who have already ‘decided’ that tree-rings are worthless as a climate-proxy. Steve McIntyre is not in this latter category, but many of his followers are.
So – my 10 pence worth.
As I started this, I put my hand up as the first volunteer to whatever we end up doing.
Comments and volunteers welcome
Rob
Here’s some initial forensics on FOIA2011.ZIP-
FOIA/
offset of local header from start of archive: 0
(0000000000000000h) bytes
file system or operating system of origin: Unix
version of encoding software: 2.0
minimum file system compatibility required: Unix
minimum software version required to extract: 1.0
compression method: none (stored)
file security status: not encrypted
extended local header: no
file last modified on (DOS date/time): 2011 Jan 1 12:00:00
32-bit CRC value (hex): 00000000
compressed size: 0 bytes
uncompressed size: 0 bytes
length of filename: 5 characters
length of extra field: 0 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: binary
Unix file attributes (040755 octal): drwxr-xr-x
MS-DOS file attributes (10 hex): dir
There is a local extra field with ID 0x5855 (old Info-ZIP Unix/OS2/NT) and
8 data bytes (GMT modification/access times only).
Comments-
The leaker(s) used an older ZIP version (2.0 as opposed to 2.3) this time around.
There are no local time attributes in the header, other than the 8 additional bytes that my version of Info-zip (Linux) doesn’t translate.
Running “all.7z” in test mode reveals some interesting results. Apparently, the passphrase isn’t required for this function.
There are 220247 email files and 1 readme.txt file. There are no subfolders, so no additional tranches requiring different passphrases. This is the Full Monty.
The email file names are 9 or 10 digit Unix timestamps.
Oldest email is 636048969.txt dated 26 Feb 1990 16:16:09 GMT. Interestingly, The next email in the sequence is dated Mar 08 1996 8:02:01 GMT- 6 years later.
Newest email is 1258124051.txt dated 13 Nov. 2009 14:54:11 GMT. This moves the dossier acquisition up one day, from 12 Nov 2009.
It doesn’t appear that the 1000+ emails contained in foi2009.zip are included in this release. I’ll have to take a closer look, however.
If anyone knows how to direct the test output from 7z file manager to a text file, please post!
Interesting, thanks. I’d also moved ‘the dossier acquisition up one day’ to 13th Nov 2009 from my first 15 minutes with emails on Tuesday, using regex under TextMate on the Mac.
The only useful information in all.7z that can be extracted are the file names, file sizes and CRC checksum value.
I have a csv file containing these, and have made an attempt to automatically match up the file list in all.7z with the actual email text files from both the “2009” and “2011” releases.
The “2009” files need a simple transformation on them to get them to produce a CRC value and file size that matches the all.7z listing. All but 8 of the “2009” files appear to be in all.7z
The 8 “2009” files not automatically matched are:
1084625760.txt
1057011929.txt
1057166231.txt
1062189235.txt
1056986548.txt
1056478635.txt
0939844657.txt
1211040378.txt
Some of those are in the “2011” set (searching by hand for the content) where the difference is that the “2011” version has an attachment but the “2009” file does not. There is at least one “2009” email that does not appear in the “2011” files, but could still be in all.7z where the difference is an attachment.
The “2011” files are a bit more problematic. Many of them (2891) directly match an all.7z file, and a couple of hundred (249) match an all.7z file and a “2009” file. There are 2152 files that don’t match an existing size/CRC pair but manually looking at a dozen or so similarly sized files from the “2009” set it seems that many of the non-matching “2011” files have either marked redactions in them, or have the reply-to header field removed–the removal of headers makes it impossible to match the files by CRC value and I’m considering other methods of automatically matching these with the “2009” files.
So, at least some of the provided “2011” mail files appear to be slightly slightly “cut down” versions of the files in all.7z (as compared with an existing “2009” version mail file).
Would anyone be interested in the CSV file?
Or a zip of the Maildir format directory structure I’m using to browse folders of the emails via IMAP and Thunderbird? (if you don’t know what Maildir is, I would guess not).
Reg. Blank,
That’s some good detective work. Can you zip the CSV file and post it at Dropbox? Or, maybe Steve can put it here, under “Links”…
I’ve added column headers to the CSV file, sorted it, and put it up at
http://dl.dropbox.com/u/51011516/FOIA_stuff/FOIA-map-20111126.zip
It is 222408 rows so will make a spreadsheet app creak a bit.
The columns should be mostly self explanatory. I’ve added a “matches” column that can be used to help filtering.
If I get time to try to match up some of the non matching stuff (and have any kind of success) I’ll upload another version.
Got it. Thanks!
More on this. New zip file available.
http://dl.dropbox.com/u/51011516/FOIA_stuff/FOIA-map-matched-20111226.zip
I’ve added another column containing the 2011 email matched up by other means to the corresponding 2009 email. There seems to be 449 distinct emails common to both releases (of which 249 are identical). Of the 200 matched files that are not identical, the differences are relatively trivial — those differences (less a couple for “redacted” emails) have been filtered into differences.txt.
There are two versions of the map CSV file, a full file with “everything” (220,000+ rows), and a cut down file with about 6000 rows which omits the “all.7z only” matches.
This spreadsheet file lists the emails contained in the “all.7z” directory with file size, file name and the (imputed) last modified date:
http://dl.dropbox.com/u/18009262/all7zEmailDirectory.zip
It may be useful in determining the flow of email during contemporaneous events. I.E., on Monday 07/20/2009 (during the CA foia request campaign) there were 270 emails generated, substantially higher than the average daily volume.
Steve is going to love this one.
http://di2.nu/foia/foia2011/mail/1285.txt
date: Mon, 8 Dec 2003 16:57:51 -0000
from: “Richard Starkey”
subject: RE: Will Hutton’s A-level essay
to: “‘Mike Hulme'”
Hi Mike
Thanks a lot for the info – extremely interesting.
I know that SBC is editor of EE and I know she is a CC sceptic. Is his the “hidden agenda”
of EE too? Do you think WH was briefed by SBC? If I get round to replying to WH, would it
be legitimate to mention the contentious (non-neutral?!?!) nature of MMO3 or would this do
more harm than good?
Any draft reply, I would of course be happy to run by you and Asher.
Richard
PS Copying to Kevin for his interest. (He might also help me draft a reply!)
—–Original Message—–
From: Mike Hulme []
Sent: 08 December 2003 14:10
To: Richard Starkey
Cc:
Subject: Re: Will Hutton’s A-level essay
Richard,
The McIntyre and McKitrick paper (MM03) has got a hidden agenda behind it. Check out this
web site for some commentary on it. As with the contentious Soon and Baliunas paper, MM03
has been published by Energy & Environment and is part of Sonja Christriansen-Boehmer’s
on-going campaign.
[1]http://holocene.evsc.virginia.edu/Mann/EEReply.html
So while not endorsing this attempt at undermining our basis for current exceptional global
warming, I must say I find myself in sympathy with much of what Will Hutton writes. In
particular his conclusion that the debate around climate change is fundamentally about
power and politics rather than the environment seems undeniable. There are not that many
“facts” about (the meaning of) climate change which science can unequivocally reveal.
I am copying this to Asher Minns, since Asher has been giving the issue of “sound science”
and Tyndall’s reaction to it some thought recently.
Mike
At 11:37 08/12/2003 +0000, you wrote:
Dear Mike
Did you see Will Hutton s article in the Observer yesterday. See
[2]http://observer.guardian.co.uk/columnists/story/0,12877,1101658,00.html
An appalling article in my view. One of the key paras is
An important and neutral paper by Canadians Steven McIntyre and Ross McKitrick suggests
that the best guess is that, while temperatures are currently rising, they probably lie
within the range for the past 600 years. Environmentalists, just as in a battle over a new
runway, are being as partisan in their use of science as their opponents.
Do you know of these (neutral!!!!!!!!!) guys and their paper. Do they have credibility?
Is Hutton s interpretation correct?
I d like to do a reply but could do with some insight into the science.
Richard
_____________
Richard Starkey
Researcher
Tyndall Centre for Climate Research
UMIST
PO Box 88
Manchester M60 1QD
Tel:
_____________
1207.txt
About trying to circumvent FOI requests, even for the ICO
cc: Tom Wigley
date: Fri Apr 3 09:39:50 2009
from: Phil Jones
subject: Re: Final version of agenda?
to: REDACTED
Ben,
UEA is being very supportive. We need a meeting to respond – it just takes time. We are now going to have to write a 2 page context letter to go to the Information
Commissioner’s Office (ICO) to explain what IPCC is, how it works, the drafts and their comments and responses – and where these can be accessed. This will all go them once we have an officer assigned.
We have emails from all on Ch 6 saying they don’t want their emails to Keith/Tim sent on, so this should be enough. The detail that Holland wants is in some
of these emails and the attachments, which are parts (or the whole) of Ch 6 in word documents. If we’re forced to send these, then we’ll send hard copies, so he can’t see who changed what – which you can in tracker.
One odd thing with the ICO is that in order for them to assess the complaint, they need to see all. We’re trying the above with a few examples to circumvent this.
If they want to see everything, then they decide to uphold the original UEA decision, the ICO can then be asked for the material by Holland! Seems a bizarre situation. We’re claiming we can’t comply on several issues. One of these is that if we do, then we’ll be ignored by IPCC when it comes to future IPCC CLA/LA assignments.It wouldn’t just be UEA, but would then apply to anyone working in the UK. The clause relates to compliance affecting international relations! The whole FOI was set up by government principally for government depts. The ICO generally finds for the complainant, but there are numerous cases where the government dept ignores the decision. The new department of Justice is
the principal dept to ignore FOI decisions!
Good summation of what the new emails mean for climate science and why the ship AGW will continue to take on water.
http://www.ft.com/intl/cms/s/0/b8bfa07e-1692-11e1-be1d-00144feabdc0.html#axzz1euGaJ0yE
Being the arch skeptic I wonder why our source FOIA was sitting on this information for 2 years without ordering it (like Jeff ID) before releasing it. Also s/he could have released more in the first batch to deliver a fatal punch to the Team. I do understand that a big bunch is kept in reserve to hatch against repercussions.
How does #0226 get into the mix?
The email is between Rahmstorf and David Rind. No UEA addresses appear. BCC perhaps?
Steve: It was copied to the “Wg1-ar4-ch06 mailing list” which included Briffa
20 Trackbacks
[…] Apparently, it was also left at ClimateAudit…. your can go here for the latest! https://climateaudit.org/2011/11/22/new-climategate-emails/ […]
[…] Climate audit https://climateaudit.org/2011/11/22/new-climategate-emails […]
[…] for more on the Climategate 2.0 email scandal, check out: Steve McIntyre, James Delingpole, Anthony Watts, and Steve Milloy. The emails can be downloaded from […]
[…] Jeff Id on the Air Vent, James Delingpole in the Telegraph, Anthony Watts on Watts Up With That, Steve McIntyre at Climate Audit, Lubos Motl, Bishop Hill, and […]
[…] now appear to have made their way onto many climate skeptic forums, such as Watts Up With That, Climate Audit, TallBloke and The Air […]
[…] now appear to have made their way onto many climate skeptic forums, such as Watts Up With That, Climate Audit, TallBloke and The Air […]
[…] https://climateaudit.org/2011/11/22/new-climategate-emails/ […]
[…] into this treasure trove, seeking still more evidence of climate-science malfeasance. Already, some blog commenters have pointed out the same pattern that was evident in Climategate 1.0, which is, broadly […]
[…] it’s old news now, but this one will run and run (we all hope). Who got the link? WUWT, Climate Audit, the Air Vent as before and this time too Tallbloke’s Talkshop and JoNova. Anyone […]
[…] ClimateAudit […]
[…] ClimateAudit […]
[…] New Climategate Emails […]
[…] nu kom ik hem tegen in het verband van Climategate 2.0 in een reactie van Craig Loehle op de post van McIntyre:Craig Loehle – Posted Nov 22, 2011 at 11:50 AM | Permalink | ReplyMy favorite one so far is […]
[…] Är dessa e-mails äkta kan man fråga sig? Ja, några e-mails som kritiska forskare har fått har i alla fall bekräftats; bl.a. av statistikern Steve McIntyre och miljöekonomen Richard Tol i kommentarerna här. […]
[…] the Green Grinches intent on stealing our freedom, along with a whole lot of our dough, a bevy of swiped email gifts being dubbed Climategate II released near midnight the 21st promise a different kind of Cyber […]
[…] the Green Grinches intent on stealing our freedom, along with a whole lot of our dough, a bevy of swiped emailgifts being dubbed Climategate II released near midnight the 21st promise a different kind of Cyber […]
[…] the Green Grinches intent on stealing our freedom, along with a whole lot of our dough, a bevy of swiped emailgifts being dubbed Climategate II released near midnight the 21st promise a different kind of Cyber […]
[…] for more on the Climategate 2.0 email scandal, check out: Steve McIntyre, James Delingpole, Anthony Watts, and Steve Milloy. The emails can be downloaded from FOIA.org. […]
[…] Ross McKitrick notes that the IPCC instrumental temperature people can’t do statistics either. […]
[…] a muchos foros y portales de escépticos en el tema climático, tales como Watts Up With That, Climate Audit, TallBloke y The Air […]