In the Guardian debate, George Monbiot’s opening question (made in good faith on his part) pertained to CRUTEM, George noting that the inquiry had been able to derive a CRUTEM-like result from GHCN data and challenging me that this had somehow rebutted my “crusade” on this point.
I tried to deal with this as quickly as I could, since I did not want to waste an already short 5 minutes to deal with disinformation. My answer – which surprised Monbiot – was that CRUTEM had been little more than a passing interest at Climate Audit on which I’d seldom commented. And that Muir Russell’s finding on the triviality of CRU’s temperature unit simply endorsed a point previously made at Climate Audit. This answer seemed to baffle George and others.
Unfortunately, Monbiot and others had uncritically accepted disinformation from the Muir Russell inquiry, which, on this point (as on some others), instead of examining (with citations) actual criticisms from sources like Climate Audit, preferred instead to construct its own allegations which, in this case, they described as “broad allegations which are prevalent in the public domain”. Lucia has often criticized such Gavinesque behavior in other contexts.
My long-standing position on CRUTEM was that CRU’s obstruction of data requests was most likely due to its desire to conceal that it did so little work on quality control; that the CRU result could be derived so trivially that, in effect, CRU no longer served any useful function in this field. Long before Climategate, I’d recommended that CRU’s responsibilities in this field be transferred to the UK Met Office and that the US Department of Energy re-allocate its funding in this area to improvements at GHCN – a point that should be considered carefully in the US DOE review of their funding of CRU (reported by Jonathan Leake here.)
At the Guardian panel, I observed that CRUTEM was an almost microscopically small issue in the Climategate emails – Climategate was about the Hockey Stick and its handling by IPCC, not CRUTEM. CRUTEM was mentioned in only 25 emails and, even then, often passim.
I’ll review some past CA posts to provide support for this.
In early 2007 here, I’d observed that the HadCRU series for gridcell 57N 77E (containing the single Siberian station, Barabinsk) could be derived from a simple anomaly calculation from a GHCN version of Barabinsk. At the time, we didn’t know what stations CRU used, but, in this case, I observed that the CRU calculation was straightforward – unlike GISS, which had all sorts of weird smoothing and adjusting, which were then a topic of interest. This post contains some interesting plots of differences between various various versions for the gridcell and station – the understanding of these differences has underpinned the desire to examine data as used by the various agencies.
In late 2008, long before my own FOI requests for CRU station data, I discussed CRU calculations in more detail here, concluding with the observation that “if, like GISS, they are doing nothing other than trivial sums on GHCN data, one feels that the money would be better spent on beefing up QC and data collection at GHCN.”
The reverse engineering of CRUTEM3 looks almost pathetically easy given that we’ve already waded through step 0 of GISS, where they collate different GHCN versions (dset0) into a single station history (dset1.) CRU doesn’t have the bewildering sequence of smoothing operations that Hansen uses at multiple stages (though Hansen, mercifully, doesn’t use Mannian butterworth smoothing).
To my knowledge, unlike GISS, CRU does not make the slightest attempt to adjust for UHI, relying instead of articles like Jones et al 1990 purporting to show that UHI doesn’t “matter”.
We can already emulate GISS step0 – not that it makes any sense, but it provides a benchmark. Here’s all that seems to be necessary to produce a gridded CRUTEM3 series given a dset1 data set. First, create an anomaly-version of the series. I have a simple function anom on hand and this could be done as follows:
dset1.anom=apply(dset1,2,anom)
Then one could make an average of dset1 series within gridcell i as follows, where info is an information dataset in my usual style containing for each station, inter alia, its lat, long and gridcell number (called “cell” here):
for (i in 1:2592) grid[,i]=apply(dset1.anom[,info$cell==i],1,mean,na.rm=T)
This would yield the CRUTEM3 series. My guess as to why they don’t
want to show their work is because they probably use hundreds of line of bad Fortran code to do something that you can do in a couple of lines in a modern language. Anyway, I’ll experiment with this at some point, but this is my hypothesis on all that’s required to emulate CRUTEM3. CRU has been funded by the US DOE; if, like GISS, they are doing nothing other than trivial sums on GHCN data, one feels that the money would be better spent on beefing up QC and data collection at GHCN.
We re-visited this issue in the Mole post last summer. I observed in a comment (along the lines of my 2008 post):
Nowhere have I encouraged readers to expect any smoking guns in this data set. Quite the opposite. My own best guess as to why they are so obstructive about the data is the specific commercial interest of CRU. My guess is that they spend negligible time on quality control, but derive a lot of funding for a prestigious data set and use the funds for other purposes. They don’t want anyone to see how simplistic their analysis is and how negligible their quality control. Nothing more, nothing less. (But that’s just a guess. The real reason may be different again.)
Reader Adam observed:
The whole CRUTEM / HadCRUT gridded series can be easily reproduced for the most of the globe with the GHCN dataset. I’ve tried it for some gridcells and it worked.
This is not actually true, though it holds for some gridcells (as I’d observed for the Barabinsk gridcell.) For example, HadCRU includes SST data, which is not in the GHCN land data set. In addition, Jones has his own 1961-1990 “normals” that he uses for standardization and an exact replication of CRUTEM cannot be accomplished without these “normals”, though the calculation can be approximated using freshly calculated normals. (Oddly enough, I have a copy of the CRUTEM2 normals from my 2002 correspondence with Jones – before I’d been blacklisted because of the MM2003 criticism of MBH98). In response to Adam, I observed:
I agree with your comments. Like you, I believe that 95% of CRU is obtained from GHCN, with a very few non-GHCN sources, of which Austria is one (Norway, Sweden, Denmark are others.) Like you, I believe that they do relatively trivial manipulations of GHCN data. AS I’ve said elsewhere, that is my best guess as to the secret that they don’t want exposed and the only commercial interest that they are protecting.
Also see my post from earlier this year discussing the “end of CRUTEM” and the desireability of this responsibility being taken over by the Met Office, a post in which I review earlier comments to this effect going back a few years.
A couple of other posts at the time of the Mole incident were here and Dr Phil, Confidential Agent , in which I observed that Jones was a temperature accountant:
Jones has spent much of his academic career as a sort of temperature accountant. Commencing in the early 1980s, he collected station data and compiled averages – a useful enterprise, but surely no more than accounting.
Muir Russell’s second of three “broad allegations which are prevalent in the public domain”:
That CRU adjusted the data without scientific justification or adequate explanation. Some allegations imply that this was done to fabricate evidence for recent warming.
I don’t know who they had in mind here, as they’ve followed the Gavin Schmidt practice of not providing a citation. And perhaps somebody somewhere has made an allegation in this form. But this is not an allegation that was made at Climate Audit. My own surmise was not that CRU had adjusted the data, but that they hadn’t adjusted the data for UHI – a surmise that has been verified.
The criticism from Climate Audit was that (1) CRU provided their station data as collated to “friends” but not to potential critics; and (2) that their excuses for not providing station data were what one London reporter (not Jonathan Leake) described to me as “deliberately deceptive”.
Muir Russell did not directly address either issue. Instead, they re-framed both questions.
Postscript: In their Appendix 7, Muir Russell say that they were able to make a concordance of 90% of CRU stations to GHCN stations despite the lack of such a concordance by CRU. This is a lower percentage than the concordance (95.6%) that I’d placed online in December 2007.
Muir Russell stated:
31. The Review Team was able to match 90% of stations given in the CRU list to GHCN (see Appendix 7). CRU has stated in a written submission [15] that that the remaining 10% can be obtained from other sources including the NMOs. Thus substantial work is required to take the CRU published list and assemble 100% of the primary station data from global repositories and NMOs. We make a recommendation for the future below.
When Willis Eschenbach managed to get the list of CRU stations as used, I did semi-automated matching of CRU information to GHCN information, discussing the matter here and posted up my concordance in Dec 2007 here, in which I’d matched 95.6% of CRU stations to GHCN sources.
Climate Audit 
69 Comments
Wow. This really was a pathetic piece of work. What were they thinking?
Good critique, Steve. And it sounds like you had a good trip.
Best wishes, Pete Tillman
As a matter of interest, can anyone give any possible candidates for provenance of the following “broad allegation” against CRU as it applies to CRUTEM?
I presume that somebody somewhere has made such a claim, but right now I don’t know who.
Ahhh… it was probably me. I was sure that they had to be doing SOMETHING to their value-added product. I never believed that it wa so simple a cave-man could do it. My apologies.
In HARRY_README there was the famous line of code (valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,
2.6,2.6,2.6]*0.75 ; fudge factor) that imposed an artificial adjustment. I (and I suspect many others) was under the impression that this was code used in CRUTEM.
I bet they conflated it with CA criticisms of all the weird GISS adjustments.
A search for similar phrasing turns up this on arstechnica,com, but in mid July:
http://arstechnica.com/science/news/2010/07/another-day-another-enquiry-and-cleared-again.ars
under the sub-heading “Instrumental Data” – is that what you were after, or have I blundered into a circular reference, citing the very remarks who’s provenance you’re after?
Re: Steve McIntyre (Jul 21 13:53),
In fact the Final Report acknowledges the lack of such allegations, in the very next paragraph, page 45:
Furthermore, this allegation was not mentioned in the original Feb 11 Issues Document!
Ross discusses CRUTEM adjustments around pages 37-41 in his submission. The closest source I can see is page 4 of the submission of Mike Haseler (“there are several credible accounts of manipulation…”)
As Steve says, the issue of CRUTEM adjustments hardly arises at all in the climategate emails. The CCER seem to have invented this as a straw man to avoid addressing the real issues.
The allegation may have come out over questions about the “disappearence” of raw data and the refusal to supply “value added”
so search around those topics.
Its also a piece of the strategy of defending against charges never made.
One of the ways that people have avoided answering pertinent questions from Climate Audit is to pretend this is a debate involving the blogosphere as a whole, and answer impertinent questions from idiots elsewhere.
I think the obfuscation has worked, and the general public don’t realise that Climategate refers to specific concerns raised by a handful of people. They think it is about the mass of uninformed commentary on newspaper comment sites, etc.
Re: Dave (Jul 22 16:02),
He Who Must Not Be Named, again
This claim by Singer was pretty specific:
http://www.independent.org/newsroom/article.asp?id=2787
Steve – is this about CRUTEM or the Hockey Stick?
The two get conflated (I think), viz:
‘Fortunately, the blade of the hockey stick is still there, showing rapidly rising temperatures in the past 30 years, thanks to the valiant efforts of Prof. Phil Jones. We are breathlessly waiting for expert scrutiny of his methods of selecting data from thousands of weather stations to arrive at a single number for “global temperature.” Perhaps he will reveal the algorithms he devised to “adjust and correct” the raw data. But unfortunately, he did not save the original temperature records; as the saying goes: “The dog ate them.”’
What? No, he does not get the two conflated! You’re talking here about Dr. Fred Singer, prof. emeritus of atmospheric physics, “former founding Director of the U.S. Weather Satellite Service”, and author of a book on the subject that is writing about.
He’s talking exclusively about the temperature record in that paragraph.
If either of you can’t see that, READ IT AGAIN!
He is saying that since the Team’s recent admissions have bent the HS handle (treemometer curve) out of its former, fairly straight shape, they are “fortunate” (from their perspective) that the TEMPERATURE record still forms (in their view) a perfect blade, thus enabling them to gloss over the inconvenience of a bent handle.
Singer is not getting treemometer data conflated with temperature data in any way! He is using wit and irony to try to make them look sillier than they already look. He is clearly doing this because he doesn’t buy their account of the temperature record, and is therefore outraged at the success they’re having in promoting it.
RTF
Correction: Prof. emeritus of environmental science. But he is an atmospheric physicist, according to the bio in the article.
RTF
The comment was in reply to Steve’s query asking for “possible candidates for provenance of the following “broad allegation” against CRU as it applies to CRUTEM?” The answer to the question “is this about the hockey stick or about CRUTEM?” is that, in terms of the rhetoric employed, extremely severe criticism of the latter (i.e. making Singer a “possible candidate”) is raised in the context of discussion of the former. That’s all I meant by conflated.
Alan, thanks for the correction. I felt it was an important point for me to make, because Singer and McIntyre are on opposite sides of this issue, and I wanted to make sure that the facts are clear to readers who might not yet be completely up on all the details.
RTF
Steve –
I think the “broad allegation” might be in the (2)harry.readme posts on CA.
Even though, I did not see a comment from you in them, a lot of the “programmer types” were rightfully critical of the “cru code”.
I would the “Trevor types” saw that back in the day (Nov 22,23) when panic set, and have not let it go.
Steve: I don’t see evidence in the links that they made the specific “broad allegation” as framed by Muir Russell.
Which part of “evidence” equates to “broad allegation” is perceptible to “these committees”? “Broad allegations” to us is coffee house talk, while the evidence is very much more long lived. When this generation in power understands that *after* the coffeehouse or bar, one still needs to make a “case and point” with this *evidence*, then we will have progress you speak of. So far, I can not make a case that people are that bright. Spin is the LAW. Spin means perception, with CA analyzing the CRU code and pointing out the blatant flaws, I am sure that Trevor and company were reeling.
As to the “broad allegations”, completely unsubstantiated, the word that should have used is “implications”.
Much of the media and casual commentators initially assumed that ‘hide the decline’ referred to the CRU instrument record. Since many AGW proponents couldn’t bring themselves to read the original sources (emails, Climate Audit, etc) they took their cue from the press.
I assume that the enquiries weren’t designed to identify what was actually wrong with the CRU but to quash negative public perceptions of it. Hence the preoccupation with CRUTEM.
It would also explain why they didn’t feel it necessary to interview any key sceptics.
It will show itself as a very short sighted plan because the issues aren’t going to go away.
Steve: Muir Russell discusses “hide the decline” in passing in another section. SUrely “hide the decline” is a specific thing that differs from a “broad allegation” “That CRU adjusted the data without scientific justification or adequate explanation. Some allegations imply that this was done to fabricate evidence for recent warming.” Surely there’s something somewhere that’s more germane.
But in the public mind ‘hide the decline’, lost data, the FOI requests and CRUTEM all got mixed up together. Even sceptics got them confused. That the FOI requests came from Climate Audit tied you into the mix up (through no fault of yours). Thus ‘broad allegation’ and not a specific allegation made by you.
A good example was the original Hide the Decline song.
Also, it has long been assumed (by us know nothings) that the CRU does something with the raw data other than bundle it all together to create a global temperature and some colourful charts. Phil Jones’ evasions, the concerns over destroyed data, the protestation that fulfilling the FOI requests would take too many hours, the alternative versions of the same data (GISS, etc) all point to there being more to CRUTEM than there actually is.
That Muir Russell proved that anyone could reproduce a generic temperature graph is more embarrassing that an attempt to cover up an arcane process of temperature adjustments.
This issue came up during the Select Committee investigation Q 38 to Q52
http://www.publications.parliament.uk/pa/cm200910/cmselect/cmsctech/387b/38704.htm
“In the Global Warming Policy Foundation’s submission it appears to be casting doubt on the reliability of the data sets held by NASA and the US National Oceanographic and Atmospheric Administration.”
This might be the primary source of the allegation.
3 specific examples of the confusion:-
http://www.energytribune.com/articles.cfm?aid=2682
Elsewhere we learn about the apparently regular manipulation of data evidence:
“I’ve just completed Mike’s [Mann] trick of adding in the real temps to each series for the last 20 years (i.e. from 1981 onwards) and from 1961 for Keith’s to hide the decline.” Hide what decline, you might ask? Well the context is a discussion of the scientific fact that global temperatures have not warmed for a decade. But surely, for a scientist, facts are facts? Why hide facts? Then we learn of the private doubts of one of those responsible for alarmist warming theory: “The fact is that we can’t account for the lack of warming at the moment and it is a travesty that we can’t. The CERES data published in the August BAMS 09 supplement in 2008 shows there should be even more warming: but the data is surely wrong.”
Showing a tee shirt with the instrument record and hide the decline.
http://www.telegraph.co.uk/comment/columnists/christopherbooker/6679082/Climate-change-this-is-the-worst-scientific-scandal-of-our-generation.html
But the question which inevitably arises from this systematic refusal to release their data is – what is it that these scientists seem so anxious to hide? The second and most shocking revelation of the leaked documents is how they show the scientists trying to manipulate data through their tortuous computer programmes, always to point in only the one desired direction – to lower past temperatures and to “adjust” recent temperatures upwards, in order to convey the impression of an accelerated warming. This comes up so often (not least in the documents relating to computer data in the Harry Read Me file) that it becomes the most disturbing single element of the entire story. This is what Mr McIntyre caught Dr Hansen doing with his GISS temperature record last year (after which Hansen was forced to revise his record), and two further shocking examples have now come to light from Australia and New Zealand.
In each of these countries it has been possible for local scientists to compare the official temperature record with the original data on which it was supposedly based. In each case it is clear that the same trick has been played – to turn an essentially flat temperature chart into a graph which shows temperatures steadily rising. And in each case this manipulation was carried out under the influence of the CRU.
What is tragically evident from the Harry Read Me file is the picture it gives of the CRU scientists hopelessly at sea with the complex computer programmes they had devised to contort their data in the approved direction, more than once expressing their own desperation at how difficult it was to get the desired results.
Proofreader’s comment: Did you intend a hyperlink near the words “reported by Jonathan Leake here” in the fourth paragraph?
Steve: I’m not sure this qualifies, but allegations are made here:
http://www.americanthinker.com/2009/11/crus_source_code_climategate_r.html
If CRU were charged a Value Added Tax……they would have to pay about 061 cents.
With the politicisation of climate science, HadCRUT and GISSTemp have
snip – too much editorializing
Contrast this with the effort put in to producing the more credible economic information such as cost of living and inflation adjustments.
Should politicians persist,l they must give the job of producing temperature series to their official statistical organisations. In Australia this would be the Australian Bureau of Statistics. We need true professionals running this area.
Can anyone tell me – did Jones lose the raw temperature data or was this just an excuse not to release it ?
Steve: He combined various versions of station data but didn’t keep the originals. It increasingly looks like his “value added” to GHCN is almost negligible. Given that Jones merely did pretty elementary temperature accounting, not keeping the working data is pretty embarrassing.
The data was not lost. CRU is NOT a archival repositry of others data. Jones obtained the raw data, corrected and collated to extract the information he required then retained only what he needed. The rest resides with the originators of the data as you would expect.
GHCN and others data is basically the same but adjusted independantly to correct for errors. It is therefore rather surprising that the various methodologies and different stations all produce the same temperature curve to within a few tenths of a degree.
I believe the suspicions regarding the temperature series may pertain more to station selection procedure (relating to potential warming bias due to urban encroachment/rural issues) rather than individual station data.
Steve; I’m asking people to point to possible provenance of actual people making the second “broad allegation”. I’m aware of the issues.
Here is a piece stating that CRU was caught red-handed fabricating data on climate change.
http://www.connect2mason.com/content/%E2%80%9Cclimategate%E2%80%9D-shows-hypocrisy-global-warming-evidence-mostly-fabricated
Good article, but why is the word ‘climategate’ unimaginative ? James Delingpole, the Telegraph blogger who was the first to pick up on this, used a now familiar shorthand. The English language survives through its ability to be a recipient one. If ‘cafe’, a French word, is the best one to describe a place where coffee is drunk, we use it. Similarly, the 4-letter suffix ‘-gate’ immediately tells the reader that this represents a scandal that is being exposed.
Don’t knock it !
Lucy Jones has just blogged about Cameron’s latest ‘gaffe’. He failed to ask for any condiments when buying a hot-dog from a stall. She describes this as ‘MUSTARDGATE’. There are limits !
Bill-gate(s)?
Steve,
Anthony Watts has done a lot of work with UHI and the various problems with recent temperature data.
Have you looked at enough of it to come to a conclusion about which temperature record- if any- strikes you as accurate?
My local congressman (a physicist who was one of the few Democrats to vote against cap and trade in the house) has responded to my notes to him with a statement that … has found more than half of warming is manmade. If you include UHI effects, this is probably demonstrably true- but has little bearing in assigning a role in warming to CO2, which remains (I think) the question of the day.
I have suggested a number of times that Watts free his data so others can criticise it (earning a ban on posting on his blog in the process). In general this data has been provided by volunteers so can hardly be owned by Watts unless he has a restrictive clause signed by surveyors.
His surface station home page has not been updated in a year. but over 80% of the us sites had been surveyed in July 2009.
I find it strange that CRU can be criticised for not releasing data (which is not theirs to release) where as Watts hides his and complains bitterly when old data published on the web gets used by others.
Steve: CRU is used in many publications and in international reports. Watts isn’t and hasn’t published yet. He intends to archive the data when he publishes. I find it strange that you don’t criticize CRU and others if this concerns you.
Steve
UHI is used to discredit many of the reports. Watts is all about bad siting and UHI. Surely this is just as important and should be in the public domain as quickly as possible?
Steve; I hope that it will be. perhaps you can spend equal concern on Lonnie Thompson. However, this has nothing to do with the Muir Russell inquiry.
Seldom have I seen such a bad case of an inflated sense of self importance. I would recommend a cup of tea and a lie down.
How many FOIs did you guys send out over an issue that you now admit didn’t matter?
Steve: Untruthful excuses by institutions do matter to me, though obviously not to you.
They don’t matter to me when considering the validity of the surface temp records. So there never was an issue and the whole “audit” was to determine….what now?
Search around 6-9 months before Willis announced his FOIA
( feb 07) I believe John A is the first commenter to suggest that Somebody Do an FOIA. The topic was UHI.
You should do what bender told me to do.
Read the whole blog.
from cover to cover.
comment by coment.
With the lack of clarity around here, I’ve given up reading very much.
Then you should give up commenting as well.
I still read you, despite your domination of the obsfucation market.
In short, you ask a question, that is readily answered. answered by searching this blog. using google. Now you thought your question was smart. BUT your question was stupid. Stupid in many regards. Stupid, because you asked a question that you could have answered yourself. Stupid because you asked a question ( trying to be cute) Without knowing the answer and assuming the answer would go in your favorite. Stupid in not understanding that an AUDIT is typically a test that normal processes have been followed. They are NOT required to have a motivation to find wrong doing or to have any “point” Other than this “does this organization have good processes and good QA/QC etc?”
Boris,
If you read the Comments leading up to the very first FOIA ( I cover this in the book) You will see what the PRECISE issue is that Willis wanted to look at.
UHI.
To look at UHI, to audit Jones contention that UHI didnt matter, Willis wanted access to the EXACT stations used and the data.
That’s a fair request. Muir Russel found that CRU was obstructionist in responding to this request. Muir russel found that CRU brought trouble upon themselves by THEIR actions. Unlike the defenders of CRU who argued otherwise,
I accept Muir Russel’s conclusions….case closed?
Hardly,
This finding leads to many other branches. But clearly, I think the first order of business is to revisit a whole host of bloggers and commeters who for MONTHS argued that CRU was JUSTIFIED in its actions. Its time for them to say they were wrong. and not buried in a comment.
There is also the issue of Jones’s representations to the press on this matter, where he blamed his refusal of 2008 FOIAs on actions taken by CA readers in 2009. Further, On the data refusals and Code refusals, there is more explanation required from Jones about specific decisions.
Then there is the Issue of WHY. having found that CRU was obstructionist. Having found that they brought the trouble ON THEMSELVES, comes the question..
Why?
Why did Jones ask people to delete mails. WHEN he knew that Palmer was going to deny the FOIA requesting those mails.
Palmer feared an appeal. he told Jones this. Question to Jones.
Did you fear an appeal as well?
Question: Dr. Jones, in a prior letter, you mentioned that you would delete data if order to supply it under FOIA. is that true?
did you write this mail?
Comes the question:
Did you suggest that these emails be deleted because you feared an appeal would succeed?
Then.. What information were you trying to hide? and why?
“Then.. What information were you trying to hide? and why?”
That he was not doing much of anything at work (basically collecting and – barely – processing raw temperature data), and even that he wasn’t doing well; that he had a very easy life doing very little of note and that probably many of his graduate students could have done much better? Something like that?
Boris Writes:
Why?
Boris:
Did you ever see the Rodney Dangerfield flick, “Back to School” ?
These investigations remind me of the ‘academic investigation’ of Dangerfield’s wealthy character [Thornton Melon, college donor / student] after he’s called out for paying others to do his classwork.
[Dean Martin] > “I’m only going to ask you one time. Did you do it ?”
[Dangerfield] > “I can’t lie to you Dean. NOPE.”
[Dean, to Business Prof] > “Well, there you go. It’s all settled.”
[Professor] > “This is an outrage!”
[Dean] > “Well, in Mr. Melon’s defense, it was a really big check. What do you want me to do, torture the man ?”
[Professor] > “I’m leaving.”
[Dean, to Melon] > “I don’t think he understands the actual amounts that are involved.”
[Melon nods.]
As Mr. McIntyre decided to delete my last post I will say it again without the humour:
In Oct 2007 Willis had the station list. 90% of the station data has been available before this time as Jones said. What has he done with the data? Is there an article (peer reviewed or not) with the results of his hard won information?
I just wish there was a way to tie Muir Russell down to a table and have him read this great post (yet again) of yours Steve as then he might realise that all the people he surrounded himself with to conduct the inquiry were either completely ignorant of the actual issues at hand or they were aware of the actual issues and chose to ignore them. Maybe then he might be forced to come out and say the inquiry needs to be redone.
Warwick Hughes has quite a few posts in 2009 on CRU data being adjusted so as to to fit. They are filed under his “Jones et al” category.
Here is a part of my submission to the Muir Inquiry, in the summing up.
“There are inescapable conclusions that:
(a) PDJ cherry picked Australian temperature data for comparison of urban versus rural sites; and
(b) PDJ became confused about which sites he had used and how many, from time to time; and
(c) PDJ now expresses confusion about the origins of the data he used; and
(d) PDJ underestimated the UHI effect, thus leading to an impression of global warming: and
(e) PDJ was aware of data manipulation for the IPCC.
It is able to be derived, on closer reading of the literature cited, that PDJ embarked on a program of deception and diversion, starting at least from interactions with him and WSH(ughes), me and others in or before 1991. In one message, early data are lost. In another, they are not.
It was essentially impossible to access the data that PDJ used in his important global temperatures. He simply refused to divulge the data. WSH was the recipient of the infamous PDJ quote “We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider.”
Oops, re Warwick Hughes’ blog on Jones et al, make that
http://www.warwickhughes.com/blog/?cat=2
CHAPTER 6: LAND STATION INSTRUMENTAL TEMPERATURE DATA
They claim that the data is available to anyone. Has this always been the case?
Steve: GHCN data is available. It’s been discussed here. CRU has used other data besides GHCN e.g. from Canada, Australia. For a long time, CRU wouldn’t even disclose what stations it used. But this is not what’s been the issue: they sent CRU data to Peter Webster, Scott Rutherford and others; if there is some value added in it, then it should be available to (say) McKitrick, Pielke, McCullough or myself.
Anybody want to ask Muir Russell for their code? hehe.
Also, CRU apparently did some adjustments that they sent in to Muir.
Their FOIA compliance needs testing. I have a months old appeal that hasnt seen the light
Steven, can you put in a valid FOI request to an organisation that has no useful information?
Steve, I have seen a lot of manipulation of past data. What actually happens is that past data measurements are REDUCED in value.
In fact, I’ve seen data that has been reduced multiple times as time progresses, causing past temperatures to be further reduced.
I cannot, right now, give you the citation for this… but I’ll look later tonight.
2ndLaw,
I’m sure I read something that seemed compelling which found that past measurements had been incorrectly reduced by adjustments.
With due respect to Steve McIntyre’s skepticism on this question (and he undoubtedly has done his homework on such matters), I do wish I could find a citation for what you are referring to.
Despite Steve’s admonitions, I can’t shake the feeling that I saw something substantial about that.
RTF
Quality control … that is it in a nutshell. As far as I can see there was no quality control at the Climategate Research Unit, they didn’t have a quallity system, they certainly did not admit their mistakes and so they certainly had no system to implement corrective action.
Basically they were a bunch of cowboys churning numbers which any decent instrumentation engineer would know were almost completely useless because the equipment and site configurations were never intended to produce globally significant meaningfully accurate results to a fraction a degree.
When I worked in industry I saw precisely this kind of shambolic behaviour we saw at the CRU. The cause was usually an incompetent manager who had a record of failing to tackle quality problems and so had devoted their time to hiding their problems rather than as competent managers do: highlighting the quality problems in order to justify the necessary resources to overcome their problems.
The simple fact is that climategate should have been the necessary push to get the CRU to admit its previous failings and draw up a plan to regain control of its quality (and the huge level of cash needed to get decent measurements), but no! We are back to square one, the same “hide it under the carpet” mentality still exists in the CRU and it will stay the prevailing culture until either the CRU closes, their abysmal quality system makes them fall so completely flat on their faces that they have no choice but to improve, or those responsible are forced out.
Meanwhile, in the USA they’ve been making FOIA decisions based upon who is asking.
http://hotair.com/archives/2010/07/22/ap-homeland-security-probed-foia-requesters-delayed-data-based-on-politics/
I never was under the impression that much criticism of CRUTEM methods came from CA. Its more been the province of Anthony, Willis, and Chiefio over at WUWT. Specifically the argument that the various adjustments resulted in a significantly higher warming trend globally then was present in the raw data, something magnified by the tendency to focus on specific cases of large adjustments (Willis’ Darwin) or a poor understanding of the methods used (Cheifio).
Now we have a more general acceptance that the methods used on GHCN are generally correct (though the hard work of folks like Jeff Id, Mosher, Nick Stokes, Chad, Tamino, and others in creating their own reconstructions), we can focus on more interesting questions like the quality of GHCN and issues relating to UHI.
Of course, the handling of UHI has been the issue all along. I’ve looked at several of the studies purporting to show that the issue is inconsequential and, in my opinion, none of these studies hold up.
I’m not sure what you mean by saying that the “methods used by GHCN are generally correct”. If you are saying that they can carry out the elementary operation of creating anomalies and taking an average, I don’t know of anyone who’s argued that there were elementary errors of this type.
I was referring more to the argument that the temperature trends shown in global temperature products were in a large part results of adjustments done to the data. Using varied methods on unadjusted GHCN data shows that neither adjustments nor methodological choices are responsible for the recent observed rise in temperatures.
I did slightly err in my prior post though. I should have said that “methods used by CRU et al in creating reconstructions based on GHCN generally reflect the ‘raw’ source data”.
Also, to preempt being accused of tilting at strawmen, examples of mostly-specious allegations against CRU/NCDC/GISTemp abound here, for example: http://www.virginiaclimate.polyrad.net/surface_temp.pdf
“broad allegations … in the public domain ” means absolutely anything on any subject and is always true. The Internet is a “really, really big place”.
Not only don’t they cite what they are talking about they don’t even SAY what they are talking about.
This is the ultimate, pure distilled essence of the straw man argument.
Please don’t waste so much time on introspection over it.
The Guardian’s CIF has an interesting piece with as sub title: “Even those carrying out formal academic research are guilty of twisting scientific facts to suit their purposes” based on a an article in the Journal of the American Medical Association.
http://www.guardian.co.uk/commentisfree/2010/jul/24/ben-goldacre-bad-science-spin
Data behind the 2005 Vose et al paper in GRL, “An intercomparison of trends in surface air temperature analyses at the global, hemispheric, and grid-box scale” – lays to rest the notion that GHCN and CRUTEM closely agree.
http://www.warwickhughes.com/papers/vose05.htm
for the 1976-2003 choose by Vose et al, 57% of grid cells differ by more than 0.1 per decade (+ or -)
Please do forgive the pedantry, which I am all too aware is passe’ these days, but from the sense of the sentence I think you mean “en passant”, not “passim”.
3 Trackbacks
[…] The Muir Russell report employed the straw man technique, as related by McIntyre: […]
[…] Inquiry Disinformation about CRUTEM In the Guardian debate, George Monbiot’s opening question (made in good faith on his part) pertained to CRUTEM, […] […]
[…] not CRUTEM. CRUTEM was mentioned in customarily 25 emails and, even then, mostly passim,” explained McIntyre final […]