Some RC commenters are bemoaning the criticism that GISS is currently weathering. For example, Tamino:
Although the error originated NOT with GISS but before numbers even got through their door, I’ve heard no cries for heads to roll at NOAA or NWS — just vicious attacks on GISS.
As so often, Tamino simply makes stories up out of whole cloth. While I do not accept NASA’s blaming of their supplier as an excuse, I’ve clearly stated my hope that the present controversy will spark a long overdue assessment of the many problems at GHCN, some of which have been covered at Climate Audit from time to time. In my most recent post, I observed:
Perhaps this may give a long overdue impetus for a proper examination of GHCN’s failure to properly update readily available station data.
In my opinion, much of the criticism that GISS is presently receiving is because of Gavin Schmidt’s rancorous and ineffective public relations campaign at realclimate on behalf of Hansen, where he’s spent more time making baseless attacks of his own, than in simply chinning up, taking his medicine and properly acknowledging both Watts Up and Climate Audit. Pit bull tactics are seldom effective in circumstances like the present.
We’ve also been criticized for not being “constructive”. Well, eliminating errors is constructive in my opinion. I’d also like to report that over a year ago, I wrote to GHCN asking for a copy of their adjustment code:
I’m interested in experimenting with your Station History Adjustment algorithm and would like to ensure that I can replicate an actual case before thinking about the interesting statistical issues. Methodological descriptions in academic articles are usually very time-consuming to try to replicate, if indeed they can be replicated at all. Usually it’s a lot faster to look at source code in order to clarify the many little decisions that need to be made in this sort of enterprise. In econometrics, it’s standard practice to archive code at the time of publication of an article – a practice that I’ve (by and large unsuccessfully) tried to encourage in climate science, but which may interest you. Would it be possible to send me the code for the existing and the forthcoming Station History adjustments. I’m interested in both USHCN and GHCN if possible.
To which I received the following reply from a GHCN employee:
You make an interesting point about archiving code, and you might be encouraged to hear that Configuration Management is an increasingly high priority here. Regarding your request — I’m not in a position to distribute any of the code because I have not personally written any homogeneity adjustment software. I also don’t know if there are any “rules” about distributing code, simply because it’s never come up with me before.
I never did receive any code from them. However an examination of GHCN code is clearly warranted and, if NOAA’s resources are as constrained as GISS’ are said to be, one would think that they would unreservedly facilitate third party examination of their code.
Verhojansk
Now there are many puzzles in GHCN adjustments, to say the least, and these adjustments are inhaled into GISS. Verhojansk is in the heart of the Siberian “hot spot”, presently a balmy minus 22 deg C. The graphics below compare GISS dset0 in the most recent scribal version to GISS dset 2 (showing identity other than small discrepancies at the start of the segment); the right compares GISS dset0 to the GHCN-Daily Average.
Over the past 20 years, the GISS version (presumably obtained from GHCN monthly) has risen 1.7 deg C (!) relative to the average taken from GHCN Daily results.
Left- GISS dset 2 minus Giss dset0 [[7]]; fight – Giss minus GHCN Daily
What causes this? I have no idea. It would be well worth finding out. The last GISS fiasco led to GISS code becoming available. Maybe the present fiasco will lead to GHCN code becoming available.
Climate Audit 
135 Comments
Code for the above analysis:
source(“http://data.climateaudit.org/scripts/station/collation.functions.txt”)
id=22224266000
giss.info$site[giss.info$id==id] # “VERHOJANSK”
ghcnd=read.ghcnd(paste(“RS0000″,substr(id,4,8),sep=””))
dset0=read.giss(id,dset=0);length(dset0)
dset0= ts(c(t(dset0[[7]][,2:13])),start=c(dset0[[7]][1,1],1),freq=12)
test=read.giss(id,dset=2)[[1]]
dset2=ts(c(t(test[,2:13])),start=c(test[1,1],1),freq=12)
test=ts.union(dset0,dset2,ghcnd)
Data=data.frame(c(time(test)),test[,c(1,3)]);names(Data)=c(“year”,”dset0″,”ghcnd”)
Data$diff=Data$dset0-Data$ghcnd
temp= Data$year >=1987
fm=lm(diff~year,data=Data[temp,])
summary(fm) # 0.34985
Data$predict=predict(fm,newdata=Data)
par(mfrow=c(1,2))
plot(c(time(test)),test[,2]-test[,1],xlab=””,ylim=c(-4,4),type=”l”,ylab=””,xlim=c(1985,2010))
title(“GISS dset2 – dset0″)
plot(c(time(test)),test[,1]-test[,3],type=”l”,xlab=””,ylab=””,ylim=c(-4,4),xlim=c(1985,2010))
title(“GISS dset0 – GHCND”)
lines(Data$year[temp],Data$predict[temp],col=2)
range(Data$predict[temp]) #[1] -0.4645281 1.0360017
My guess is you mean either “… seldom effective…” or “… often ineffective…”. 🙂
Mark
Steve – yes
One should check for the NOAA seasonal and monthly adjustments that drive annual adjustments that seem to be the heart and soul of much of Climate “Science”.
Perhaps Tamino’s point is that you have something like half a dozen posts now on an important issue, but one only blown out of poroprtion with denialists who have an agenda. Instead this becomes a welcoming mat for conspiracy theories and the like. It would have been easier to notify GISS or a problem and leave at that, but the manner in which WAWT and CA approach these issues is not intellectually respectable, hence the tone given by Gavin Schmidt.
Re: Chris Colose (#4),
After perusing on your blog, I can only reaffirm my own belief that people who live in glass houses should never hurl boulders.
Re: Chris Colose (#4),
I see you chose to go the old fashioned route of shooting the messenger for bearing bad news. What you conveniently ignore, Chris, is that the response of Anthony and Steve was caused in large part by the manner in which they have been treated by GISS and other ‘establishment’ science organizations. Steve has documented numerous attempts to gather data and report problems quietly. Strangely, they get ignored until Steve raises a fuss on his blog. Additionally, with respect to GISS Steve has found multiple errors with GISS and GISS has never publicly referenced him. Why would they be so petty? Gavin’s tone is condescending and arrogant. He too brought this upon himself by the way he acted. The behavior of Gavin, GISS, et cetera is what is truly not intellectually respectable.
Re: Chris Colose (#4),
Chris, IMHO, this whole issue would have never come to light, but for Anthony and Steve’s blogs. The people that are paying for the high salaries and the enormous “overhead” for these folks have a right to know about any incompetence, and there appears to be plenty at NASA and the other governmental agencies that “support” this group. An email by Steve Mc would have simply been silently ignored and covered up. If the governmental folks were “intellectually responsible,” you would have a point. But…. But I’m sure you know all that. Blogs are becoming very important, indeed! The MSM had better wake up, because they are being outstaged, big time.
Re: Chris Colose (#4),
What twaddle. AFAIK they were both polite and factually correct. What else does “intellectually respectable” mean?
Reaffirming allegience to the small god of AGW Climate Models every morning?
I would agree that NOAA has some “splainin” to do here. As an American taxpayer, I am astounded that the data massaging done by NOAA is not transparent.
That does not and should not mean that NASA is off the hook for their lack of even rudimentary QC in delivering their derivative product to the public.
Billions of taxpayers dollars go to modeling climate and nobody thinks to spend more than a minimal amount to examine the quality of the underlying data. I expect more from my tax dollars.
I note there is huge difference between the temperature history of the southern hemisphere versus the north (and the tropics as well.) The Siberian hotspot is one of the big drivers of this increasing differential.
Here is chart of south versus north at GISS. The current October anomalies would be 3 or 4 times bigger than the differential in this chart.
Sorry check my above comment, the differential between the hemispheres is only 0.53C
http://data.giss.nasa.gov/gistemp/tabledata/SH.Ts+dSST.txt
http://data.giss.nasa.gov/gistemp/tabledata/NH.Ts+dSST.txt
While if you compare it to the GISS Zonal anomaly chart for October, I don’t see how you could get these numbers. There is a massive difference between the hemispheres.
John lang – if you check this site http://www.climate4you.com/ and scroll down to the monthly records you will see that Russia is constantly showing hotter than normal results. Jan 2007 is particularly strong as are various other months.
I have to change my above post (was caught by the spam filters.)
The GISS south versus north data only shows a differential of 0.53C.
While the GISS Zonal anomaly chart shows an absolutely huge differential. Something is very wrong here.
Re: John Lang (#8),
John, I expect it has to do with surface area, which of course decreases rapidly towards the poles, so the contribution to overall global anomaly decreases. However I would like to see an explanation why there is almost a dipole in anomalies from north to south at higher than 60 degrees latitude. Is it occuring often?
I’ve commented before on other threads regarding the mystery of the black magic of how the GHCN compressed thousands of stations(5259) worth of daily data into a few hundred (846) stations of monthly data. And this for Canada alone.
This is a mirage, and happens because, unlike the folks at Real Climate, Steve does not censure anything that does not explicitly conform to his POV. He will cut people off of they go way off track or get offensive, but he does not censure opinion or statements that don’t agree with him. This is, for the most part, an open forum.
As I wrote over at WUWT, it would be nice if you guys could refrain from classifying everyone not in you camp as “deniers”. Many (most) of us don’t deny CO2 is a greenhouse gas, or that it has a effect on the climate. The main disagreements are either on the magnitude of the effect, or on the lack of openness of the methodologies.
SonicFrog – your “denialists who have an agenda” is an ad hominem attack suggesting you have little to contribute but bluster.
Why have not all the questions about GISS/GHCN data and processing been promptly and amicably resolved when they were previously brought up?
Can you point to any climate model that has been statistically validated against the satellite tropospheric temperature data over the last decade from its previous projections?
Re: David L Hagen (#13),
I think you forgot to allow for windage.
I believe you were aiming at Chris Colose (#4).
(Colose but no cigar).
If NASA is using the data, it is up to THEM to assure the quality of the data. PERIOD. This is a very good example of governmental incompetence, IMHO.
To return to the topic of this post: the responsibility of NOAA is to gather and collate and QA/QC the data. To allow thousands of world weather station sites that one can access online to go missing is simply hard to understand (see posts on CA about New Zealand and South America etc etc). To use seat-of-the-pants adjustments that are based on a couple of papers by Karl and others, without a public manual showing how the method was tested and verified, is hard to justify. RELEASE THE CODE. The responsibility of NASA here is: 1) if the data is garbage either don’t use it or make darn sure the public is warned about data deficiencies. and 2) if the data are preliminary don’t go around announcing “hottest month in a milllllion years”.
Re: craig loehle (#15),
If I were running the NASA show, I would not publish any statistics, before I had ALL the data and had assured myself that it was OK. What is this nonsense about releasing October numbers, before all the data are in? That is stupid.
Re: jae (#19),
Well, that’s the thing isn’t it? Why are they in such a hurry?
Every month it seems there is a kind of race between RSS, UAH, GISS and HadCru to get their numbers out.
More like reporters than scientists, don’t you think?
Re: deadwood (#23), Re: craig loehle (#15),
If I were running the NASA show, I would not publish any statistics, before I had ALL the data and had assured myself that it was OK. What is this nonsense about releasing October numbers, before all the data are in? That is stupid.
Re: deadwood (#23),
Yes. And I think that will eventually be the Achilles Heel of the whole bunch. They don’t have the facts, so they play these “press release” media games, referee each other’s “peer reviewed” articles and then refuse to provide data and code, create “new” statistical procedures, label all people that disagree as “denialists,” hide behind models that few can comprehend, etc., etc., etc. You have to be blind not to see what is going on.
That was an attack??????????? David, judging from your comment, I think we are on the same side of the fence here. I was refering to Tamino when I wrote
it would be nice if you guys could refrain from classifying everyone not in you camp as “deniers”. Many (most) of us don’t deny CO2 is a greenhouse gas,
You’re right of course. As a non-statistician, and a geology school dropout, I don’t have much to contribute as far as the number crunching goes. Which is why I don’t comment much. I will be the first to admit much of this is way, way, way out of my league.
That said, I do, however, get very annoyed with this boolean “either / or” filter applied to anyone expressing any kind of opinion on the subject – whether it’s a knowledgeable (or un-knowledgeable, if that be the case) disagreement of some of the science, methodologies, or questions that the science “should” be settled, being labeled as a “denialist”. Quite frankly, it’s a bit like Bush-think – “Either you’re with us, or you’re against us”. (apologies to Republicans, no offense meant, I was one… once).
Heh, I just made the same “Colose-al” error!!! 🙂
cbone,
I don’t buy it. McIntyre is named and thanked several times at GISS and RC, here for instance.
http://data.giss.nasa.gov/gistemp/updates/200708.html
Whether people ‘accept’ physics of greenhouse gases or gravity or evolution is of little relevance to me. Regardless of that, there is also the issue of presentation and communication, and I have criticized people on the “AGW side” for supposed “academic pieces” which I thought were sloppy or had a non-academic tone to them. I’m really in no mood to discuss the countless back and forths in the blogosphere, however I do respect McIntyre’s willingness to correct data. The problem is the relentless complaining, the tone, and accusation. And allowing all sorts of comments is not exactly a good thing. Most of the time the complaining is not justified and the “data” that doesn’t exist, does exist. There are much better approaches that can be taken that would generate more courteous responses from other people. I simply dropped by to see several posts on this partiuclar NOAA/GISS data issue, and the kind of responses it has generated in the less-academic side of blog land. Sorry if you disagree, but that is my final word on the subject.
C
Re: Chris Colose (#22),
As us old farts say, the squeaky wheel gets the grease, Chris. From time immorial, this has been true. The more squeak, the more grease. And these guys need a LOT of grease. IMHO, you are being very idealistic and “young” to assume that a little whimper and a simple email will change the almighty bureaucracy. This nation (USA) is suffering from exactly that simplistic, idealistic idea that “we can just sit down and talk it all out, because everyone is reasonable.” You appear to be from that camp. Name a better approach, if you have one. We need it!
Re: Chris Colose (#22),
<blockquote.Sorry if you disagree, but that is my final word on the subject.
Do you promise? Because, based on the few posts you have made you are only interested in protecting the ‘establishment.’ If you had taken the time to read back through this blog you would easily see that Steve’s attitude and tone is fully justified. In fact, it is quite amazing that he is still as polite as he is with the way he has been treated.
Censoring posts the way they do at RC only makes sense if you are more interested in PR spin rather than a legitimate discussion of the science. It actually makes sense, after all RC masquerades as a science site yet in reality it is funded by an environmentalist PR firm.
Re: Chris Colose (#23),
A search of the GISS website turns up only one instance of Steve’s name. This hardly seems to fit the description “named and thanked several times”.
Re: jc-at-play (#55),
Maybe you missed all those times he’s been thanked at RC.
Must number at least in the dozens…of zeroes.
Re: John M (#56),
According to google ‘Steve McIntyre’ appears 385 times on RC. I didn’t dig through them all to see how many, if any were thank you notes…
The only conspiracy I can see going on is a conspiracy of ineptitude.
Chris Colose’s suggestion that somehow information could be communicated in some better way is unrealistic. If Chris really delved into what happened, tracing a time line, he’d discover that early blog posts were fairly mild. For example around 11 am on Nov 1, I posted:
GISS October Anomaly: Warmer than September. I was amused by the hoping around of GISS temperatures– as theirs do hop around more than others. But my readers are aware that my position is this is entirely understandable, and no sign of anything suspect.
various people started to look at the data. This included Fred Nieuwenhuis, the now well knowns “Chris” and a few others who found the data puzzling. Meanwhile other bloggers likely posted data.
As the day wore on, various commenters spoke at a variety of blogs who also posted the puzzling GISS anomaly. I can’t see the precise timestamp, but John Goetz posted GISS Releases (Suspect) October 2008 Data at WTUT– possibly a few hours after I did. His title simple reflected the truth: The data looked very weird.
Various individuals in comments wrote various things at various blogs. Did some say outrageous thing? Sure. It’s a blog. Outragous things are posted at many blogs including DotEarth! But, the more data oriented looked into issues. Many cross post here, at WUTU, my blog etc.
The theory that somehow someone would have silently identified this, communicated the issue to GISS discretely is unrealistic. To a quite large extent, the problem was uncovered as a result of conversations at a bunch of blogs. The problem was also obvious. Say Chris has not posted what he found at WTUT… soon enough JohnS noted the problem as did other people!
Those who wish things were otherwise, need to learn the world has changed.
As for Chris’s suggestion that people other than Gavin are responsible for Gavin’s tone: Gavin is a middle aged man; he is responsible for his own tone.
As for piling on: Much of what seems to be perceived as piling on at RC is direct responses to Gavin’s many, varied odd statements in his recent post at RC, followed by additional odd statements in comments. He expressed a point of view. Others are responding, and happen to be explaining why his point of view is flawed in a number of ways.
As in the physical world, that conversation will continue until people get bored with it and, for the most part, the issue will be a thing of the past mentioned from time to time. Sort of like the past problems with satellite data.
Re: lucia (#28),
I agree wholeheartedly. We are seeing a paradigm shift in how science is seen and reviewed, because of the Internet. Even Joe the Plumber is now involved, and many of us “common folk” can smell bogus science. I can’t predict the shape of the new paradigm, but I’m sure it will be a LARGE shape. And the “peer-reviewed” journals have to be aware of it. The whole idea of scientific truth being confined to “peer-reviewed” journals is GONE.
Sorry for the repeated post. I am not responsible for this; my computer and secretary are responsible. I am never responsible for my mistakes.
Thanks for the token credit Lucia! 🙂
I find it interesting that Chris colose, gavin, and tamino all “saw” the same thing.
The “saw” the “denialist” camp turning GISS into “villians” This is an interesting phenomena. Gavin read WUWT and saw the mistakes. comments that went overboard or got things wrong. What he didnt see was John Goetz pointing the finger at NOAA and also isolating the problem down to a difference between the daily files and the monthlies. he “saw” what he wanted to see. Only later would gavin “see” his own mistake. The same thing, one could argue, happened when people at NOAA reviewed the global map. They saw a hot siberia. didnt think to check it because it confirmed their view of things. the same with whoever posted the GISS numbers. They “saw” what they expected to see. To tell the truth people who “saw” GISS as the main culprit also “saw” what they wanted to see.
re 28. Lucia,
…past problems with Satellite data… very wry.
re 33. Cbone. RC also prohibits comments of praise. When hansen released the code I posted a simple thank you to RC. No sarcasm, no wit, no mosh pit. Damn it was hard to write that. Do you think that made it through the filter? heck no.
Now obviously that’s an isolated case.
I so knew the drama was gonna keep flying so on the way home I picked up a bottle of vodka and a can of fish eggs… quite appropriate for a Russian born problem.
Steve, the problem is somewhere between the Russian observers and NOAA, so just looking at the NOAA may be inclomplete when it could just as possibly be some comrades on the ground (consuming too much vodka and fish eggs).
I just re-read the several posts that I’ve made on this topic. I’m a bit puzzled at the invective being addressed against the tone of these posts – I can’t see anything in any of the posts that asserts that GISS are “villains” or that can be fairly described as “character assassination”.
I try to avoid that sort of thing. Maybe one of my critics can identify offending language for me (other than the phrase “watch the pea”, which has already been discussed and which, as Lucia observed, has different connotations and should not occasion umbrage.) IS there anything else?
Re: Steve McIntyre (#39),
I appreciate your willingness to invite criticism and your capacity for introspection. With respect, I have no idea what Lucia was trying to say– one of us had too many brownies or the other too few 😉
In my earlier post on the “processing” thread we were looking at different side of the same coin– my point was not that you complained about NASA reluctance but that you did not make any mention of the positive aspects of the events reserving the entire post for failures and shortcomings. Others took license and started piling on.
Ever the optimist I tend to think that if criticism were couched within a framework of general optimism, and there is certainly cause for optimism, the serial complainers and whiners would, over time, follow your lead. On the other hand, I do not run a successful blog so what do I know?
Re: Alan Wilkinson (#49),
Calm down. We have a saying: “There are no cows on the ice”
hmmm, well apparently I cannot disagree about the presentation of others without bringing in faith climate models, the religion of AGW, Gavin and RC’s secret environmentalist funding, and the the rest of the nutjob talk.
Lucia– You seem to think I have a problem with communication? The issue, which is admittedly beyond any single persons control, is the need to make everything a huge fiasco which is exclusively confined to a narrow set of groups. Posting on blogs rather than personal exchange with GISS was not my point… You may have missed RC’s paragraph or the general point it was making
//”I believe they had two sets of data: One would be released if Republicans won, and another if Democrats won.”, “could this be a sneaky way to set up the BO presidency with an urgent need to regulate CO2?”, “There are a great many of us who will under no circumstance allow the oppression of government rule to pervade over our freedom—-PERIOD!!!!!!” (exclamation marks reduced enormously), “these people are blinded by their own bias”, “this sort of scientific fraud”, “Climate science on the warmer side has degenerated to competitive lying”, etc… (To be fair, there were people who made sensible comments as well). “//
These types of comments are exclusive to the people writing them, but they have been provoked by the relentless swarm of attacks on GISS and other scientists.
Let me ask the general readership here a question. If the temperature anomaly was found to be incorrect, but on the “too low” rather than “too high” side, then what would have happened?
a) All things would have continued as they actually did
b) The same type of people making the above remarks would have shouted proof of global cooling
c) Corrections would not have been made
d) something else
Be honest!
Steve: Chris, to my knowledge, none of the comments that you cited here were made on this blog. Blog policies prohibit such language here and I try to snip or delete offending posts if and when I notice it. Most posters here generally comply with these rules. I can only speak for what is done here. On what basis are accusing me or Climate Audit of this sort of intemperate language?
Re: Chris Colose (#40),
No, I think you are naive to believe that obvious blunders won’t be discussed in comments at blogs. It is naive to believe that all bloggers are going to stay awake all night to heavily moderate comments and/or change their moderation policies for Gavin’s sake.
I barely moderate. Anthony moderates lightly. Steve moderates– bot often after the fact.
So, you found a few people who posted disagreeable comments at WUWT? The internet is wide. Deal.
Complaining that all commenters everywhere act the way you or gavin would wish is as futile as complaining that rain falls down.
#37. There are layers of problems here. As I’ve observed before, the opportunity that is presented here is to find out how GHCN works and why, for example, they’ve been unable to locate data from Wellington NZ since 1990.
As I’ve observed before, NASA’s attempt to blame their supplier for them problem would not cut it in any other walk of life. It may be true, but they’ve warranted GHCN quality control and purport to have their own QC. We’re learning a bit about just how non-existent the QC is in both institutions. It really doesn’t matter whether the error was at GHCN or GISS. GISS presented the product to the public, so they’re responsible for errors up the food chain.
Any corporation would have bit the bullet, acknowledged responsiblity and said that they would deal with the problem. The NASA tactic of attacking the messengers is not one that is very likely to convince impartial third parties.
Steve–
I apologize for my remarks. I didn’t think much of what I said was controversial, but I am supposing my outlook is misunderstood as pertaining to a specific quote that you said rather than the broad picture of how a relatively simple issue goes through weeks of blogging fiasco before it is forgotten. I suppose Lucia is right– I live in an ideal world.
#42. I don’t use that sort of language and I get annoyed at people accusing me of using that sort of language when I don’t. Plus I set blog policies prohibiting people from pointless speculation on motives or purporting to link these things to policy. So please examine what we actually say here before making accusations.
In my opinion, NASA and Gavin Schmidt are the primary instigators of the blog outburst. The initial posts had a teasing tone. I, for one, wasn’t particularly engaged in the issue at the time, as I have long overdue things to do on proxies. As I noted elsewhere, I sent a very polite email to Hansen informing of the problem, presuming that they were unaware of it.
They should simply have acknowledged the problem, acknowledged Watts Up, sent me a polite acknowledgement of my email saying that they were working on the problem. Gavin should have posted at Watts Up that he had contacted Hansen about the problem and that they would attend to it.
A business-like handling of the situation would have ended it. Instead, they took a variety of cheap shots and rude comments, refused to source the blogs at the GISS website. Now they’ve refreshed my interest in the matter.
I’ve been perplexed by what’s going on with GHCN for a long time – why can’t they locate data for Wellington NZ? What accounts for the gap between GHCND and GHCN versions of Verhojansk etc. How does GHCN do their adjustments? I’m pretty sure that one of the outcomes of this is that GHCN is going to have to pony up their code and so we’ll finally get some answers on long-standing questions that have been stonewalled. If these agencies archived their data and code in a professional manner in the first place, that would save them a lot of aggravation. (This refers to GHCN and others, not to GISS whose archiving of code and data, however grudgingly done, is now at a far higher standard than their rivals.)
Steve– I agree names should be mentioned on GISS; having gone through the updates at NASA and RC I don’t see where you feel there are misplaced remarks in either given those types of comments at WUWT. Do you feel the aim of some is simply to discredit GISS, Hansen, etc? If so, would you ignore those remarks if you were Gavin or address them? In fact Gavin did mention the sources of where these problems were found, but threw in the paragraph about the more nuttier comments.
Many climate scientists are now in a postion where any “errors” that come about are not only recognized as scientific mistakes by peers, but they are in danger of character assassination by random internet users and their crediblity as scientists challenged. I believe it is only reasonable for some sort of hostility against repeated offenders. Do you disagree?
How disingenuous of Schmidt/Hansen to downplay the importance of verifying temperature data and sending the blame over to their supplier. In this AGW epoch we are in temperature means everything. Billions of dollars in grants, thousands of jobs, and potentially trillions in tax rests upon one thing–temperatures. AGW is about nothing but temperatures. Feelings of victory and failure on both sides of this issue are connected to the latest temperature data. It is crystal clear that due diligence in verifying temperature data is in order, even verifying the verifications.
Gavin Schmidt’s conspicuous attempt at harming Steve McIntyre’s reputation have evoked me to say what I really think–but if i did it might be out of bounds. I’ll refrain.
I think this whole affair has been tremendously constructive.. with RC acknowledging the errors ect and WUWT and CA not really condemning just pointing errors in data. It the first time I have seen an open discussion between pretty opposing camps…In the end I think what will happen pretty sooner than later is that the Co2 story will go out the window and the UHI story will prevail. After all this is human induced LOCAL warming (which probably does not affect overall global average temps). I think in the end, Pielke and Co will prevail and the money will be spent on more appropriate things.
re #46 should have said constructive discussion and land changes induced by humans not only city UHI
Scientists have always been in this position, with or without the internet. Look up the history and struggles of Mandelbrot or Hugh Everett, the guy that came up with theory of parallel universes, and compared to them, Hansen, Gavin, Jones, Mann et. al. have it easy.
Chris:
I too wish that there would be some sort of truce between the two camps. Steve and company have called for as much many times. A little humility wound go a long way. There is a lack from the other side of the fence. Example: after Steve found the “Y2K” error, serious or not, Hansen’s response was not to thank Steve, but to label him, and us by proxie, as “court jesters”. Any request for code, something that, by scientific standards, should be available for all to examine (even me, an unqualified hack) is met with “Please do not communicate with me in the future. Ben Santer”. There are plenty more responses of a similar nature in teh archives if you care to look.
For as long as I’ve been visiting this blog, two’ish years, I can only remember one time where Steve had really lost his cool.
Again, I would love to see the day when all parties could get past this stupid bickering, so I will make the unofficial plea for a truce, and hope that all parties can burry the hatchet and start working toward a common cause, the advancement of science.
Mike (sonicfrog) Alexander
(geology school drop-out)
Re: sonicfrog (#48),
It won’t happen, because there is big money on the table and admitting your predictions were wrong now has such major consequences, including the loss not only of credibility but of livelihoods and empires.
Like it or not, this is now war to the death. There might be the odd gentleman warrior, but they will be the exceptions. “Kill the infidels” is the general imperative.
Re: Chris Colose: (#40)
I believe the error would have been caught before release if it was in the other direction and of the same magnitude, They would have looked at the result and thought “to low, whats gone wrong?” and investigated. As another poster has said, +ve anomalies are expected so an abnormal one doesn’t raise any questions. On the other hand a large -ve anomaly would result in the source data being checked for errors before anything is released.
Re: Terry (#50),
The past tells us what happens.
http://www.climateaudit.org/?p=2955
I am glad people are starting to see the true colors of *some* academics coming through. Thanks again Steve.
Steve, “watch the pea” clearly refers to a shell/card game, which is at best practiced by carnies, at worst by confidence men. In either case the intent of the game is to deceive the player. By using this term you are clearly implying that either GISS or GHCN is intentionally attempting to deceive the users of its data. If you want to claim the high ground here, you could do better than this.
Hadcrut3 is in. Their October is 0.069 warmer than the September.
re: Steve McIntyre (#39)
At Appomatox, General Grant passed down an order that there were to be no jeers or catcalls when the surrendering Confederate Army marched in to stack arms. The Union troops not only understood, but applauded and cheered their recent enemy. The wit with which you skewer the members of the “team” is probably at least as hurtful as the name calling which you try to avoid. I suspect few of Mencken’s targets were fond of him. It isn’t a matter of who started it. No one likes to be proven wrong, or to be needled about it, even politely. I’m not suggesting a change in your approach, and particularly not in your constant drive toward accuracy in process and data. Those you show up, however, will inevitably see your wit in presentation as akin to the end-zone celebrations which have degraded American Football.
Steve – while I don’t agree for the most part with Chris Colose’s viewpoint and can’t say that you have directly asserted GISS were villains or made any direct character assasinations, the tone of these posts as well as the comments by several commenters here have been snarkier than usual and are approaching the tone of commentary at Open Mind.
Probably time for all of us to tone it down a bit.
Re: Bob North (#61),
Why is it Steve who has to wave the white flag? How about Gavin or better Mann come here, make a post apologizing for any past animosity or misunderstanding and promising that Steve can post at RC without being censored. I suspect you’ll see a much less snarky site here. Even better agreeing to debate the statistical issues in an open fashion would be a revolutionary advance.
I’m sorry that the bridge collapsed but it isn’t my fault the grade of steel we used didn’t meet code, blame my supplier.
Fight the graph? OK.
I think most of those people criticizing this blogs comments or Steve directly are like characters in a bad movie who try to divert attention from the real issue by pointing in a different direction and scream at the top of their lungs “look, its a pink elephant”.
AGW is cult. They treat anyone who disgrees with them as idiots or heretics or both.
Lecturing this blog in particular is shameful!
Hansen has called for “deniers” to be put in jail. Skeptical scientists are routinely slandered by alarmists as corrupt. They are accused of being bought off by oil companies, for example. Alarmists are demanding that governments enact the most draconian imposition of taxes and regulations in the history of the world. And Chris Colose thinks that some of the readers commenting on some skeptical blogs are not nice enough?! He senses some hostility?! Really?
Gavin’s boss wants to put these folks in jail. That’s no reason for them to get a little excited.
A commenter above gets right to the heart of the matter. These temperature records are at the heart of the debate. If Jim Hansen wants to use the temperature records to support his call to throw people in jail, it shouldn’t be a surprise that those people should take a special interest when the records are revealed to be shoddy and those keeping the records to be incompetent.
NASA should have acceptance criteria for all suppliers and should perform incoming inspection of received materials. They also should be keeping supplier score cards and be holding quarter supplier review. This all came about after the o-ring problem. It sounds like Hansen and crew are not complying with these regulations.
I have written a summary of the GISS October 2008 story here.
Regarding the question of who is to blame, I neither know nor care – it is GISS that publishes the temperature data the world relies on, so they should take responsibility. Of course, if they followed the basic scientific principle of obtaining data direct from the primary source rather than through a chain of middlemen, things would be much clearer and there would probably be fewer errors.
re: Kim (#60)
He already had, even while feeling bad, and with the total ruthlessness he correctly felt to be necessary. My only point is that he accepted the surrender with class, thus perhaps slightly – but only slightly – reducing the ongoing hostility.
That being observed, I confess I really enjoy Steve’s witty needling of those whose drawers he demonstrates to be below their rumps. It’s not surprising, however, that their reaction is less than jolly.
Re: Charlie Iliff (#69),
They’ve surrendered?
Maybe. But if it looks like the carnies are actually playing “watch the pea”, why not point it out? And this “high ground” sounds kinda subjective.
When one sees a recurring pattern of identifiable behavior, one can start drawing conclusions about it. Which is, dare I say, a “scientific” response.
http://nodis3.gsfc.nasa.gov/displayDir.cfm?Internal_ID=N_PD_1280_0001_&page_name=main
Steve Mc: It looks like the second half of your post is being ignored in all this fray, but it seems important. Have you looked at comparisons between dset0 and daily data for other locations besides Verhojansk? You showed an enormous discrepancy.
Or as Gavin Schmidt puts it in his answer (to comment 174):
The GISTEMP analysis is not … the ‘historical record’
Let’s be fair. In his missive to Gavin, Steve Mc should have included a consulting invoice for preparing a simple R-script to catch the monthly data replication error. No matter the flaws in data supplied by GHCN, any reasonable QC range test would have caught the >5 sigma anomalies before GISS went public with October results. That outsiders have to concern themselves with such basic matters speaks volumes about the reliability of what passes as the “global temperature”–a less-than-scientific construct to begin with.
And Steve Mc should be commended now for showing how the suspect trend in Verhojansk data is introduced by mysterious GHCN adjustments. This artificial recent trend is by no means confined to that station, but is endemic throughout Siberia and the post-Soviet sphere. It is here that most of the apparent global warming is concentrated.
Anyone with an open mind who watched this whole thing unfold saw that it was Gavin who got defensive and “went on the attack” first. Yes, Steve made some pointed comments at the way Gavin was handling the situation in further blog entries, but his initial entry was simply explaining the data mix up and that he had notified GISS.
Gavin’s response to the blogosphere noticing the mistake was to acknowledge it (by way of blaming NOAA), and then launch an attack on skeptics. Simply inexcusable. Chris or Tamino or anyone else who is trying to fault McEntyre and not Gavin has to be blind to what actually occurred.
Sheeesh – those guys at RC have been insulting us for years – they’ve used every insult under the sun to describe us. I have absolutely NO sympathy for them or their current predicament.
I think there remain a few very important questions.
Was this the first iteration of the Siberian gambit?
Has it been refined or was this a clumsy mistake?
Are there additional more subtle pieces of this puzzle?
Has this scenario been floated in Australia, India or China?
What other large, sparsely populated areas might be susceptible to this type of problem?
What can be done by the private sector, at low cost, to ferret out these problems before they can become a permanent part of the record?
Should private programmers develope routines that will help keep our public agencies apprised of problems with their data?
Could this type of effort contribute in a meaningful, non-confrontational way to benefit all parties?
What are the possible benefits and pitfalls that this type of effort would encounter?
Could this a worthwhile endeavor, or would it be a fool’s mission?
Mike Bryant
It is a fairly interesting hypothetical legal question whether GISS or NOAA would be responsible for any harm caused by reliance on the bad data. Setve’s analogy of GISS to a distributor is a reasonable place to start. Generally, under most state laws, both a manufacturer and a distributor can be strictly liable for injuries caused by a defective product. The distributor (GISS) will likely have a claim back against the manufacturer (NOAA) but this in no way eliminates its its own responsibility (in the eyes of the law, at a minimum) for distributing shoddy products. A distributor is not entitled to just sit back and blame the manufacturer but has its own duty of care to ensure quality standards.
Please note – I am in no way saying GISS or NOAA has any legal liability here. That is clearly not the case. I am just having fun extending Steve’s analogy.
It is a fairly interesting hypothetical legal question whether GISS or NOAA would be responsible for any harm caused by reliance on the bad data. Steve’s analogy of GISS to a distributor is a reasonable place to start. Generally, under most state laws, both a manufacturer and a distributor can be strictly liable for injuries caused by a defective product. The distributor (GISS) will likely have a claim back against the manufacturer (NOAA) but this in no way eliminates its its own responsibility (in the eyes of the law, at a minimum) for distributing shoddy products. A distributor is not entitled to just sit back and blame the manufacturer but has its own duty of care to ensure quality standards.
Please note – I am in no way saying GISS or NOAA has any legal liability here. That is clearly not the case for a host of reasons. I am just having fun extending Steve’s analogy and seeing how legal theory apportions blame.
OK, in deference to critics,I’ve changed the title of the other post to “Watch the Ball” – something that we squash players are told to do all the time.
Now people have been pretty strident in criticizing my posts, so I assume that there must be more to their complaints than this, but that’s the only phrase that anyone’s come up with. Is that it? Surely there must be something more to warrant charges that I’ve been engaging in “character assassination”.
As to snark, I consciously adopt a tone of irony (snark, if you will) rather than getting angry about things. Oddly enough, Eli Rabett recently lauded snark in public discourse, linking approvingly to an article which said of a prominent public figure:
In this respect, I’d also like to repeat a response to a critic on an earlier occasion who said:
Now my understanding of Oscar Wilde’s definition was different (and I do try to comply with this definition):
However, I observed the first understanding perhaps explained the following Gavin Schmidt comment:
Under the first definition, Gavin’s beef is:
It’s distressing to think that so much of their sense of well-being is tied up in my ability to express myself.
Re: Steve McIntyre (#81), Without a little humor the auditing here would get a little bit thick.
Chris Colose (#23)
Could you provide a parallel example from Gavin at RealClimate? I read his site periodically, and have yet to see any such praise. I appreciate your GISS example, and found it to be a good example of how the system should work.
cbone (#57),
I can’t tell if your tone is sarcastic here, but my impression is that there have yet to be any thank you notes for Steve on RealClimate from Gavin or other listed contributors. I’d be pleased to see any if you can point them out.
The magnitude of Google results you point is misleading, as many RealClimate pages appear multiple times in the results pages unless you very carefully construct your query. I tried a number of queries (still with duplicates) in which I found insults but no praise.
Steve, as a consumer of your blog-anations, I enjoy you writing style. Snark keeps me coming back. I find it witty.
I (not too) frequently have to say as a manufacturer when I address a product issue with my customer – “This is the reason, but it is not an excuse.”
Whatever the reason, the responsibility lies here, and I have to make the necessary modifications in construction to make sure it does not happen again, and then modify the testing procedure so that if it does happen again, it is caught before it hits the shipping dock.
I value my customers highly.
wow. Stan’s post (#66) is really well put.
They seem to have no answer to the content, so complaints about tone are to be expected. That is distressing, but I certainly didn’t mean to be “strident” in criticism, but just to say I’m not surprised when they react to the snarkiness (Is that a word?). I certainly did not mean to imply any support for the content or tone of any of Real Climate’s attitude or Gavin Schmidt’s personal attacks. In an earlier age, I’d proudly be your second, except that your earlier observation of the team’s inability to shoot straight might put me at risk well outside their target area. You’d certainly be safe.
I look forward to announcements of substantial changes in data evaluation from measuring station to GISTEMP, but am not holding my breath despite the number of species my exhalation might doom to extinction.
Please keep it up.
Steve – Thinking about it a bit more, this was not coming anywhere close to the tone at Open Mind. Your snark usually has at least an underlying element of humor. My apologies. Dave Dardinger – Never said anyone should wave the white flag, just to be civil for civility’s sake. In other words to the right thing even if those with whom you disagree don’t.
Well let’s see this for example,if a carpenter were building your house and had some 2×10’s that had dry rot on them and he used them for floor joist, would you take the excuse of “the lumberyard shipped some that had dry rot” I think not because you would expect the carpenter to indentify dry rot. The same thing should be expected of NASA or whomever uses the temperature data.
I have learned to never use any data from NOAA or NASA or GISS. Too much massaging and too many data revisions for me.
Just give me the raw data, I will critical evaluate it and come to my own conclusions.
Have they fooled themselves?
Ego and consensus is part of the human condition, it drives us forward and trips us over, look around you!
RE: #81
Steve,
In answer to your question, no, there is something else which caught my attention, at least. Your comment #7 in the Memo to Gavin Schmidt directly accuses him of plagiarism. Can you prove that Gavin read your post, was truly enlightened for the first time, and decided to change his response based entirely on your statements? Perhaps prove is a step too far, but how sure are you that this is what really happened? Have you considered 3 other possibilities:
1. Gavin already knew about the issue and something independent of your blog jogged his memory.
2. Gavin knew this already but remembered after reading your blog post.
3. Gavin knew this already but thought his post intimated these details but later realised after 1 or 2 that his statement could easily be misinterpreted and changed it accordingly.
Neither of these possibilities constitutes plagiarism. I think, based on Gavin’s own statements, that 3 is the most likely having been enlightened of the misunderstanding by your post. He pretty much says this. Given the perfectly reasonable and innocent explanations I think a direct accusation of plagiarism is inappropriate. If you add this to pea switching title we have a pattern of behaviour: something happens which has any number of perfectly innocent explanations, you assume the worst, make a serious accusation and we later find the problem has a reasonable explanation.
This is not an isolated case either. The instance of you scraping the data from GISS and the blocking of your IP comes to mind; you claimed they deliberately blocked you, but it was an offending IT security protocol which prevented you from script scraping the data en-masse. Hansen was perfectly fair and reasonable with you about this. In fact, as I recall, Hansen only admonished you after the 0.05oC/1934/1998 affair because of your apparent indifference to the misreporting your Y2K fix. As you may recall, many many media and blog folks confused the USA with the entire globe.
On the subject of snark; snark is all well and good and it has its place, Eli snarks, Deltoid snarks, quite a few AGW proponent blogs snark and you should, of course, be welcome to snark too. However, I find that when reading about science, snark is best left aside. I personally find it a lot easier to read about scientific information in the absence of snarky remarks. I’ve found examples from AGW blogs where science and snark have been mixed and I generally get put off. I’ve found this on occasion at Eli’s and RC (once). I think the argument, if it’s scientific, is far more effective without snark. Snarky, though scientifically correct posts, in my experience only preach to the converted and put off neutrals, and I would call myself a neutral on the subject of proxies, for instance, having yet to see certain issues adequately dealt with.
Steve: There’s no doubt in my mind that none of your alternates 1-3 apply here. In my opinion, Gavin used “results” from Climate Audit. I do not think for one minute that he had independent knowledge of obscure information about (say) Erbogacen; his posting about such things within one hour of my posting about it here is not a believable coincidence. He did not give any “credit”, let alone, “appropriate credit” to me or Climate Audit. You say that I “directly accused” Gavin Schmidt of plagiarism. I wish people would occasionally pay attention to how I express things. I made no “direct accusation”. Gavin’s failure to credit CA is a fact, which I’m sure you won’t deny. There is overwhelming circumstantial evidence that Gavin’s results were derived from CA and he gave no working papers evidencing that he derived these results independently. I also provided a definition of plagiarism from Columbia University where Hansen sometimes hangs his hat. I arranged some facts, yes, but show me where I “directly” accused Gavin of plagiarism. But enough about me, do you think that Gavin plagiarized Climate Audit?
You say:
Your facts are wrong here. There was no such IT security protocol. My scraping script was puttering along quite happily and was manually blocked in the middle of the run. I contacted the data manager and informed him that I was a researcher and not a robot, but I was still not allowed to continue. So I publicized the matter at the blog and within a day, Hansen decided that I could download after all. Whether they would have relented without the adverse publicity at the blog is anyone’s guess. I think not, but I realize that people could disagree. But yes, I was deliberately blocked for a while. And remember that this is in a context where I’ve been blocked from Mann’s FTP site, from Scott Rutherford’s FTP site and the University of Arizona (Hughes) website, so it’s not entirely unreasonable to be on guard for this sort of stunt.
You say:
This is totally untrue. On Aug 6, I’d finally arrived at a pretty good understanding of what was going with this data and sent a polite notice to Hansen and Ruedy notifying them of the problem. To my surprise, on Aug 8, the entire U.S. data set had been replaced with no covering commentary and prior versions deleted. I noticed that the new results changed the “leaderboard” of the warmest years (something that we’ve heard about from time to time) and posted the new leaderboard on Aug 8. Almost immediately Climate Audit was taken out of action, attributed by some to increased traffic and by others to DOS attacks (I’m not in a position to render an opinion.) On Aug 10, Hansen sent out an email to Andrew Revkin and others saying that “it was a tempest inside somebody’s teapot dome, and that perhaps a light was not on upstairs”, a statement that he later claimed was not “ad hominem”. Notwithstanding this insult, on Aug 11, I published a balanced assessment at Climate Audit and concurrently at Watts Up, providing a balanced assessment, leading as follows:
If Hansen or Gavin Schmidt were worried that my views were being misrepresented, then all they had to do was to link to this article at either Climate Audit or Watts Up. Had they done so, perhaps they would have avoided some of the media misunderstanding. But you can hardly blame for me for their failure to link to Climate Audit.
Re: plimple (#91),
Steve M. said:
funny!
Re: plimple (#91),
Steve McI inline comments:
And yet Hansen was on record as saying the following:
“The U.S. annual (January-December) mean temperature is slightly warmer in 1934 than in 1998 in the GISS analysis (Plate 6). This contrasts with the USHCN data, which has 1998 as the warmest year in the century. In both cases the difference between 1934 and 1998 mean temperatures is a few hundredths of a degree. The main reason that 1998 is relatively cooler in the GISS analysis is its larger adjustment for urban warming. In comparing temperatures of years separated by 60 or 70 years the uncertainties in various adjustments (urban warming, station history adjustments, etc.) lead to an uncertainty of at least 0.1°C. Thus it is not possible to declare a record U.S. temperature with confidence until a result is obtained that exceeds the temperature of 1934 by more than 0.1°C.”
Craig Loehle #90 re Steve McIntyre #81: yes! I laughed till the tears ran down my cheeks!
Mike Bryant #78: yes. and I look forward to the day when some private enterprise will break through the current green stranglehold and fund “transparent science” complete with their own bite-the-bullet apologies about big business’ earlier stupidity in using covered-up cherrypicking to deny AGW rather than use truth and transparency.
OK, so you claim there was no direct accusation of plagiarism. As you say, you placed the pieces on the table with the effect of letting others arrive at that conclusion. Despite the non-existent direct claim of plagiarism you are trying to defend such a position. I gave my position in the post above. I think Gavin was aware of this issue (having referred directly to the 4 stations in question before making the change from “report” to “have”), he wrote about it in the comment in an unclear manner, he read your blog, saw you complaining, realised his english was unclear and corrected his statement to be clearer. That’s certainly possible.
Why don’t you ask him about it? That’s what I’d have done before posting what you posted. Maybe you would get him to acknowledge you if he didn’t arrive at that conclusion himself.
I found your balanced assessment to be quite indifferent to the misreporting of the Y2K fix.
Can you explain the relevance of this:
“If Hansen or Gavin Schmidt were worried that my views were being misrepresented, then all they had to do was to link to this article at either Climate Audit or Watts Up. Had they done so, perhaps they would have avoided some of the media misunderstanding. ”
I don’t see how Hansen or Schmidt linking to your article would have prevented Rush, or Instapundit or the other right wingers, that went to your website for information, from misreporting your findings.
Hansen was right when he said it was storm in a teacup dome. I think it was wholly inappropriate of him to say the lights were off upstairs. Well done for finding that error, BTW.
Steve: You accused me of being “indifferent” to misreporting of my views. I’m not. But I’m sure that you’ll understand that it is impossible for me to possibly deal personally with all the misreporting of my views. That’s one of the reasons why the blog exists. The link in question acknowledged that “the Hansen error did not have a material impact on world temperatures”, though it also said that it had have a very substantial impact on some individual U.S. stations and a “significant” impact on the U.S. average. (Remember that the exercise started in response to grandiose claims that GISS software and quality control were adequate to fix defective station histories – a claim that seems even more implausible right now BTW).
None of us can “prevent” people from misreporting findings. However, in Hansen’s shoes, if I felt that the media was misreporting my findings, then I’d publicize the exact words of my critic so that my critic’s words were readily available on my website. What better way to defuse things to quote my critic on key points. He could have quoted me as agreeing here that “the Hansen error did not have a material impact on world temperatures”.
Of course, we know why Hansen didn’t do this. He wasn’t prepared to say my name.
Steve, they shouldn’t worry about how snarky you are, but whether you are actually a Boojum.
I can’t help but think that this entire episode would have proceeded quite differently if Gavin had responded like a responsible person, “Thanks for bringing this to my attention, I will make sure that the proper people are alerted and something is done as quickly as possible. I don’t know how this could have happened, but I will find out, and I’ll do everything in my power to make sure it doesn’t happen again. Please give me a day or two to get back to you and let you know what I found out.”
Steve: I’ve made the same point on a number of occasions.
#93. I placed the pieces on the table in case someone wants to provide an explanation for the circumstantial evidence. However your following statement completely fails to provide an explanation as Gavin’s earlier use of the 4 stations derived from results that he apparently appropriated from Climate Audit without any acknowledgment or credit. You said:
He had referred to the 4 stations in question because of a prior comment that I had made at http://www.climateaudit.org/?p=4332. Let me review the chronology and you tell me whether you still think that Gavin “independently” realized the issues with the 4 named sites of Irkutsk, Bratsk, Ergobacen and Kirensk. Following refers to Nov 13.
During the night, John S sent me an email advising me that Irkutsk was still screwed up in the “fixed” file. I was looking at station information in the area that morning. At comment #66, 10.39 am Eastern (blog 9.39 am), while I was doing so, PaulM posted that the ‘corrected’ GISS data still had problems with Irkutsk and Bratsk (2 of the 4 sites in question). Shortly afterwards, at 10.42 am Eastern, PaulM posted at realclimate advising them that Oct 2008 temperatures in Irkutsk and Bratsk were normal, but his realclimate post said nothing more than that.
About 15 minutes later, in comment #51 here at 10:59 am Eastern (9.59 am blog), PaulM pointed out here that both cities had 999 for Sept and figures of 9.9 and 8.1 for Oct in the ‘corrected’ GISS data, concluding that the Oct numbers for these 2 stations were still the September numbers,
In comment #55 here, 11.15 Eastern (10:15 am blog), I posted a comment which added Ergobacen and Kirensk to the two sites mentioned by PaulM (Irkutsk, Bratsk), the first post that grouped these 4 sites. I confirmed PaulM’s observation for the 2 sites, stating that, despite the “fix”, these 4 stations still had September data in October as follows:
Within an hour, in an inline comment to a realclimate comment at 11.34 am Eastern (the inline comment would be after 11.34 am, but before 12.13 pm), Gavin reported that there were problems with these 4 stations without providing any “credit” to Climate Audit:
In comment #66 at the CA thread at 1.13 pm Eastern, (12.13 pm blog), I reported the above response by Gavin, objecting to the (continuing) failure to acknowledge CA as a source:
I added:
I wrote this point up as the next thread here showing in detail the working steps that I had gone through to identify these 4 stations, thereby proving the error.
Shortly thereafter, Gavin changed his realclimate response cited above from “reported” to “have in the GHCN file” and added the further inline response:
Prima facie, it looks pretty cut and dried to me that Gavin has used “results” from Climate Audit and failed to provide any “credit” to us. I believe that you agree that GAvin did not provide any “credit” to Climate Audit. I’ve laid out a chronology evidencing that Gavin used results from Climate Audit, which would therefore be the use of results from Climate Audit without credit.
I believe that the circumstantial evidence for use of results without credit is strong enough that you should be asking Gavin Schmidt to explain himself, rather than asking me to explain myself. Perhaps there is an “innocent” explanation , as you say. If there is, I’m all ears. I’d like to hear it.
I’m gonna put on my unsufruct gorilla suit and open another bottle of vodka and fish eggs, this is getting good!
… and before I down too many of these things, Steve Mc gives the other side as good as they give… in fact, he rarely goes to the extent that they do, I see no problem with anything of what he does when it comes to the conflict.
I’m pulling up a chair too.
Hey, are them fish eggs any good?
If Gavin was smart he would’ve taken notes and kept his mouth closed.
Harry9000… Dont believe it if just says Beluga on the can… you have to see the eyes on the embryo… oooooh yummy!(no scarcasm intended)
Mike Bryant, I’m thinking he’s over sensative… it’s not like Lucia who jumped into it for the ratings (she dont even know the meaning of Unsufruct Gorilla in a conversation about Climate Auditors catching GISS with their pants down and temps up)
Steve,
Thank you for providing a clear chronology. I had not seen your comment in the earlier thread now renamed “watch the ball”. BTW, I think this renaming gesture was widely appreciated. I agree that Gavin doesn’t credit you. I agree it is clear that he read CA and used the information here. Apologies for wasting your time. As I said, ask him about it, you could get him to credit you. My other suggestion would be to add the chronology as an update to the thread titled Memo to Gavin. That way it’s clearer to your readers as to what precisely happened. At the moment the dots are spread across different threads throughout different posts.
Re: plimple (#103),
One ALWAYS appreciates good grace in blog interchanges. This is one blog reader who would welcome your further participation. Clearly there is a good deal of material relating to climate change that needs auditing. Auditing is to be welcomed as its end result is the placing of scientific results on a firmer footing. Let the results then speak for themselves against what the proponents of any of the schools or isms might make of them. Steve’s work is wholly constructive and ought to be supported by all, regardless of Confession.
I didn’t know what an unsufruct gorilla was either.
Re: Mike Bryant (#105),
Evidently, the correct spelling is usufruct. If you google MikeC’s spelling, you find MikeC’s comments. 🙂
Lucia, I found the older spelling in Google Books. It has something to do with being granted the use of property not yours.
Mike–
I found this usufruct. It’s got the definition you suggest. Maybe they are both right? Anyway, Hansen used the one without the “n” in his “usufruct and the gorilla” missive, which also includes his “court jester”idea. Evidently
“Court jesters serve as a distraction, a distraction from usufruct. Usufruct is a mtter that the captains wish to deny. . .”
I haven’t quite figured out what the gorilla is, it seems I need to download Hansen’s “The 800 pound gorilla: The threat and taming of global climate change”, or “Gorilla”, which is adated from “The threat to the planet”. Hansen gives a link to a preprint of “Gorilla” evidently hosted at NASA, but it’s dead. So, who knows what the gorilla is supposed to be.
Anyway, it’s true I didn’t know what “unsufrunct gorilla” meant when Mike C dropped the term in comments at my blog. I can honestly say, I still don’t know what it means!
I don’t know either but I suppose with a few unsufruct gorilla suits, a couple court jester outfits, vodka and caviar you could have a helluva party.
Besides, I have a feeling that the 800 lb. gorilla is really a comatose spider monkey hidden in a shoe box.
If I were a Team member–glad I’m not–and believed I had just written up an especially excellent and robust piece of work, know what I would do? As I sent it into the journal I would also send the paper and every single bit of data and code to Climate Audit and REQUEST that CA audit it. An old friend once said to me, “The hammer that crumbles putty hardens steel.” If I believe I have steel, where else can I find a hammer that equals CA? (As has been well chronicled on this blog, not in “peer review”.) Now then, is not my attitude here the true scientific attitude, egos aside?
Drudge has a link to this story of interest:
http://www.telegraph.co.uk/opinion/main.jhtml?xml=/opinion/2008/11/16/do1610.xml
Re: Green Man (#113),
Thanks for the link. Good article. This from it:
GISS found a way to ask for more grant money.
Re: GeneII (#114),
I meant to say good in that it brings the story to the mainstream. But it does have some little inaccuracies.
UK TELEGRAPH
Looks like some major media outlets are picking up on this story…further adding to the GISS embarassment.
http://www.drudgereport.com/ (see 3rd column)
http://www.telegraph.co.uk/opinion/main.jhtml?xml=/opinion/2008/11/16/do1610.xml
“The error was so glaring that when it was reported on the two blogs – run by the US meteorologist Anthony Watts and STEVEN MCINTYRE, the Canadian computer analyst who won fame for his expert debunking of the notorious “hockey stick” graph – GISS began hastily revising its figures.”
UK TELEGRAPH:
“A GISS spokesman lamely explained that the reason for the error in the Russian figures was that they were obtained from another body, and that GISS did not have resources to exercise proper quality control over the data it was supplied with. This is an astonishing admission:”
Astonishing indeed.
Ooops
GreenMan beat me to it I see.
Scrutiny, double-checking, ferreting out sloppiness, scepticism etc. are all important elements of science. Everyone’s work gets, and must get, that kind of treatment if you want things to advance.
So I really don’t understand the concerns of people like Chris Colose. Steve McIntyre only does what good science demands. How many times has GISS had to go back and correct their calculations because someone pointed out gross errors?
Again, I think No. 118 says it all.
We’re not talking about minor errors here. What we appear to have is a system that is completely in shambles and incapable of providing reliable data. And while the GISS office alludes to a lack of manpower as the reason for poor quality control of its incoming data, its personnel seem to have plenty of time doing other things like operating blogs that ridicule other views, traveling overseas to make speeches, testifing in courts on the behalf of environmental activists and holding new conferences to call energy producers “criminals”.
The public would only like to see more professionalism and attention to data processing at an institutuion that once was one of the most prestigious in the world. That’s all.
By a system being completely in shambles (119) please note this also includes the NOAA, and I did not mean to single out GISS only.
Folks, please stop getting carried away here.
This incident shows some problems in the present system, but not that the system is a total “shambles”. I regularly caution people not to go a bridge too far and this is another occasion where people are doing so.
Unfortunately, when people go too far, it tends to de-sensitive opponents to the portion of the criticism that is valid. So please take a few valiums.
Having said that, unfortunately, these incidents seem to be about the only way at dealing with stonewall tactics on data transparency. Here the issue is no longer GISS which, after the previous incident, is commendably open in its methods – though the methods themselves are obviously needlessly and pointlessly convoluted for a simple calculation. The problem areas are CRU and GHCN and the latter will obviously be hard-pressed to avoid disclosure of source code. CRU will be harder nut to crack, but this incident won’t help them either.
Re: Steve McIntyre (#121),
I could not agree more with Steve M in this post. It is unfortunate that it is evidently impractical to steer all of “the bridge too far” comments and those of the defenders of the “consensus at all costs” into the other CA posting section.
It would appear to me that it is time for the climate science community to consolidate its temperature measuring and recording efforts. That might require some analysis of the current systems and in the process forcing them to be more transparent, but that is a good thing.
Re: David Smith (#123),
David, your anecdote brings to my mind some differences and innate characteristics that I feel are often overlooked in these discussions. Those differences, in my mind, are that government and government sponsored organizations have very different incentives then private businesses in their reactions to “consumer” problems as noted by posters here. There are very nice and accommodating people in these government organizations, who derive incentives from their personal characters, regardless of their organization’s incentives.
That Gavin Schmidt and his defenders blame their vendor for their “consumers” problem and without even sensing the impropriety of that stance, or contemplating how it sounds to the consumer, is not beyond my expectations considering the incentives of the group to which he reports. Neither are the personalities and how they are allowed to be expressed (or not be self-suppressed) in these situations by a Schmidt or a Santer surprising to me when considering the incentives of their organizations. In fact, neither does their reaction to Steve M and other interested parties requests for information surprise me. I am glad these people make the requests, and in Steve M’s case to publicize them, because in my view it shows an inherent organizational limitation.
Thanks Steve McI.
Here’s a short story where NOAA/USHCN moved pretty well to correct a small data error:
While working with rainfall data for Louisburg NC, I noticed an outlier of 63.03 inches of rain in one day. I went back to the scanned original and saw the value should have been 3.03 inches – a typo. I sent a note to the site administrator who forwarded it to the responsible agency. The agency person agreed with the finding and entered the error into Datzilla, son of Bugzilla to correct it. This all happened promptly and with nice communication to me from the parties.
(At the moment this effort has encountered a problem with an apparently unrefreshed edition of rainfall data existing on the internet, a secondary problem made visible by their investigation, and they’re working on repairing that.)
This was a very minor and inconsequential matter but it does show that, on a day-to-day working level, some parts of NOAA are responsive and undefensive in their dealings with the public.
Re.:81 I remember a warning when commenting at The Reference Frame (Dr. Motl’s blog, see the CA blog roll) and previewing the comment, “pls. correct any typo and remove insults not supported by the facts” :-). I fully agree with what Steve says.
best
#40 Chris Colose
“Let me ask the general readership here a question. If the temperature anomaly was found to be incorrect, but on the “too low” rather than “too high” side, then what would have happened?
a) All things would have continued as they actually did
b) The same type of people making the above remarks would have shouted proof of global cooling
c) Corrections would not have been made
d) something else
Be honest!”
Chris I was the first person to post about the errors on WUWT, and so in a sense it would be reasonable for me to take your remarks quite personally.
I’ve just pulled up the final (corrected) data for a few north Russian stations picked at random. They are between 7C and 10C colder for Oct 08 than for Sep 08.
So if temperatures had somehow come up too low by the same margin (i.e. some 14C to 20C colder than they initially did), northern Russia would have undoubtedly come up a reasonably dark shade of blue on the GISS anomaly map, not to mention Britain, Ireland, Finland etc.
Given I tend to look at the RSS satellite global anomaly when it comes out each month – as I did this month when it was first brought to my attention here:
http://global-warming.accuweather.com/2008/11/microwave_temperature_images_f_1.html
I would be equally surprised by a strong negative or positive divergence of the surface data from this – for October, April or any month for that matter, and would not hesitate in posting any errors I discovered whatever their effect on global temperatures. (Note I posted on a number of blogs last month that many stations on the Antarctic Peninsula had seen record breaking mean temperatures in September, just as I posted more recently that RSS -60S/-70S zone saw its coldest October in the satellite era in Oct 08)
Indeed, having read the original WUWT post on the GISS data, one of the first things I did was to compare the satellite images with the GISS images. And in fact based on this I was initially cautious about questioning the GISS Russian data, as you can see from this post I made before I spotted the errors. (At this point I was unaware that the 4C+ anomaly region in northern Russia included many positive anomalies much much higher than 4C)
“Chris (14:58:07) :
I agree with Fred that the Oct 08 GISS anomalies for Britain appear to be plain wrong – i’ve posted about this on Lucia’s blog.
However, there’s no doubt that anomalies really were pretty high over Russia. This is confirmed by the MSU/AMSU TLT anomaly map:
http://www.remss.com/msu/msu_data_monthly.html?channel=tlt
(Click next to “Anomaly” at the top of the screen)
Also, note that UAH has “NoExt Land” (i.e. land masses north of the tropics) for Oct 08 at +0.62C making it the third warmest after 2005 (+0.75C) and 1998 (+0.70C)
http://vortex.nsstc.uah.edu/data/msu/t2lt/uahncdc.lt ”
So your points a) and c) are definitely wrong, as is by implication your point d).
Your point b) is correct; however, you may be missing the tongue-in-cheek nature of some of the remarks referred to, and the “letting off steam” nature of others (c.f. some of the posts of “dhogaza” on Open Mind and occasionally RC – I pointed one of these out in a recent post to RC that got rejected, and now it appears the offending post by “dhogaza” has been removed so i can’t link you to it)
Gavin Schmidt’s admission, to quote the Telegraph:
“…GISS did not have resources to exercise proper quality control over the data it was supplied with.”
So, I neither believe I’m in need of valium, nor am I getting carried away.
A babyfood, pharmaceutical or nuclear power company would be in mean heap of trouble if it ever made such a public statement. They’d be shut down in a blink.
I think this is very serious indeed.
Anyway, with that said, I have no further comment on this.
Steve: They’re not a baby food, pharmaceutical or nuclear power company. Look, I’ve done as much as anyone to draw attention to defective QC in this field. I’m amazed at just how little effort exists and by the tantrums of those responsible. But having said that, let’s have under-stated rather than over-stated comments.
Re: Pierre Gosselin (#126),
Indeed, we already have a fair share of overstated comments from climatologists : the NSIDC’s analysis of October said this:
That “+7°C anomaly” is precisely at where the GISS & NOAA have blundered temperatures but it seems the NSIDC doesn’t care. BTW, the last sentence is priceless, one may wonder what would be NOT consistent with rapid ice growth.
RE: Archiving code…
I’m amazed, but not really shocked that these organizations have lasted and published and gained public confidence for so long without code archival being a basic function of their way of doing business. It tends to support a perspective I tend to favor that the world is actually currently split down-the-middle between those who understand computers as a tool, and those who do not. There seems to be a generation that doesn’t really understand what it is that computers do, and how best to use them. For those people, Computers are a tool, like a whiteboard or a sensor. They cannot be used as fact generators, their programs have definite structure with advantages and tradeoffs, and they essentially will always lie to those who are not careful because they only do what you tell them to, not what you want them to.
In research, this means that your algorithms should frankly be published just as your equations are. You cannot write, “Well, I got this number from untitled-1.nb, or default.xls.” and hope to get away with it as far as I’m concerned. Any researcher worth his salt who uses computers for simulation/data-interpretation puts more emphasis on methodology anyway, as it is the methodologies that are more interesting than the results.
So, really, as far as I’m concerned the GHCN, GISS, NASA are all just displaying what I see on a daily basis as an aerospace engineer. There’s a definite generational gap still moving it’s way through society of those who really understand what computers are, and those who dont.
I agree with Pierre Gosselin. When isolated incidents accumulate it gets more and more difficult to treat them as isolated incidents any longer.
Does a psychological inclination to get the right results exist? Let’s test this with the following: If it had been March/April and a similar error had occurred, giving a global cold anomaly, would they have checked (and double checked) the results before publishing them? I claim they would have.
This particular incident is not big, nor particularly interesting. It is the general pattern that is important. If such a general inclination exists, then it will also affect the parameterization of the GCMs. And at the end of the day, the reliability of the GCMs, is the key question in climate science.
If three or four changes come out of this incident, what would you have them be? Precisely what changes would we all like to see, and how many changes would it be?
I know I would like to see the mercator projection maps done away with.
Re: Mike Bryant (#131),
1) Transparency in all things
2) Verification on temperature data from outside source required before data is released
3) Love of science more than love of money, politics, fame, and environmentalism
p.s. I know these are not going to happen. Don’t worry, I’m not holding my breath.
Steve:
For you or anyone looking for the actual location of the Verhojansk station, I finally spotted the buildings up north of town, nowhere near the locations previously given. Enter 67.565295,133.41239 into Google maps. The main building and the instrument field just to the south should be visible at maximum zoom. See the discussion over on WUWT for further on the site.
As the auditing of the temperature data sets continues at CA, I commented in a post that it is important to keep all temperature trend measurements in these data sets in perspective. It helps in prevent posters from going a bridge too far – at least in my view. To that end, below I have posted a table giving some statistical details on the 1979-2007 temperature trends for UAH, RSS and GISS (and one item for HadCRUT). I also included below a graph for the UAH, RSS and GISS global temperature trends for 1979-2007 and another showing the 1998-2007 correlation between UAH and RSS and UAH and GISS and GISS and HadCRUT.
I have seen statements from John Christy that the UAH data set is independent of the land and ocean surface measurement, like GISS and HadCRUT. I have seen comments from posters here at CA that claim there is a connection of UAH to the surface measurements no matter how vaguely defined, but I have never seen evidence for a connection unequivocally presented. Obviously UAH and RSS use the same MSU measurements and make different adjustments.
The trends are differences are the largest between UAH and GISS with the RSS trends lying somewhere in between. In order to do a proper and simple minded statistical analysis one would be required to determine the more valid data set between the dependent sets UAH and RSS. I used the surface record of GISS because it is the set most analyzed at CA and data from it were more readily available to me on my computer.
I am not prepared to distinguish between the UAH and RSS data sets here, so instead I use a comparison of UAH with GISS as the two data sets giving the largest differences in trends. It should be noted that from a quick perusal of the first listed graph below that subtle differences between the UAH and RSS data sets leads to a substantial difference in the trend, i.e. 0 142 and 0.183 degrees C per decade, respectively. Notice also that the GISS trend differs from both the RSS and UAH trends most visually in the period from 1998-2007. The second listed graph shows that the UAH and RSS trend measurements are in excellent agreement from 1998-2007 while that between UAH/RSS and GISS is considerably less. As a matter of fact, the agreement between GISS and HadCRUT for that period is not that good considering that these measurements are not independent.
The table compares the data source or set, the region and the season for the statistics, listed in order left to right, of the adjusted R squared for the regression of the annual temperature anomalies versus the years from 1979-2007, the trend in degrees C per decade, the standard deviation of the trend, the lag 1 auto correlation of the residuals from the regressions and the standard deviation of the regression residuals (or the de-trended anomalies).
There are the obvious differences noted between the data sets for the trends and R^2 that in turn vary with region of the globe and the seasons. The NH shows statistically significant (using rejection at p equal or less than 0.05 that I use throughout my comments here) larger trend than the SH for all the data sets. A breakdown of the global trends by seasons shows none of these seasonal differences are statistically significant. The difference between GISS and UAH for the NH trend is statistically significant (and after adjusting the standard deviation of the trend for the auto correlation using the factor at the bottom of the table after the treatment used by Santer et al. (2008)).
The largest differences in seasonal trends for the NH occur between winter (DJF with the greater trend) and summer (JJA with the lesser trend). This seasonal difference for the UAH data set becomes very close to being statistically significant after adjusting for auto correlation.
I looked at the standard deviation of the regression residuals with an eye to explaining, perhaps, differences in measuring techniques. Interesting was the small differences between the UAH and RSS residual deviations for all regions and seasons as compared to that with GISS and in the single case of HadCRUT. An F test showed that the differences were statistically significant for only UAH or RSS versus HadCRUT for the global annual trends. The UAH/RSS to GISS differences in some of the comparisons did approach statistical significance.
I conclude this post by suggesting that climate scientists or the organizations in which they are represented if truly interested in getting the temperature trends correct would take a long and hard look at these difference in temperature data sets. Otherwise I see it as “its good enough government work” – and that may well be the case.
Re #138 Kenneth, here’s a plot of the difference between UAH and GISS anomaly estimates for the Southern hemisphere:
My curiosity centers on 2008, which is on the far right side inside the green oval (the January, 2008 value is +0.31, for orientation). Did something happen around January, 2008 to cause UAH and GISS to diverge? Too early for conjecture but it does deserve some tracking over the coming months.
3 Trackbacks
[…] wrote about this station at Climate Audit, citing a puzzle in the data, here is an excerpt of his post: Verhojansk Now there are many puzzles in GHCN adjustments, to say the least, and these […]
[…] data and a “warm” Siberia. CA link to questions about other potential issues in Russia. CA link to discussion on who should be blamed and who loses more credibility. Final CA link on discussion of what caused the problem in the first […]
[…] data computed with poor quality control are useless as a basis for public policy decisions. See this and this; see this for a listing of such articles. Plus this describing one of the many oddities in […]