Memo to Gavin Schmidt

From: He Whose Name You are Not Allowed to Utter

Gavin, you said:

[Response: There are still (at least) four stations that have Oct data in place of september data but that didn't report september data (Kirensk, Irkutsk, Bratsk, Erbogacen). I expect that the SEP=OCT check that NOAA did, just didn't catch these. Still, this is embarassing - but will be fixed today. Nobody is ‘indifferent'. - gavin]

As you said elsewhere:

Why anyone would automatically assume something nefarious was going on without even looking at the numbers is a mystery to me.

Why would you assume that Erbogacen, Kirensk, Bratsk and Irkutsk did not report September data? I hope that you didn’t do so “without looking at the numbers”. Just to make things easy for you, here is a script that will do download daily GHCN data for you.

download.file(“http://data.climateaudit.org/data/giss/giss.info.tab”,”temp.dat”,mode=”wb”);load(“temp.dat”)
source(“http://data.climateaudit.org/scripts/station/collation.functions.txt”)

id1=c(22224817000, 22230230000, 22230309000, 22230710000)
giss.info[!is.na(match(giss.info$id,id1)),c("id","site")]

# id site
#1729 22224817000 ERBOGACEN
#1777 22230230000 KIRENSK
#1779 22230309000 BRATSK
#1791 22230710000 IRKUTSK

site1=giss.info$site[!is.na(match(giss.info$id,id1))]
id0=paste(“RS0000″,substr(id1,4,8),sep=””)
chron=NULL
for(i in 1:4) chron=ts.union(chron, read.ghcnd(id0[i]))
dimnames(chron)[[2]]=site1
chron

As you see, Erbogacen and Kirensk both reported September data to GHCN (daily). So your statement that these stations didn’t “report” September data is incorrect. Exactly why the September daily data didn’t get incorporated into the GHCN monthly data is one of many climate science mysteries on which we would welcome enlightenment.

The GHCN daily file seems to have lost track of Bratsk and Irkutsk after the year 2000, though GHCN monthly has, for the most, kept track of these two stations. Obviously Irkutsk and Bratsk both “reported” September data. They are both reasonably large cities whose temperatures can be located on the Internet Irkutsk Bratsk. Exactly why NASA GISS (and NOAA GHCN) were unable to locate this readily available is a mystery that has puzzled us at Climate Audit for a long time and perhaps you can enlighten us on why this task has seemingly baffled the NASA “professionals”.

In the meantime, perhaps you should withdraw your claim that these stations failed to “report” September data and replace this with a more accurate statement, saying that, for reasons that you (and many others do not understand), this data was not incorporated in the GHCN monthly file.

Perhaps this may give a long overdue impetus for a proper examination of GHCN’s failure to properly update readily available station data.


42 Comments

  1. keith
    Posted Nov 13, 2008 at 1:19 PM | Permalink

    LOL, grabs popcorn sits backs and watches the drama which was once known as “science”

  2. cbone
    Posted Nov 13, 2008 at 1:30 PM | Permalink

    This is quite interesting. It is quite comical the gyrations and contortions that Gavin goes through to avoid giving any credit to the skeptics who found this rather egregious error. Gavin is clearly in damage control mode, unfortunately for him he forgot the control part…

    • stan
      Posted Nov 13, 2008 at 8:04 PM | Permalink

      Re: cbone (#2),

      Gavin is clearly in damage control mode, unfortunately for him he forgot the control part…

      The control part was once part of Quality Control. When the Quality Control disappeared, unfortunately, they lost all control.

  3. Vincent Guerrini Jr
    Posted Nov 13, 2008 at 1:36 PM | Permalink

    Thanks to diligent audit and reporting by CA and AWUW and others, but mostly THE DATA, I sense a momentous change in the tone of the RC crowd.

    snip-policy

  4. RomanM
    Posted Nov 13, 2008 at 1:38 PM | Permalink

    That’s not nice, Steve! Now, Gavin’s gonna have to ask Tamino to teach him R. :(

    The script works like a charm. That’ quite a set of programs. Nice job.

  5. Posted Nov 13, 2008 at 1:55 PM | Permalink

    “…Can’t We All Just Get Along???…”

  6. AlanB
    Posted Nov 13, 2008 at 1:58 PM | Permalink

    Riposte from Gavin at RealClimate

    [Updated response: I originally mispoke in the above comment since the data is missing from the collated file rather than non-existent in any file. I'm happy to correct any mis-interpretation that might have caused. That collation is the responsibility of NOAA and any queries as to what goes into it or why should be directed to them. Now if people want to correct their insinuations that including Northern Canada data in the last update was akin to a shell game, we might be getting somewhere. - gavin]

  7. Steve McIntyre
    Posted Nov 13, 2008 at 2:00 PM | Permalink

    Columbia University defines plagiarism as follows:

    The appropriation of another person’s ideas, processes, results or words without giving appropriate credit.

    NASA spokesman Gavin Schmidt continues to appropriate “ideas” and “results” reported here without providing any “credit”.

    Here’s one more incident.

    Without providing any credit to this post, he changed the word “report” to “have …in the GHCN file”. He added – again providing no credit to the above post:

    [Updated response: I originally mispoke in the above comment since the data is missing from the collated file rather than non-existent in any file. I'm happy to correct any mis-interpretation that might have caused. That collation is the responsibility of NOAA and any queries as to what goes into it or why should be directed to them. Now if people want to correct their insinuations that including Northern Canada data in the last update was akin to a shell game, we might be getting somewhere. - gavin]

    He may be “happy” to correct any misinterpretation, but one more occasion, he has “appropriated” a Climate Audit result without providing “appropriate credit”.

    As to northern Canada, in the post in question, I reported that there had been no relevant change in the Canadian data between the data that I downloaded prior to the problem being noted and the “corrected” version. I asked:

    There was no new information from GHCN on the Canadian Arctic Islands. So what accounted for the sudden hot spot in the Canadian Arctic Islands?? Why can Hansen obtain values for October in the Canadian Arctic Islands today when he couldn’t on Monday? Maybe NASA spokesman Schmidt can explain exactly how Hansen “fixed” the problem.

    These are fair questions, particularly in the circumstances.

    And Schmidt provided an explanation, as I requested. He said that the webpage did not use Nov 10 data but what was in effect an already obsolete version downloaded the previous week. The Canadian data apparently entered the network between the time of their download and their posting of results. I don’t understand why they didn’t use an up-to-date version of the data, but that’s a different question. I promptly (at 11.30 am today) posted an update to the post acknowledging this as soon as an explanation became available.

    “Watch the pea”. I ask people to “watch the pea” all the time.

  8. AlanB
    Posted Nov 13, 2008 at 2:24 PM | Permalink

    Gavin responds:

    There are still (at least) four stations that have Oct data in place of september data but that didn’t report have september data in the GHCN file

  9. Old Dad
    Posted Nov 13, 2008 at 2:33 PM | Permalink

    Dear Steve,

    I think Mr. Schmidt needs to stop commenting on this issue. If I understand him correctly, he has no direct responsibility for the data in question, but he is a spokesman. He should immediately turn the investigation of the problem over to a qualified, internal team close to the problem, find out what really happened,and then speak.

    I have no reason to suspect malfeasance. My guess is simple incompetence. And the errors may or may not point to a larger problem withe the GISS data, but Mr. Schmidt’s serial bumbling on this data certainly doesn’t give me confidence.

    Point of information. As I understand NASA’s position on the GISS data. It’s an outsource–fair enough. It’s reported before all the data has been collected, and then updated and corrected as data comes in. OK. Are their adequate disclaimers on the early data warning us that the data may not be final?

    I’ll be the first to admit that I don’t understand the protocols of scientific inquiry, data archiving, storage, peer review, etc. But as a retired high tech business man, what I see strikes me as damn strange.

  10. plimple
    Posted Nov 13, 2008 at 2:56 PM | Permalink

    Steve,

    Reading Gavin’s statement at face value I took it to mean that he previously knew about the issue you’ve highlighted in your post but that he mischaracterised it using the word “report”. You’re assuming he read your blog to get this information but its possible he already knew this but “mispoke”. Indeed, NOAA don’t “report” September values because they don’t “have” them.

    I’m not sure that making the serious allegation of plagiarism is justified in any way.

    Your likening of the Canada data to the pea switching scam was, and is, a false mischaracterisation. Yes, you have acknowledged Gavin’s response graciously, but you’ve made no attempt (please, correct if I’m wrong) to adjust the tone in the bulk of that post away from insinuations of fraud, which is, again, a very serious allegation. If you’re going to paint the discussion in such a context its no wonder that things get polarised and Gavin gets shirty.

  11. Harald Bange
    Posted Nov 13, 2008 at 3:00 PM | Permalink

    I tried to post a copy of your Memo to Gavin on his RC site, but it hasn’t cleared his review …..yet

  12. ianl
    Posted Nov 13, 2008 at 3:04 PM | Permalink

    This comment is purely anecdotal, but by coincidence I was in Irkutsk on November 4-6 on project work. I arrived there from further east in Siberia, travelling mostly by motor vehicle and train.

    The more eastern area of Siberia was definitely on the cool side (about -10C and snowing lightly most of the time).

    As I progressed west towards Irkutsk, the on-ground snow and ice lessened progressively and Irkutsk (the largest of the Siberian towns) had very little on-ground. Irkutsk temperatures while I was there were around -3/-4C.

    My Russian colleagues commented that the Irkutsk snow seemed late this year, but that the 2007/8 winter was the severest they could remember.

    • ianl
      Posted Nov 13, 2008 at 8:51 PM | Permalink

      Re: ianl (#12),

      Well, there’s no discrepancy now on the ground.

      The night-time temperatures in the Irkutsk region,just one week later, are now down around -25C

  13. Posted Nov 13, 2008 at 3:08 PM | Permalink

    But Galvin doesn’t do GISTEMP, he only works in the same building. Poor Gavin. What the heck does he actually do?

  14. Sylvain
    Posted Nov 13, 2008 at 3:15 PM | Permalink

    <blockquoteBut Galvin doesn’t do GISTEMP, he only works in the same building. Poor Gavin. What the heck does he actually do?

    Grabbing on any occasion to complain about and appropriate the work of he who should not be name.

  15. SteveSadlov
    Posted Nov 13, 2008 at 3:21 PM | Permalink

    An end to end housecleaning at NASA, a stem to stern absolute upending, inclusive of questioning of every activity, every program and every individual empoloyee’s role and value add, was needed 20 years ago. NASA is so overdue for the mother of all reorgs.

  16. Posted Nov 13, 2008 at 3:37 PM | Permalink

    This is getting too good so I had to write something Global Warming Update: Russia Causes Climate Change. It probably won’t get picked up, but stranger things have happened.

  17. Tim
    Posted Nov 13, 2008 at 3:50 PM | Permalink

    This is my new official favorite website, I can’t wait to see what he tries next!

  18. Mark T.
    Posted Nov 13, 2008 at 3:52 PM | Permalink

    The epitome of “bureaucracy.”

    Mark

  19. Jeremy, Alabama
    Posted Nov 13, 2008 at 3:57 PM | Permalink

    Steve,

    You could do worse than run a script daily to (re-)check all these hockey team data sources. New data could be checked hourly for a few days. Any changes could be logged and published as a history trail, further elevating the already stellar reputation of your site.

    In other words, you could provide the data configuration management that they themselves refuse to do.

  20. Basil
    Posted Nov 13, 2008 at 3:57 PM | Permalink

    The pea has moved again. October is now down to 55.

  21. PaulM
    Posted Nov 13, 2008 at 5:00 PM | Permalink

    Gavin makes a complete fool of himself every time he opens his mouth.
    I particularly enjoyed his comment “The corrected data is up. Met station index = 0.68, Land-ocean index = 0.58, details here. Turns out Siberia was quite warm last month.”
    Neither he, nor any of his fan club at RC, nor any of the top climate scientists at GISS, thought to carry out even the most simple check on whether this was true or not.
    This sorry episode will lead to many more awkward questions about the GISS temperature record.
    That’s the trouble with molehills, Gavin. You start off with just one little one and before you know it they are springing up all over the place and your garden’s not looking so good.

  22. Posted Nov 13, 2008 at 5:01 PM | Permalink

    Gavin commented that for NASA to do their own QA would cost $500,000/ANNUM

    Steve should put in a tender. Nice work if you can get it.

  23. Steve McIntyre
    Posted Nov 13, 2008 at 5:22 PM | Permalink

    #21. Already obsolete. The data (digital) is now 0.64 and 0.55. Perhaps we should convert that to Dow Jones units.

    After temporarily touching an alltime October high of 8600, GISS plunged 2200 points to 6400, including a decline of 400 points since Schmidt posted at Watts Up. The most recent GISS figure does not include some irregularities reported today, the results of which are pending.

  24. Vincent Guerrini Jr
    Posted Nov 13, 2008 at 6:02 PM | Permalink

    I Still think this a momentous change on the part of Gavin. I think indirectly he is acknowledging the contribution of CA in this regard. It indicates that there is a new found respect for the audit (“Skeptical side”.. my words). Let it be, in might change things at NASA and save us a lot of wasted money in the long term.

    • Posted Nov 13, 2008 at 6:16 PM | Permalink

      Re: Vincent Guerrini Jr (#25),I Still think this a momentous change on the part of Gavin. I think indirectly he is acknowledging the contribution of CA in this regard. It indicates that there is a new found respect for the audit (“Skeptical side”.. my words). Let it be, in might change things at NASA and save us a lot of wasted money in the long term.

      The only thing Gavin is acknowledging is that there was a mistake. Not his, not Giss’ just a mistake. He separates himself from GISTEMP and blames NOAA. Gavin has done no wrong. The less you do the less your are wrong. He alludes to ModelE errors, but doesn’t get into details. Gavin needs to get a spine.

  25. Steve McIntyre
    Posted Nov 13, 2008 at 6:08 PM | Permalink

    GISS was off another 400 points today with a noticeable change around the Irkutsk sites, that were the topic of today’s post. Once again, NASA used the ideas presented at Climate Audit without providing “credit”.

    Jeez, I just noticed – northern Canada has reverted back to what it was on Monday. WTF??

    • craig loehle
      Posted Nov 13, 2008 at 7:58 PM | Permalink

      Re: Steve McIntyre (#26), It seems whenever I see an anomaly map, Siberia is always warmer. Sometimes the Midwest is warm, sometimes Europe is warm, but for a warm anomaly to stick around as a permanent feature of the globe seems simply bizarre—unless it results from unresolved urban heat island or other time-dependent biases.

    • Henry
      Posted Nov 13, 2008 at 8:32 PM | Permalink

      Re: Steve McIntyre (#26), northern Canada may have lost data on Thursday and the Yukon heated back up, but Australia has kept its new data and Britain still has its Wednesday cooling. Are they targeting someone in Canada?

  26. Posted Nov 13, 2008 at 6:25 PM | Permalink

    I don’t see this going after Gavin in a personal sense as some have taken the CA position, to me its just giving some enlightenment to how the whole team falsely comes off over there as infallible, on the whole. Its only comical because they always pretend to see the whole sphere of the situation, and they are purportedly the only ones that do, to see them drop the ball and them pretend they arent even playing the game is truly hilarious.
    Gavin may be the front man, but its just good to kick the whole organization in the pants once in a while. The speed at which the RC post came up was amazing as well, sometimes it goes weeks over there without a new post. That speed shows the importance of RC to the NASA crew as (mis)information tool, and that is scary to have an arm of government running so loose with a mandate, with no lawful controls in place over its actions.

    • Posted Nov 13, 2008 at 6:37 PM | Permalink

      Re: Max (#28),

      The front man is Jim Hansen. Gavin is filling in trying to separate himself from the issue. He is using a blog not the official GISS site to communicate. THIS IS A FRIGGIN’ GOVERNMENT ORGANIZATION! If Gavin has nothing to do with GISTEMP why the hell is he saying anything? Where is big Jim? Please get the irony!

  27. Paddy
    Posted Nov 13, 2008 at 7:58 PM | Permalink

    A commenter to the previous post stated: “There is a new law called the Data Quality Act(DQA) which is supposed to stop these type of shenanigans.” He is correct. To remedy the admitted lack of compliance, a friend such as Senator Inhofe, should request the Inspector Generals of NOAA and NASA to investigate these agencies’ failure to vet and assure the quality of their public output. The use of a private blog like RC to provide suspicious (false) explanations is unethical. The public is entitled to receive official, transparent and verifiable data and explanations for their failure to do so.

  28. Dishman
    Posted Nov 13, 2008 at 9:51 PM | Permalink

    I don’t think Gavin has done much to set the tenor.
    I believe far more responsibility for that falls to Dr. Hansen.
    Perhaps this would be a good time for Gavin to look at the tone set by his boss, and contemplate how that would look to people who don’t think he’s always right. Perhaps he could try to think of himself being on the outside, looking in.

    Perhaps he might find some new understanding through this event.

  29. Bob Koss
    Posted Nov 13, 2008 at 11:04 PM | Permalink

    This might be slightly off-topic since it’s not about the current year of data, but it certainly is a curiosity. Move if you think appropriate.

    The GISS Australian data that is missing for 2006-07 is so standardized it is hard to believe.

    80% of the 43 stations have no annual temperature for 2007 and those stations have the same missing months of data. Jan, Feb, Mar, May. They all have Apr.

    2006 has annual temperatures for 80% of the stations, but Oct, Nov, and Dec are missing. The 20% that have no annual temperature are missing almost identical months.

    Here is a spreadsheet for Australia 2006-07.

    Does anyone know if Australia uniformly shut down most of their network for extended periods over the previous two years?

    Steve: Good post. The NA distribution is insane.

  30. Bob Koss
    Posted Nov 14, 2008 at 1:38 AM | Permalink

    I just checked Australia in GHCN and it is the same as GISS for 2006-07. All 43 stations in 2008 have essentially complete data for the year. Wonder what happened in 06-07.

  31. Patrick M.
    Posted Nov 14, 2008 at 6:25 AM | Permalink

    If you look at the “Watch the pea thread” you can see that Australia heated up at the same time as North America. Now NA has gone back to where it was but Australia has not.

  32. Sam Urbinto
    Posted Nov 14, 2008 at 3:13 PM | Permalink

    As I asked on the BB, what is in store for October, November and December for the anomaly?

    The last 10 years were

    1998 .447
    1999 .320
    2000 .217
    2001 .540
    2002 .453
    2003 .610
    2004 .573
    2005 .647
    2006 .637
    2007 .470

    Are we in for another 1984? A 1992? A 1997? Well it’s pretty sure it will be between 0 and .65

    And really, what’s +/-.325 amongst friends.

  33. KimberleyCornish
    Posted Nov 15, 2008 at 4:11 PM | Permalink

    Working on the principle that (even in the post-communist era) no information emerges from Russia without scrutiny and approval by the Russian intelligence agencies, is there any way that reports of a warming Siberia work to the political advantage of Russia? That is to say, do the Russians have a political motive to massage reported temperature data upwards? (I am not saying that they do, but would be interested to see feedback.)

  34. Posted Nov 17, 2008 at 12:14 AM | Permalink

    Steve

    There is something that is incongruous about the Siberian warming. If you have watched the march of the Arctic ice on a daily basis this year (I have) the Arctic ice marched like Sherman to the coast from the end of September till about the end of October when it completely filled the region abutting the Russian northern mainland. Here are the trendlines from Cryosphere.

    Lapatev Sea

    http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/recent365.anom.region.8.html

    Kara Sea

    http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/recent365.anom.region.7.html

    East Siberian Sea

    http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/recent365.anom.region.9.html

    Chukchi Sea

    http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/recent365.anom.region.10.html

    Barents Sea

    http://arctic.atmos.uiuc.edu/cryosphere/IMAGES/recent365.anom.region.6.html

    Even though there is an October or early November “blip” in the ice area, each of the specific areas sans one on the Russian side of the Arctic is hundreds of thousands of square miles more in extent than at this time last year and in aggregate is over 1 million square km larger in extent than last year. Yet Siberia is shown as much warmer than the rest of the Arctic regions?

    Something is clearly amiss here.

  35. bender
    Posted Dec 21, 2009 at 6:36 PM | Permalink

    Kirensk, mentioned above, is one station fingered by Kamel to be wholly inconsistent with Jones’s Siberian data. Comments, Gavin?

One Trackback

  1. [...] “Jeez, I just noticed – northern Canada has reverted back to what it was on Monday. WTF??” (Post [...]

Follow

Get every new post delivered to your Inbox.

Join 3,381 other followers

%d bloggers like this: