Climate Audit

Luboš Motl

Congrats, Steve, to all these things. I guess that Schmidt being a NASA spokesman is one of the jokes I haven’t quite understood. I suppose David Mould is now the NASA spokesperson and the Hansen folks haven’t yet overrun space shuttles and similar gadgets – that would be more scary for the US astronauts than their jealous colleagues.

Fred

still seems to me the most significant part of the whole story is the exposure that NASA is not using raw data, but rather adjusted data to produce their graphs and predictions. Because they refuse to share with the rest of the world how the adjusting algorithms (Al Gore-ithms ?) work, they are effectively saying they reject Scientific Method 101 and embrace Faith Based Science.

People should be very worried when the High Priests of the Temple start operating on a “Trust Me” basis

Allan Ames

Steve, Congratulations. Since every event that supports AGW seems to get press, press is certainly appropriate for the situations that are exposed as errors. And while the press gets much wrong, it knows a David vs. Goliath story when it sees one.

Hans

“Nasa climate change error spotted by blogger” [Steve, feel free to remove this or move it Unthreaded, if you want – Hans]

The TELEGRAPH paper in the UK names Steve McIntyre as:
“A US blogger” who “has caused a stir in the climate debate by forcing Nasa scientists to admit errors in some of their data showing increases in global warming.”

If you want to read this, it is at:
http://www.telegraph.co.uk/earth/main.jhtml?xml=/earth/2007/08/16/eaclimate116.xml

The article says that “Amateur meteorologist Steve McIntyre, who has in the past challenged ‘the hockey stick’ model of climate change data used by green campaigners, emailed Nasa suggesting there were anomalies in their data.”

I will let you read the rest yourself but this bit (aside from the US blogger bit) makes me laugh:
“Dr Gavin Schmidt, a climate modeller at GISS and author of the website RealClimate.org, described the changes as “very minor re-arrangements”.”

Kenneth Fritsch

Re: #4

still seems to me the most significant part of the whole story is the exposure that NASA is not using raw data, but rather adjusted data to produce their graphs and predictions.

I doubt that that exposure will mean much to the general public, but to those of us with a closer interest there are several revelations about US and global temperature measurements (in their final and adjusted forms), that I have learned by participation at CA and remain a bit puzzling to me:

That adjustments are relegated to the US.

That the amount of the adjustment in the US is comparable to the warming anomaly.

That the details of the means to make adjustments are not known outside those doing it.

That the assumptions of station quality control, or a lack thereof, are not revealed or apparent.

Why, if adjustments are required and carried out for US measurements and are a significant portion of the warming trend, they are not considered necessary for other nations?

That GISS evidently has determined it must provide its own adjustments to USHCN data sets that are adjusted by their own methods.

M. Simon

#11 Francois,

Bridges are built with a lot of safety margin because it is hard to calculate all the stresses or problems caused by shoddy construction.

Since every one knows this, there is the tendency to ‘shave’ – missed a quality inspection? Do the paperwork, the safety margin will take care of it…

For a flying machine there is less margin of error or safety factor.

B. Stroeher

There is an extensive article in the “DER SPIEGEL” in German language.

Date 16. 8. 2007

It is more in favour of Steve, especially also on the “Hockey-Stick”.

Try to translate it.

http://www.spiegel.de/wissenschaft/natur/0,1518,500274,00.html

Dodo

Much ado about “nothing” continues at Real Climate: they seem to have taken notice of remarks concerning the GISS global temperature history graph. It has been removed from the Aug. 10th blog entry, apparently for repairs.
http://www.realclimate.org/index.php/archives/2007/08/1934-and-all-that/

Gavin Schmidt seems to allow a quite open debate about the temperature history. And judging from his many remarks to skeptical comments, it seems to be a full-time occupation for him, just like CA is for Steve. As a European, one must thank the American taxpayers for providing us with such a service. Let’s hope the Canadian government starts paying Steve.

Adrian G

Good job spotting the Errors Steve, and I’m also glad the site is up and running again.

The response to Steve’s findings are most interesting. Perhaps the changes are
only ‘minor re-arrangements’, but this is not the point in my view. The point is the quality
of the data and its analysis. Starting with the MBH Hockey stick, we have seen that the data and methods
are not checked or audited. I know that this is the norm in most academic cases, but when the
results (eg. MBH 98) become the cornerstone of the IPCC and cause billions of dollars of expediture, then
the data and methods must be carefully checked.

Now the NASA errors appear, and it makes me wonder which research is reliable. What we need
is complete transparancy with the research.

I seem to remember, then when I studied science in the University, all my methods and data
were rigourously checked and analysed. It was virtually impossible for me to fudge or
get any consistencies past my teachers. We were also taught that a viable theory must be reproducable..

Filippo Turturici

Would it be possible that Steve make a similar research for Europe?

E.g. Udine-Campoformido, North-East Italy, used by GISS, has totally wrong data since 2004, using data just from 8AM to 5PM for making averages, so overestimating mean monthly temperatures (overall during summer months, when overestimate is 2-3°C).
Other stations data seem more correct; but, e.g. Verona-Villafranca airport, used by NOAA, in the last years is often warmer than the corresponding rural station, and it is often among the warmest (both for values and for anomalies) of all North Italy.
There is also a problem of average value: as for USA, here rural stations show nowadays mean temperatures not above ’30-’40ies mean values, with a temperature drop between about 1955-1960 to 1990-1995 (laso, a strange case, plain stations show the chillest decades in ’70-’80ies, while mountain ones show them between ’50ies and ’70ies, with the ’80ies warm almost as ’90-’00ies); and, NOAA use a 1968-2006 average, while many Italian and European graphs and maps based on NOAA archive data start just from 1948 (just at the beginning of cooling).

M. Simon

Jeff,

Newspapers (and by extension reporters) are always claiming to have been duped by politicians especially (I will name no names). How much easier to take the voice of authority in science.

Expect to see a lot of hand wringing about the inadequate training of science reporters about 6 months to a year after the public consensus on global warming breaks.

Reporters no longer have the “Your mother says she loves you? Check it out” attitude. That is the real problem.

Al Gore gets an Oscar for a “documentary” whose facts are not holding up well. Isn’t that a big deal? You know the crickets are lovely this time of year. You can hear them so clearly.

There is seepage. And the ‘net. There will come a break.

Sam Urbinto

Hmmm. That 1934 thing is rather confusing so let me try and summarize the bit about the adjustment code. The point of view of Gavin over there seems to be twofold, overall, after clarifying his early views a few times.

One is that all their code (“badly written fortran”) for doing this is undocumented and a mess, and that many of the calculations are done by hand or seperately or something (“by one person”), so there is no 1 piece of software to get, and it would take too much time (that they don’t have) to turn it into something useful. And that no need to verify them more than they have been already. It has to be demonstrated they don’t do what they’re supposed to, because it already has, before they would put together a ready to run software package for this purpose.

The other is that the papers explaining what they do to the data (web site, 2 Hansen et al papers). So it’s trivial to recreate the methods and in doing so, it would verify everything to just implement theirs in a different way and see if everything matches. That everything described in English can be understood better than obtuse software. Replicate it! You’ll get more out of it, that’s the message.

As far as things already being validated, speaking about something that was done by Eli (in “There’s a hot time in Marysville or how not to RTFR”) as revealing that

a) the GISS adjustment is a simple two-piece linear trend and b) the urban trends after adjustment are the same as the rural trends. That’s prima facie evidence that the code does what it claims to do.

I’m unsure of where exactly the reverse engineering is on that blog post there, but maybe I don’t know what to look for…

My comments to this all is if the code is in the state it is said to be, how do they even know what it does or if it’s accurate? I’m not saying they don’t know, or that it isn’t accurate, but that sounds strange to me. If it’s so easy to recreate the methods they use, why aren’t they at least detailed out as pseudo code in one location? Then that’s the other strange thing, if it’s so easy to implment, why don’t they just have one program themselves? If it’s a simple two-piece linear trend, why do they have working codes that are “a mixture of scripts, programs, utilities, dead ends, previous methodologies and unused options”

Additional strange things are why do they seem to be so adverse to just giving everything out as is, whatever the condition, and why should there be some requirement to ‘demonstrate’ it hasn’t been validated ‘enough already’, or as he says “there is no reason to think there are problems in the GISTEMP analysis”. Then he says it’s “strong testimony that the calls for greater access are simply rhetorical tricks” and that it’s a fact that “on-one has attempted to replicate the analysis from the published descriptions.” (Rather than re-do or reverse engineer, I take it.)

For some strange reason, I get the idea this issue isn’t that straightforward to implement after all.

steven mosher

RE 25 & 26

Ya think I’m getting under gavins skin yet?

I still have 1/4 streamers of ugly ass NASA code stuffed away in boxes in my garage.
Ugly Fortran doesnt scare me. I survived untangling that junk. It put you in a Zen
state. It’s like knitting, only backward, with your brain…

You should see the stuff he snipped… Pittman You need to get on an FOIA request.
Nasa has some interesting policies on Software and the Cheif engineer and CIO have
some interesting responsibilities…

I’ll see If I can find the links again.. but google nasa software policy and follow
your nose…

Steve McIntyre

Given that Schmidt is acting as a NASA spokesman on this issue and that NASA has obviously acqueisced in Schmidt’s defending NASA as a “private citizen” at realclimate, how appropriate is it for a NASA spokesman to be censoring criticism of NASA?

David Stockwell

#26 John, I posted this in another thread – too many of them now – but the aspirated sensor problems were clearly known about in 1993 when this was published. Kessler on HO-83 in Albany NY http://adsabs.harvard.edu/abs/1993BAMS…74..215K

steven mosher

RE 28.

YUP. That is the doc I found the other day. there are other related documents as
well. I believe I mentioned this in one of my comments back to gavin and I think he snipped it.

Gavin says the code is an undocumented mess. I’m wondering if the code has been through IV&V?

My sense is Dan Hughes should be all over this.

Also this.

http://opensource.gsfc.nasa.gov/

And this.

http://www.whitehouse.gov/omb/circulars/a130/a130trans4.html

Let me see what Else I can come up with this weekend and work from there.

John F. Pittman

#28 Steve.

Sorry, but we don’t need a thread because of the following

NASA Policy Directive 2820.1C NASA Software Policy
Responsible Office: Office of the Chief Engineer, Michael Ryschkewitsch

Also, this office (office of chief engineer)(4) Release software in accordance with NPR 2210.1, External Release of NASA Software, consistent with law and applicable agreements, for commercial, industrial, educational, and governmental purposes.

From NPR 2210.1

h. In general, Software, as defined in Section 2.1, is not considered a record under the Freedom of Information Act (FOIA) and, therefore, is not subject to the mandatory release requirements of the FOIA. Requests for software under the FOIA should be coordinated between the Center FOIA Office and the Center Software Release Authority.

John F. Pittman

This has promise. However, it does not have the legal track record of the FOIA. I think that it would be appropriate to ask the Office of Chief Engineer how his department plans on meeting this OMB requirement, and quote k below if information is not made available. Especially something like Hansen or Mann code where the public is being asked to spend billions, and definetly has a stakeholder issue in making sure that the information not only meets NASA and OMB’s requirements, but that it is available for the public.

http://www.whitehouse.gov/omb/circulars/a130/a130trans4.html
From this OMB:

Government information is a valuable national resource. It provides the public with knowledge of the government, society, and economy — past, present, and future. It is a means to ensure the accountability of government, to manage the government’s operations, to maintain the healthy performance of the economy, and is itself a commodity in the marketplace.

The free flow of information between the government and the public is essential to a democratic society. It is also essential that the government minimize the Federal paperwork burden on the public, minimize the cost of its information activities, and maximize the usefulness of government information.

4. Applicability and Scope:
a. The policies in this Circular apply to the information activities of all agencies of the executive branch of the Federal government.
b. Information classified for national security purposes should also be handled in accordance with the appropriate national security directives. National security emergency preparedness activities should be conducted in accordance with Executive Order No. 12472.
a.
1.
2. How must agencies implement Records Management?
Agencies will:

(a) Ensure that records management programs provide adequate and proper documentation of agency activities;

(b) Ensure the ability to access records regardless of form or medium;

(c) In a timely fashion, establish, and obtain the approval of the Archivist of the United States for retention schedules for Federal records; and

(d) Provide training and guidance as appropriate to all agency officials and employees and contractors regarding their Federal records management responsibilities.
3. How must an agency provide information to the public?

Agencies have a responsibility to provide information to the public consistent with their missions. Agencies will discharge this responsibility by:

(a) Providing information, as required by law, describing agency organization, activities, programs, meetings, systems of records, and other information holdings, and how the public may gain access to agency information resources;

(b) Providing access to agency records under provisions of the Freedom of Information Act and the Privacy Act, subject to the protections and limitations provided for in these Acts;

(c) Providing such other information as is necessary or appropriate for the proper performance of agency functions; and

(d) In determining whether and how to disseminate information to the public, agencies will:
(i) Disseminate information in a manner that achieves the best balance between the goals of maximizing the usefulness of the information and minimizing the cost to the government and the public;
(ii) Disseminate information dissemination products on equitable and timely terms;
(iii) Take advantage of all dissemination channels, Federal and nonfederal, including State and local governments, libraries and private sector entities, in discharging agency information dissemination responsibilities;
(iv) Help the public locate government information maintained by or for the agency.
(i) Establish and maintain communications with members of the public and with State and local governments so that the agency creates information dissemination products that meet their respective needs;

(j) Provide adequate notice when initiating, substantially modifying, or terminating significant information dissemination products; and

(k) Ensure that, to the extent existing information dissemination policies or practices are inconsistent with the requirements of this Circular, a prompt and orderly transition to compliance with the requirements of this Circular is made.

fFreddy

There is a typo in Chris’s link above; this one works.

Chris is right, it’s a good article. I particularly hope that Townhall readers read through to the end of the transcript where Steve describes the farcical lack of disclosure from climatologists.
(I refuse to call them climate scientists.)

bernie

fFreddy:
You mean climatologists are like phrenologists!

fFreddy

bernie, exactly so, though I was thinking of numerologists …

Jack

Fumento writes in the article:

If you follow the global warming debate, you “know” that nine of the ten warmest years recorded in the U.S. lower 48 since 1880 have occurred since 1995, with the very hottest being 1998.

But whaddya know! Those figures are wrong. Data from NASA’s Goddard Institute for Space Studies (GISS) now show the hottest year since 1880 was 1934. Nineteen-ninety-eight dropped to second, while the third hottest year was way back in 1921. Indeed, four of the 10 hottest years were in the 1930s, while only three were in the past decade.

Reading the discussion comments here, it seems that everyone is still congratulating Steve on his discovery of the NASA glitch. But doesn’t Fumento still get the difference between the U.S. data and the global data wrong in what he writes above? Even before Steve’s discovery, in the GISS analysis of the U.S. lower 48, nine of the top 10 warmest years did not occur after 1995. That was true globally but not in the U.S. mainland. Is Fumento engaged in some clever writing here?

Even though the data needed correcting, I can see why Hansen’s ire is piqued given the continued recurrence of this conflation.

Douglas Hoyt

I thought Gore was claiming that 9 of the top warmest years occurred after 1995.

JS

Jack # 45,

The error in the NASA data Steve M found is nothing compared to the errors in the data from microsite effects and adjustments. That’s why Hansen, CRU, NOAA etc are trying to hide their computer codes and algorithyms. But this is the free part of the world, it won’t be long.

bernie

Jack:
I agree. Precison and accuracy are what I am looking for regardless of the commentator’s overarching position re AGW. Ultimately, I for one will tip my hat to Hansen et al IF their conclusions are supported by the data. Fumenta is wrong on a number of points in his article and should issue a correction/clarification. Besides it would be excellent PR for those seeking to audit the data.

Sarah

It appears that word of the correction has not got out yet.

In a story today The independant is still scare mongering about how all the hottest years have occured in the past 20 years.

http://news.independent.co.uk/uk/transport/article2878783.ece

Jack

Steve:

Somebody must have seen the movie! (Chuckle).

Amanda Carpenter made a similar mistake. Among others. One boggles at the lack of comprehension of this rather simple distinction (probably due to the urge to “make a point” before checking one’s facts… we’ve seen that demonstrated frequently).

You, on the other hand, have been quite circumspect about what was corrected. One could wish that the pundits, from Limbaugh on down, had been as careful.

Thanks for taking note of this.

Michael Jankowski

The cynical ones out there (like me) would think GISS and Gore would sing a different tune if it were the US record which had all the warmth. We’d hear about how the US record is the best in the world, how large a land mass the US is, etc.

Anyhow, here’s Gore before the US Senate saying 2006 was the warmest year on record in the US (apparently based on NCDC as opposed to GISS?).

http://epw.senate.gov/public/index.cfm?FuseAction=Files.View&FileStore_id=e060b5ca-6df7-495d-afde-9bb98c9b4d41

Sam Urbinto

Oooops, sorry steven, wrong person I see, no W.

Regarding my #25, #26,27, well of course that’s just something Gavin Schmidt said, which to me is not about the methods of doing so that he’s actually talking about. (And my post was just mainly a summary of what he said as things went along. If you read the entire thing at RC, you’ll see he spends what seemed to me to be a lot of time changing the reason for not giving up the code, consistent with my belief nothing anyone said or did would make a difference, which I why I never bothered with it) One person even says over there “Why do something original, when you can throw darts at the work of someone else?”!!! (Excuse me, an audit of the code IS something original. Just because Gavin wants Steve to replicate it instead and won’t provide it isn’t Steve’s fault.)

That quote you talked about, I got this as the point, that ‘a) and b) prove the code works, story over.’ Couple that with the idea “there is no reason to think there are problems in the GISTEMP analysis”, and you get those two means that ‘there is no reason to look at the code and so I’m not giving to you. Seems a rather poor excuse, even if a)&b) do prove the code works and regardless if there is no reason to think the analysis has any issues or not. So I’m just ignoring the ifs because I don’t need them to be true or false to think that’s a poor excuse anyway, even if “code works” means “code validated” and vice versa.

Regardless, from what you’re saying, a)&b) don’t prove the code works. My contention is that they aren’t equal anyway (output versus how), so showing the code works (or not) is meaningless. What “the output is” isn’t equal to “how is it written and what does it do exactly”.

It seems obvious to me that the end result is what the start goal was: ‘I am going to do anything and make any excuse I need to as long as I can justify not giving the code out. If I get backed into a corner I will either leave, ignore, edit or repeat an earlier point.’ Or at least it seemed to me as I read along the explanations and excuses. (It didn’t just start out ‘build your own method and compare’ it developed into it.)

For example, this took 350 comments (plus any deleted ones!) to develop into, and it’s still just poor excuses gathered over the course of the “discussion”!

We publish hundreds of papers a year from GISS alone. We have more data, code and model output online than any comparable institution, we have a number of public scientists who comment on the science and the problems to most people and institutions who care to ask. And yet, the demand is always for more transparency. This is not a demand that will ever be satisfied since there will always be more done by the scientists than ever makes it into papers or products. My comments above stand – independent replication from published descriptions – the algorithms in English, rather than code – are more valuable to everyone concerned than dumps of impenetrable and undocumented code.

What kind of silly excuses are these? We give out a lot of other stuff, so we’re not giving McIntytre the code. People always want more so we’re not giving McIntyre the code. The papers describing what we do are enough, so we’re not giving McIntytre the code. The dumps are impenetrable and undocumented, so we’re not giving McIntytre the code. Algorithms not in code or pseudocode but described are more valuable, so we’re not giving McIntytre the code.

And the inherent message “Redo it yourself and if you get the same results the code works, we’re going to ignore that you want to audit the code and algorithms not see if you can get the same output we do. We don’t want to give you the code, so we’re not going to.”

I will give Gavin Schmidt a nod for answering all the posts he left up there, but unfortunately the excuses just don’t hold up.

Jack

Thanks, Steve Mosher. Clearly a reference to the global record. I had to do quick checks; it was easy to find NOAA’s U.S. ranking for 2005, which was 13th. Certainly not 1st or 2nd (GISS and NOAA, respectively, for global temperatures).

BarryW

I’m beginning to think that the problem with the code is not that the won’t give it to you, it’s that they can’t give it to you because it doesn’t “exist”. I worked in the air traffic control R&D world for many years, and often when we analyzed problems (and I’m not proud of this) we would write quick and dirty routines or scripts to do the data processing. Sometimes those throwaway programs would wind up being used for years to process data by other users (“hey Frank can I borrow that script you had for processing arrivals?” ). My guess is that you would find a Unix shell script here, a Perl script there, some Fortran, and g*d knows what else, probably some obscure readme files or knowledge handed down verbally. Even if it worked perfectly well they would be embarrassed to show it to anyone outside of their shop.

fFreddy

Re #64, Jack

Which I don’t even understand. Mechanisms?

Presumably, mechanisms of “adjusting” raw temperature measurements to produce the reported temperature history.