Carnage

Feb 5, 2009 – 11:10 PM

Moving right along since the problem with Harry was identified on Super Bowl Sunday, BAS reports:

This is a list of corrections that have been made to the AWS data tables and a link to the table before the corrections were applied, any suspected errors should be reported to Steve Colwell

(2/2/09)The AWS data for Harry have been corrected after is was reported by Gavin Schmidt that data from Gill had been added where data for Harry did not exist. The incorrect data file for Harry temperatures can be accessed here

(4/2/09)The AWS data for Racer Rock since April 2004 have been removed from the READER website as the values appear to come from a different station even though they were transmitted on the GTS (Global Telecommunications System) as 89261 which the WMO (World Meteorological Organization) still list as being Racer Rock. The incorrect data file for Racer Rock temperatures can be accessed here

(4/2/09)The AWS data for Penguin Point since January 2007 have been removed from the READER website as the values received on the GTS appear to come from a different station and this AWS is reported as being removed at the start of 2007. The incorrect data file for Penguin Point temperatures can be accessed here

(4/2/09)The AWS data for Clean Air since January 2005 have been removed from the READER website as the values received on the GTS appear to come from a different station and this AWS is reported as being removed at the start of 2005. The incorrect data file for Clean Air temperatures can be accessed here

I compliment British Antarctic Survey for being professional in dealing with the situation. Clean Air is in Steig Table S1, Harry in Steig Table S2. Penguin Point and Racer Rock are in neither table. But it seems not beyond imagination that the exclusion of bad data might make them eligible for Table S1 or S2 – whatever the eligibility criteria (and I don’t pretend to have any idea how eligibility for these exclusive clubs is determined.) So the mere fact that they did not contribute to the present recon, if they didn’t, doesn’t mean that they shouldn’t have – once the records are corrected. Time will tell.

While Steig has asserted that I should know the impact of these various errors, I don’t. Maybe they’re negligible, maybe not. We’ll find out. In Steig’s shoes, as I’ve mentioned before, I would deal with present situation by placing end-to-end source code online so interested parties can readily assess the impact of these errors for themselves. If he’s certain that the errors “don’t matter”, that’s the best way to extinguish speculation.

Being stubborn about this is going to be unsustainable and he might as well get it over with.

Just my opinion.

Update: See here for a list of 6 surface (nor AWS) stations where problems of one sort or another are pending.

Reblog this post [with Zemanta]
By Stephen McIntyre | Posted in Antarctica, Steig at al 2009 | Tagged , , , , , | Comments (152)

CA struggles under load #94

Feb 4, 2009 – 5:42 PM

For the umpteenth time (as long time CA readers will know) the blog is under heavy assault from people whose only wish is to read the latest from Steve McIntyre unfiltered.

As a consequence, the MySQL db has caused lots of timeouts as the number of readers hits the maximum number of connections (which were probably the defaults) and the maximum numbers of lots of other parameters.

Mr Pete and Anthony have both been on the case and have added more hamsters onto the treadmill to keep CA running. They do ask for your patience – if CA appears to be down, give it 30 minutes and then try again – they are working on this issue.

You can see from our stats, the graph looks eerily familiar…

By John A | Posted in General | Tagged , , | Comments (57)

Antarctica – digging out the data

Feb 4, 2009 – 2:35 PM

It seems that we are all “wild about Harry” recently, and no good kerfluffle would be complete without some pictures of the weather stations in question. It seems “Harry” got buried under snow. Why is this important? Well, as anyone skilled in cold weather survival can tell you, snow makes an excellent insulator and an excellent reflector. Snow’s trapped air insulative properties is why building a snow cave to survive in is a good idea. So is it any wonder then that a snowdrift buried temperature sensor, or a temperature sensor being lowered to near the surface by rising snow, would not read the temperature of the free near surface atmosphere accurately?

As I’ve always said, getting accurate weather station data is all about siting and how the sensors are affected by microclimate issues. Pictures help tell the story.

Here’s Harry prior to being dug out in 2006 and after:

Harry AWS, 2006 – Upon Arrival – Click to enlarge.

Harry AWS, 2006 – After digging out – Click to enlarge.

You can see “Harry’s Facebook Page” here at the University of Wisconsin

It seems digging out weather stations is a regular pastime in Antarctica, so data issues with snow burial of AWS sensors may be more than just about “Harry”. It seems Theresa (Harry’s nearby sister) and Halley VI also have been dug out and the process documented. With this being such a regular occurrence, and easily found within a few minutes of Googling by me, you’d think somebody with Steig et al or the Nature peer reviewers would have looked into this and the effect on the data that Steve McIntyre has so eloquently pointed out.

Here’s more on the snow burial issue from Antarctic bloggers:

Continue reading →

By Anthony Watts | Posted in Antarctica, Steig at al 2009, Surface Record | Tagged | Comments (78)

Gavin’s “Mystery Man” Revealed

Feb 4, 2009 – 9:40 AM

On Monday, Feb 2, Gavin Schmidt explained some “ethics” to realclimate readers as follows:

[Response: People will generally credit the person who tells them something. BAS were notified by people Sunday night who independently found the Gill/Harry mismatch. SM could have notified them but he didn’t. My ethical position is that it is far better to fix errors that are found than play around thinking about cute names for follow-on blog posts. That might just be me though. – gavin]

As readers know, I was interested in who was the scientist that, unbeknowst to me, had “independently” identified the problem with Harry – a problem overlooked by BAS, NASA GISS for a year or so anyway; and a problem which had been missed by his realclimate coauthors, Steig and Mann, during their preparation of Steig et al 2009, and which had been missed by the Nature peer reviewers. And remarkably this had been “independently” identified just after I had noted the problem at Climate Audit and Climate Audit readers had contributed ideas on it, even during the Super Bowl.

Yesterday, I inquired about the identity of Gavin’s “mystery man”? Today (Feb 4) the British Antarctic survey revealed the identity of Gavin’s “mystery man”. It was… Continue reading →

By Stephen McIntyre | Posted in Steig at al 2009 | Tagged , , , , | Comments (408)

Gavin’s Mystery Man

Feb 3, 2009 – 1:29 PM

On Sunday, Feb 1 at 4:41 pm Eastern (3:41 blog time), I published a post describing West Antarctic stations. In that post, I observed that there was a very limited amount of station data (6 stations) in the West Antarctica area re-interpreted by Steig et al, that one of the stations (Harry AWS) had an extreme trend (0.81 deg C). I’d noticed some peculiar features about this series, but it was getting late in the day, it was Sunday and I had family things to do in a while, so I ended my post as follows:

Stay tuned for some interesting news about Harry.

The two main things that I’d noticed by then had been 1) a huge difference between the GISS version of Feb 2008 and the current version; and 2) that Harry had been installed in 1994, making the provenance of Harry data prior to 1994 a mystery – which I asked readers to think about around 5 pm Eastern (4 pm blog time). At the time, I said (more than once) that I didn’t know (“dunno”) whether the problems with Harry “mattered” – and that figuring that out would be hard without examining exact code.

The post attracted a fair amount of reader interest. At 5:19 Eastern (4:19 blog), reader Dishman speculated that the problem with Harry might come from the Argos number of 8900 moving around. I kept track of things for about an hour and around 6 pm Eastern (5 pm blog), I reported on scraping some source data from Wisconsin. By then, it was supper time. I started getting ready for Sunday dinner and watching some football. (Super Bowl is not a religious holiday in Canada, as it is in the States, but I do watch a lot of sports – less than usual because the Toronto Raptors basketball team are playing so poorly this year.)

The comment section of the thread was quite lively through the Super Bowl. I did a little work late in the evening, checking in just after midnight, noticing by that time that the provenance of the pre-1994 Harry data was “Gill” and being pretty sure by this time that BAS “Harry” was a splice of Harry and Gill. A CA reader had made a similar observation by the next morning (Feb2).

During the morning of Feb 2, I re-did my calculations, re-scraped the data, went over things again and wrote up a post on Harry, which I released at 1:49 pm Eastern (12:49 blog time). As I’ve explained from time to time, I often use the blog as a sort of diary – so that writing a blog post on Harry wasn’t distracting me from other analysis: it provides detailed documentation of the analysis that there was a problem with Harry, that I could refer to later. (I know that some readers aren’t interested in this sort of thing and it creates an editorial unevenness here, but obviously enough readers like it to create an audience.) In that sense, writing a blog post didn’t materially delay the reporting of the problem; it ensured that I’d documented the issues thoroughly.

As I was finalizing my post, I re-checked the BAS page and found that the Harry data had been changed – a point which CA readers had noticed a bit earlier, when I re-checked the thread.

Over to RC.

Their Antarctic thread had been closed, but they re-opened it. A couple of hours after my post (3:35 pm), bernie commented here as follows:

SM at CA has identified what appears to be a major error in the Steig et al paper that suggests that the perceived trend is an artifact of this particular error. Perhaps this is an opportunity to mend some fences and work towards a common goal of better data and clearer methods.

Gavin:

[Response: No-one should be against better data. It would have been nice had SM actually notified the holders of the data that there was a problem (he didn’t, preferring to play games instead). If he hadn’t left it for others to work out, he might even have got some credit 😉

A couple of hours later, Steve Reynolds:

Isn’t it rather petty (as well as possibly unethical) to refuse credit because you don’t like the source or their methods of communication?

Gavin:

[Response: People will generally credit the person who tells them something. BAS were notified by people Sunday night who independently found the Gill/Harry mismatch. SM could have notified them but he didn’t. My ethical position is that it is far better to fix errors that are found than play around thinking about cute names for follow-on blog posts. That might just be me though. – gavin]

An hour later, Steve Mosher:

RE 148: gavin, Steig asked SM not to communicate with him anymore.

Gavin:

[Response: This data error has nothing to with Steig or any of his co-authors. SM should have contacted BAS. I’m sure BAS are grateful that someone told them about the data mess up – it just didn’t happen to be SM.

The Harry error had been sitting in the BAS data for maybe a year. None of the authors of Steig et al authors nor any of their reviewers had noticed the error. And then remarkably both me and Gavin’s mystery man “independently” found the Gill/Harry mismatch within a couple of hours of each other, with Gavin’s mystery man, by sheer coincidence, doing so just after I had published notice of the problem at Climate Audit and after Climate Audit readers had turned their attention to the problem.

While Gavin complained about me not immediately contacting data authorities, I actually have a pretty good record of notifying data authorities of things that I’ve noticed. I sent Hansen an email on the Y2K problem when I identified it, as well as, on the October 2008 Siberia problem when I was aware that this had been noticed by a CA reader (and, as it turned out, by a Watts Up reader as well). I’ve notified WDCP of problems in dendro data. As of Sunday supper time, while I knew that there was something about Harry, I hadn’t fully diagnosed it and it didn’t seem like something that I needed to notify BAS about urgently – particularly when the incorrect data had already been used by Steig and there wasn’t anything that I could do about it. As I’ve explained to readers, I often write blog posts as sort of a work diary. My blog on Harry was to some extent a diary of what I’d found out about the series, and I certainly wanted to double check things (and determine more about the provenance) before contacting BAS.

On the other hand, Gavin’s mystery man seemed almost desperate to pre-empt me on this. Sometime during Super Bowl Sunday evening, it seems, Gavin’s mystery man contacted BAS about problems with Harry. BAS responded quickly to Gavin’s mystery man and on the (EST) morning of Feb 2, the Harry data was changed.

By the afternoon of Feb 2, not only had the data been changed, but Gavin Schmidt somehow knew something about the circumstances of the change, including the surprising news that a mystery man had “independently” discovered the Harry/Gill union, only a few hours after I’d published notice of a problem with Harry at Climate Audit and many Climate Audit readers had volunteered information on the problem.

Anything’s possible.

So who was Gavin’s mystery man and how was it that Gavin knew so confidently that the mystery man had identified the Harry/Gill problem “independently” of Climate Audit. One more thing: when did Gavin himself learn that the mystery man had identified the problem with Harry? At the time, realclimate had two threads devoted to Steig et al. If it was so urgent that BAS be notified of the problem that a delay until Monday was too late (as Gavin implies in his criticism of me), then once Gavin knew about the problem with Harry, did realclimate have an obligation to notify their readers? Just asking.

Answer here.

By Stephen McIntyre | Posted in Steig at al 2009 | Tagged , , , , , | Comments (132)

MM In Print

Feb 2, 2009 – 11:52 PM

A reader just sent me the MBH reply, which is amusing. It’s late more on this tomorrow.

MM comment and MBH reply

By Stephen McIntyre | Posted in General | Comments (74)

Dirty Harry 4: When Harry Met Gill

Feb 2, 2009 – 12:32 PM

Yesterday, I noted that Steig had criticised previous developers of Antarctic gridded temperature data for not having “paid much attention” to West Antarctica (e.g. the NASA GISS trend map left the area blank due to lack of data meeting their quality standards) and reproached his predecessors (including, it seems, even Hansen) for “calculating with their heads”, rather than “doing the math”, which, in his case, seems to be inseparable from the use of Mannian algorithms.

In various venues, Steig and coauthors have said that they “get” similar results using their surface/AWS reconstruction and satellite (AVHRR) reconstruction, with the similarity of the AWS result being used to reject concerns over possible problems and biases in the AVHRR reconstruction (where the monthly data remains unarchived and the data set has received negligible attention as compared to microwave and surface data).

In yesterday’s post on the West Antarctica sector of interest, I observed that GISS and Steig both used the same surface and AWS data – the blank area in GISS came not from failing to “do the math”, but because the station quality didn’t meet minimum GISS standards (which are not particularly arduous.) Differences between GISS and Steig arose, at least, in part because of Steig’s inclusion of these West Antarctica stations, which I listed: Byrd, Russkaya surface and Byrd, Siple, Mt Siple and Harry AWS stations. I plotted the AWS reconstructions (recon_aws) from Steig’s SI, showing that only Harry had a pronounced 1979-2003 trend (the period illustrated in Steig Figure 4), with Mt Siple AWS actually being negative. I ended the post with a teaser, saying that there was trouble with Harry.

FUBAR
Having identified Harry as a highly leveraged series in the AWS reconstruction, I decided to plot the GISS version against the READER version- this is the sort of routine plot that I do all the time. I had several versions of scraped GISS data, I had a vague recollection of a screw-up last June involving READER data (more on this in a moment) and thought – hmmm, let’s compare an old version with a new version, which yielded a plot looking like the one below (the script for this is in the first comment). As you see, there are HUGE differences between the new Harry and the old Harry.

Steig didn’t archive the data as used. But it turns out that it is possible to show that Steig used the “new” Harry version. Steig gives lat-longs for 63 recon_aws series, which can be matched to lat-longs of READER AWS stations (which I’ve matched to GISS IDs). It turns out that, through the wonders of RegEM, the Steig AWS reconstruction does not merely approximate the target AWS series – it matches it perfectly. So you can use the Steig AWS as a type of fossil version to check whether Steig used the old Harry or new Harry versions. As shown below, he used new Harry. The 1979-1999 anomaly version of new Harry pretty much matches the Steig version perfectly. There is no possibility that Steig used the old Harry version.


Figure 2. Comparison of Steig recon_aws version to new Harry version

Harry
The Harry AWS station is described here . It is located at 83.003S 121.393W 945 m (a different location than shown in Steig Table S2). It was installed in Nov 1994. Original AWS data at this site can be matched to READER data- for example, the file ftp://ice.ssec.wisc.edu/pub/aws/climate/1995/READER1355_1995.dat has a header line

Station Argos ID: 1355 Station Name: Harry Year: 1995

and the monthly values can be matched to READER Harry in 1995 (rounded at READER to one digit). Jan and Feb 1995 values are -10.6 and -14.1 respectively in both sources.

There was no particular pattern to the nomenclature in the Wisconsin data, so I ended up reading all the files, collating the header lines and making a details file, which is available at A/data/steig/details.wisc.tab (details). Using grep, I searched “Harry” and “HARRY” and got 5 files.

year id info
1355 1994 1355 Station Argos ID: 1355 Station Name: Harry Year: 1994
13551 1995 1355 Station Argos ID: 1355 Station Name: Harry Year: 1995
13552 1996 1355 Station Argos ID: 1355 Station Name: Harry Year: 1996
890012 1999 8900 Station Argos ID: 8900 Station Name: Harry Year: 1999
890013 2000 8900 Station Argos ID: 8900 Station Name: Harry Year: 2000

I collated the original data and compared it to new Harry (shown below only for years of overlap) and there was a match up to rounding between new Harry and Wisconsin Harry for 1994-1996 and 1999-2000, but the provenance of the other years was a mystery.


Figure 3. Compare Wisconsin Harry to New READER Harry.

Gill
CA readers made a number of useful suggestions yesterday on data provenance. I was fairly determined to track down the provenance of the “other” Harry data and by this time, I’d downloaded all the Wisc data (collated at CA in data/steig/wisc.tab). In the end, I simply examined all the Jan-Feb data in selected years to see if any data matched “Harry”. And thus, we met Gill. The “other” Harry data is derived from Gill. I’d figured this out yesterday when I wrote the teaser – a CA reader also figured this out for 1987-1989 this morning. But it’s not just 1987-1989; it’s 1987-1993 and 1997-1998. Also the “old” Harry was actually “Gill”. The graphic below compares READER/GISS New Harry with original Gill. Values are identical from 1987 to July 1994 and in 1997-1998. Values are different from Dec 1994 to Dec 1996 and for 1999-2000 where “Harry” has been spliced in the READER/GISS version.

Figure 4. Compare original Gill data to READER New Harry.

Gill is located on the Ross Ice Shelf at 79.92S 178.59W 25M and is completely unrelated to Harry. The 2005 inspection report observes:

2 February 2005 – Site visited. Site was difficult to locate by air; was finally found by scanning the horizon with binoculars. Station moved 3.8 nautical miles from the previous GPS position. The lower delta temperature sensor was buried .63 meters in the snow. The boom sensor was raised to 3.84 m above the surface from 1.57 m above the surface. Station was found in good working condition.

I didn’t see any discussion in Steig et al on allowing for the effect of burying sensors in the snow on data homogeneity.

The difference between “old” Harry and “new” Harry can now be explained. “Old” Harry was actually “Gill”, but, at least, even if mis-identified, it was only one series. “New” Harry is a splice of Harry into Gill – when Harry met Gill, the two became one, as it were.

Considered by itself, Gill has a slightly negative trend from 1987 to 2002. The big trend in “New Harry” arises entirely from the impact of splicing the two data sets together. It’s a mess.

If you now turn to the READER information page, you’ll see that Harry is right underneath Gill. Did that contribute to the screw-up? Are there other corresponding errors? Dunno.

British Antarctic Survey Erases Harry Data
Yesterday, I observed that there was trouble with Harry, notifying readers that today’s post was on the way. Triggered by this teaser, a reader pointed out that Gill and Harry had been spliced (something that I’d noticed yesterday and was one of the reasons why I put out a teaser for today’s post). About 1 pm Eastern today, when I went to verify some information at BAS , I found (as a CA reader had also just observed) that BAS had erased the “New Harry” data and a correct Harry version suddenly appeared (different from either old Harry or new Harry) – which, in its way, is a pretty compelling verification of the points made above. The British Antarctic Survey issued only the following notice, notably omitting any credit to Climate Audit (following the example set by Hansen and Mann):

Note! The surface aws data are currently being re-proccessed after an error was reported in the values at Harry AWS (2/2/09)

They did not provide any description of the error. They erased the incorrect data without retaining any copy of the prior version. You can still get the incorrect data at GISS, who haven’t changed their data yet (and I’ve saved a copy which I’ll put online at A/data/steig.)

June 2008
Does any of this bring back memories of a prior incident? Last June, we noticed data for the Southern Ocean shifting at GISS as we examined the derivation of Wellington NZ (an arbitrary example, but an interesting one in the present context.) See here. In the next comment, I noticed problems with Chatham Island (which is in the READER database, though we hadn’t turned our attention to it at the time); John Goetz identified the Chatham Island data as coming from the BAS site noting:

the record on the UK site has some errors,

We noted that GISS seemed to manually correct the BAS errors, see the next few comments 61-70, wondering what the basis of the edits was. Subsequently, we observed that GISS noted a few days earlier (without reference to Climate Audit) that changes had been made to Southern Ocean data due to problems with the READER collation:

June 9, 2008: Effective June 9, 2008, our analysis moved from a 15-year-old machine (soon to be decommissioned) to a newer machine. This will affect some results, though insignificantly. Some sorting routines were modified to minimize such machine dependence in the future. In addition, a typo was discovered and corrected in the program that dealt with a potential discontinuity in the Lihue station record. Finally, some errors were noticed on http://www.antarctica.ac.uk/met/READER/temperature.html (set of stations not included in Met READER) that were not present before August 2007. We replaced those outliers with the originally reported values. Those two changes had about the same impact on the results than switching machines (in each case the 1880-2007 change was affected by 0.002°C). See graph and maps.

Even though errors had been identified in one portion of READER, I guess they didn’t bother checking other READER data sets e.g. http://www.antarctica.ac.uk/met/READER/aws/awspt.html to see if maybe the problem that caused the Chatham Island problem occurred elsewhere. Had they done so, I’m sure that Steig et al would have appreciated it.

Postscript: See https://climateaudit.org/2009/02/03/gavins-mystery-man/

By Stephen McIntyre | Posted in Hansen, Steig at al 2009 | Tagged , | Comments (243)

West Antarctic Stations

Feb 1, 2009 – 3:41 PM

As noted yesterday, the main enterprise of Steig et al is a re-interpretation of West Antarctica, saying that “no one really has paid much attention to” this area in the past and saying that their study improved over prior studies because previously:

“People were calculating with their heads instead of actually doing the math,” said lead author of the study Eric Steig, of the University of Washington in Seattle. “What we did is interpolate carefully instead of just using the back of an envelope.”

I observed yesterday that NASA GISS used the same source data for surface and AWS stations as Steig (READER) but were unable to calculate trends in West Antarctica (blank on the trend map -see yesterday’s post), a “defect” that Steig attributed above to Hansen’s group “calculating with their heads instead of actually doing the math” and “just using the back of an envelope”. Hansen’s bulldogs, Gavin and Tamino, who usually defend Hansen against the smallest slight or perceived slight, are strangely silent to Steig’s damning accusations against Hansen.

Today I’m going to look at the station data to see whether Steig’s accusations against Hansen are fair or not.

Figure 1 below is a location map of stations listed in Steig Table S2. The green boundary is more or less the outline of the “West Antarctica” hot spot in Steig’s graphics. There are 5 dots in the GISS-empty portion of this area, which I’ll discuss after you have a chance to locate the dots: 3 blue, one green and one red.


Figure 1. Location of Steig Table S2 Stations

Four of the stations are AWS stations shown in the bottom of Steig Table 2, excerpted below: MT Siple, Siple, Byrd and Harry. There are two surface stations: Byrd (in the same spot as Byrd AWS which operates a number of years after the surface station) and Russkaya (on the coast). I had a little trouble with my original plot of station locations because I used the information from Steig’s SI shown below.


Excerpt from Steig Table 2. First two columns are lat and long.

In Steig’s location table, Mt Siple and Harry AWS stations have identical latitude and longitude (first two columns). In fact, they have different locations: Mt Siple is shown with its correct location (a blue dot on the coast), but Harry’s coordinates are wrong. Harry is the green dot in this map. Are these coordinates incorrect only in the Nature SI? Dunno. Would it even “matter” in Mannian methodology if they were incorrect? Dunno. It hasn’t “mattered” in any other Mannian effusion and, in this case, it might not matter if Harry were on Broadway or in Antarctica. It’s hard to say. You’d think that they’d be on the look-out for geographic mislocations given Mann’s prior rain in Maine/rain in Spain problems, but I guess not.

Below is some information (including looked up GISS numbers) on these six West Antarctic stations, that appear to underpin the Steig reinterpretation for their “AWS” reconstruction. I looked up these 6 data sets in GISS dset0, dset1 and dset2. Hansen only qualified one of the six stations as meeting his dset2 standards (Byrd 1957-1975.) Thus, in Hansen’s view, the data quality problems (coverage, length of record etc) with Russkaya surface and Mt Siple, Siple, Byrd and Harry AWS stations all disqualified them from dset2 status. In addition, the coverage period for Byrd surface dset2 annual average was only from 1957-1970, thus not qualifying for a 1979-2003 trend. That’s why West Antarctica is blank in the GISS trend map.

name lat long id start end gissid
8 Byrd -80.0 -120.0 89125 1957.000 1975.000 70089125000
35 Russkaya -74.8 -136.9 89132 1980.250 1990.083 70089132000
43 Mt_Siple_AWS -73.2 -127.1 89327 1992.167 2006.000 70089327000
44 Siple_AWS -75.9 -84.0 89284 1982.000 1992.167 70089284000
45 Byrd_AWS -80.0 -120.0 89324 1980.167 2008.750 70089324000
46 Harry_AWS -73.2 -127.1 21355 1987.000 2002.917 70021355000

Let’s now turn to Steig’s allegation that Hansen (like all of Steig’s predecessors) “calculated with his head instead of actually doing the math” and that he “just used the back of an envelope.” While I may seem like an unlikely defender of Hansen and GISS, fair’s fair. I see no evidence to support Steig’s allegation. On the contrary, there is clear evidence that Hansen put the information for all six stations into the GISTEMP algorithm. Whether the GISTEMP algorithm is any good is a different question, but Steig’s characterization of the process, appealing as it might be to Hansen critics, is surely uncalled for.

Here’s a plot of the READER version of the six stations. Given the importance that Steig placed on re-interpreting West Antarctica, it would have been nice if he had included a plot of the data, as even RegEM covariance matrices ultimately are related to data. At a first glance, this doesn’t seem like a whole lot of data to work with.


Figure 3. Station Data: 6 West Antarctica Stations

Steig’s recon_aws reconstruction offers anomaly reconstructions for 63 AWS sites, including the 4 West Antarctic AWS sites, which are plotted below, with their slopes over the 1979-2003 (used in the color trend maps yesterday. One of the four sites (Mt Siple) has a slightly negative trend; two of three (Siple, Byrd AWS) have modestly positive trends; while one site (Harry) – the one with the incorrect location – has a very strong trend of 0.81 deg C/decade.


Figure 4. Steig reon_aws reconstructions

Stay tuned for some interesting news about Harry.

By Stephen McIntyre | Posted in Steig at al 2009 | Tagged | Comments (92)

Steig versus Hansen

Jan 31, 2009 – 11:53 AM

In Dec 2004, Steig and Schmidt wrote

The Antarctic Peninsula, site of the now-defunct Larsen-B ice shelf, has warmed substantially. On the other hand, the few stations on the continent and in the interior appear to have cooled slightly (Doran et al, 2002; GISTEMP). At first glance this seems to contradict the idea of “global” warming, but one needs to be careful before jumping to this conclusion.

More recently, in Jan 2009, Steig wrote in explanation of Steig et al here

So what do our results show? Essentially, that the big picture of Antarctic climate change in the latter part of the 20th century has been largely overlooked. It is well known that it has been warming on the Antarctic Peninsula, probably for the last 100 years (measurements begin at the sub-Antarctic Island of Orcadas in 1901 and show a nearly monotonic warming trend). And yes, East Antarctica cooled over the 1980s and 1990s (though not, in our results, at a statistically significant rate). But West Antarctica, which no one really has paid much attention to (as far as temperature changes are concerned), has been warming rapidly for at least the last 50 years.

I was intrigued by the statement in Steig et al that “no one really has paid much attention to [West Antarctica ]”. How could this be?

The graphic shown below (Steig Figure 4e) shows temperature trends in Antarctica for 1979-2003 as calculated in Steig et all. The Antarctic Peninsula can be seen going off to the northwest. “West Antarctica” is the “hot” region adjacent to the Peninsula.

Figure 1. Excerpt from Steig Figure 4. Color scale goes from -0.5 to +0.5 deg C/decade.

One of the main theses of Steig et al is that a more accurate accounting of Antarctica yields an overall profit: the profits in the Peninsula and West Antarctica outweigh slight losses/trading dollars in East Antartica. Antarctic temperature trends are not something that I’ve previously taken a position on nor had I familiarized myself with the data. As someone approaching the matter freshly (as I mentioned before), it seems surprising to me that Antarctica wouldn’t have warmed along the rest of the world; it also seems to me that there are modeling issues specific to Antarctica and modelers might well get inconsistent results in this continent without meaning that everything that they ever did was worthless (though it might be worthless on other grounds).

I downloaded GISTEMP binary data and calculated 1979-2003 GISTEMP trends for 1200 km smoothing (250 km smoothing is much sparser) and obtained the plot shown below. (I haven’t figured out how to show the scale on the plotting software, but it’s -.5 to 0.5 deg C/decade (I used the package maps – but will experiment with some of Roman’s code in the future). I required 280 out of 300 months for a trend (the availability fell off very sharply.)

Figure 2. GISS Trends 1979-2003. Color scale goes from -0.5 to +0.5 deg C/decade.

Interestingly GISS uses exactly the same READER data set as Steig – see here where there are references to READER pages. (I’ve matched GISTEMP numbers to all the READER data for future reference.)  To the extent that Steig claim that they can get the same result without satellite data, this means that it because they process the data differently than GISS – not that they are using different data.

Both maps show strong warming on the Peninsula, but GISS smoothing did not generate sufficient data in West Antarctica to show a trend one way or the other. Steig and Mann have deduced that this region with missing data in GISS is nonetheless warming strongly. Could be.  If this were a mining map, one would certainly look for ore in regions adjacent to known ore and West Antarctica is adjacent to known Peninsula warming.

There are some other interesting differences: the “cold” area from 90-180E is mildly warming onshore in the Steig rendering. Would be interesting to know what accounts for the differences.

But first I’m going to look at what surface and AWS data is available in West Antarctica – to see what the data looks like before it goes into RegEM.

As a further observation on the inconsistency between GISS and Steig results for the Antarctic: Steig describes himself as an isotope geochemist. It’s nice that Steig’s taken time from his busy studies of isotope geochemistry to try to sort out GISTEMP data in the Antarctic – but isn’t this the sort of thing that GISTEMP should have done in the first place?

It would also be nice to hear from GISS on this: what do Steig’s results say about GISTEMP methodology? Steig didn’t analyse why GISS methods seem to have been incapable of discerning recent warming in West Antarctica (an unexpected sort of oversight), but shouldn’t there be some kind of effort at detailed reconciliation.

By Stephen McIntyre | Posted in Steig at al 2009 | Tagged , , , , , , | Comments (52)

Santer's Boss Seeks to "Clarify Mis-Impressions"

Jan 30, 2009 – 9:51 PM

David Bader, PhD, the Director, Program for Climate Model Diagnosis and Intercomparison, writes today seeking to “clarify several mis-impressions on your “climateaudit.org” web site” regarding the archiving of Santer’s data, the correspondence being shown below.

Readers may recall an earlier post here in which I requested data from Santer et al 2008, in response to which Santer refused to provide the data, circulating his discourteous refusal to 17 coauthors (none of whom had been copied in the original inquiry) and the journal editor. Santer:

… I see no reason why I should do your work for you, and provide you with derived quantities (zonal means, synthetic MSU temperatures, etc.) which you can easily compute yourself. I am copying this email to all co-authors of the 2008 Santer et al. IJoC paper, as well as to Professor Glenn McGregor at IJoC.

I gather that you have appointed yourself as an independent arbiter of the appropriate use of statistical tools in climate research. Rather that “auditing” our paper, you should be directing your attention to the 2007 IJoC paper published by David Douglass et al., which contains an egregious statistical error. Please do not communicate with me in the future.

I have reviewed this post and fail to see any statements by me that could possibly be construed as contributing to a “mis-impression”. On Nov 24, I reported on my unsuccessful FOI request to NOAA for the data, in which all of the NOAA coauthors claimed to have never seen the requested data. Again, I am unable to see any statements by me that could be construed as creating a “mis-impression”. The facts are what they are.

On Dec 28, I reported on efforts to obtain the data through the journal, reporting that these efforts had also been unsuccessful, as the publisher of the journal, the Royal Meteorological Society lacked a data archiving policy. A positive outcome of this effort was that the Society plans to review their lack of policy at a forthcoming editorial meeting. Again, I am unable to see any statements by me that could be construed as creating a “mis-impression”. The facts are what they are.

Around January 18th, I received a snail mail letter from the U.S. government dated Dec 10 (snail mail indeed), advising me that the FOI request had been placed in a queue and would be responded to when it got to the top of the pile. I didn’t plan to hold my breath.

On Jan 26, Ross and I submitted an article on Santer et al 2008, noted up the next day here ; I reported in the post that our submission included comments on the data refusal. On January 27, I received an email from a reader notifying me that the reader had just been notified by the U.S. Department of Energy that the data had been placed at a public archive. I promptly communicated this information to readers. I noted that, while the reader had been so notified, I had not received equivalent notice, again, a matter of fact. I am unable to see any statements by me that could be construed as creating a “mis-impression”. Later that day, I consulted the new archive and noted with some amusement that the file unzipped to a directory entitled “FOIA”. [Update Jan 31 – for “clarification”, I do not imply that the data was released “because” of our journal submission on Jan 26. The CA reader in question had been on a lengthy business trip. As noted below, he had been informed on Jan 14 that Livermore was planning to release the data and that he would be informed of the url when available; he followed up upon return from his business trip on Jan 26 and obtained the url on Jan 27, whereupon he informed me. Had he not been traveling, he might have learned the url on an earlier data. Livermore did not inform me of either their plans or the actual archiving and my knowledge of the situation came only from from this CA reader.]

Earlier today, Dr Bader wrote as follows:

Dear Mr. McIntyre;

I want to clarify several mis-impressions on your “climateaudit.org” web site with respect to the Synthetic MSU data sets on the PCMDI website.

1. The data were released publicly on 14 January 2009, at which time our Department of Energy sponsors and NNSA Freedom of Information Act officials were notified. These data were released voluntarily by the Lawrence Livermore National Laboratory and we were never directed to do so as a result of a Freedom of Information Act (FOIA) request. Furthermore, preparation of the datasets and documentation for them began before your FOIA request was received by us.

2. The long disclaimer on the web site beginning with the sentence, “This data available on this site was prepared as an account of work sponsored by an agency of the United States government. …..” is standard language on all published material from Lawrence Livermore National Laboratory and is not specific to this dataset.

David Bader, PhD
Director, Program for Climate Model Diagnosis and Intercomparison

Needless to say, I was extremely surprised by this letter. If Lawrence Livermore was actively preparing the data sets for public release “before” my request for the data, then surely Santer had an obligation to simply say so, rather than withholding the information that a public release was planned in the near future and challenging me to recreate their monthly results from first principles. Secondly, I was surprised to learn that the directory entitled FOIA had nothing to do my Freedom of Information Act request. Perhaps FOIA in this context stands for something else.

I replied to Bader, citing verbatim my original request for data and Santer’s refusal and asking:

If, as you say, preparations for the release of this data had already begun, could you explain why your employee failed to advise me of this at the time. In addition, I received no notice of these plans pursuant to my FOI request (to which I have received two separate acknowledgements).

You say that, on Jan 14, 2009, the data was “released publicly” on Jan 14, 2009 and DOE were notified of this. On Jan 14, 2009, one of my readers received an email from DOE saying that the data was being finalized that week in preparation for posting and the lab was seeking final approval from their site office to post the data, undertaking to send you the url when it was available, providing notice of availability on Jan 26. I received notice from the reader on the following day, Jan 27, and promptly recorded this notice on my blog.

At no point prior to your email did anyone from your organization notify me that the data was now “publicly available” despite my outstanding request.

I will post a notice at my blog of your position, but I’m sure that you will understand if I make editorial comments on them.

Given the information in your email, I hereby file a complaint about the handling of my request for data and request that you investigate how it was handled.

I also observed:

if you unzip the data sets placed online, they unzip into a folder entitled FOIA. Does FOIA in this context have another interpretation other than Freedom of Information Act that I should be aware of?

Bader replied:

My previous email was to clarify certain factual information with respect to the release of the Synthetic MSU data. Respectfully, I believe it to be counter productive to engage in an exchange regarding the history of your correspondence with Dr. Santer or Department of Energy officials on a matter I believe to be completely resolved.

I wrote back to him:

What is the factual information that you wish to correct? I am quite prepared to correct any errors: could you please provide me with specific points of fact that you believe to be in error, as I have reviewed the posts in question and cannot any “errors” in what I posted.

This request failed to elicit any details on any factual errors that could have contributed to any “mis-impression”. Bader replied:

I will make a sincere attempt to advise you of our good faith efforts to release the data in question. What you may not realize is that there are time consuming, but legitimate and typical review processes at most laboratories. For example, it takes 2-4 weeks from the time a manuscript is completed at my lab before it is transmitted to a journal for submission.

1. I was not aware of your FOIA request to the NNSA until sometime in early to mid December. At that time, we had already begun the process of preparing the Synthetic MSU datasets and their documentation for public release as a low priority activity. In their original format and without documentation, they were of little value to the broader scientific community for which they were intended.

2. When contacted about the FOIA request by LLNL FOIA officials, I inquired as to whether our plans for data release met their needs, and was advised that it would.

3. Given the demands on my staff’s time, the length of time required for the official laboratory “Review and Release” process and the upcoming holidays and vacations, I asked LLNL officials who handle FOIA requests if January 15 was a reasonable deadline for the release of data. I was advised that the date was acceptable.

4. The data were made available on January 14, at which time I notified LLNL staff who deal with FOIA requests and Department of Energy officials.

Should you intend to publish my correspondence with you on your site, I request that you post this entire email, without embedded editorial comment. Since you have clearly stated your intention to file a formal complaint in a previous message, I think you can understand that further correspondence with me on this issue would not be productive.

Santer et al 2008 was submitted on 25 March 2008, revised 18 July 2008 and accepted 20 July 2008.

Obviously, I have a number of questions about this matter, most of which will undoubtedly occur to readers. So I’ll defer editorial comment for now.

By Stephen McIntyre | Posted in Santer | Tagged | Comments (192)