Yesterday, I noted that Steig had criticised previous developers of Antarctic gridded temperature data for not having “paid much attention” to West Antarctica (e.g. the NASA GISS trend map left the area blank due to lack of data meeting their quality standards) and reproached his predecessors (including, it seems, even Hansen) for “calculating with their heads”, rather than “doing the math”, which, in his case, seems to be inseparable from the use of Mannian algorithms.
In various venues, Steig and coauthors have said that they “get” similar results using their surface/AWS reconstruction and satellite (AVHRR) reconstruction, with the similarity of the AWS result being used to reject concerns over possible problems and biases in the AVHRR reconstruction (where the monthly data remains unarchived and the data set has received negligible attention as compared to microwave and surface data).
In yesterday’s post on the West Antarctica sector of interest, I observed that GISS and Steig both used the same surface and AWS data – the blank area in GISS came not from failing to “do the math”, but because the station quality didn’t meet minimum GISS standards (which are not particularly arduous.) Differences between GISS and Steig arose, at least, in part because of Steig’s inclusion of these West Antarctica stations, which I listed: Byrd, Russkaya surface and Byrd, Siple, Mt Siple and Harry AWS stations. I plotted the AWS reconstructions (recon_aws) from Steig’s SI, showing that only Harry had a pronounced 1979-2003 trend (the period illustrated in Steig Figure 4), with Mt Siple AWS actually being negative. I ended the post with a teaser, saying that there was trouble with Harry.
Having identified Harry as a highly leveraged series in the AWS reconstruction, I decided to plot the GISS version against the READER version- this is the sort of routine plot that I do all the time. I had several versions of scraped GISS data, I had a vague recollection of a screw-up last June involving READER data (more on this in a moment) and thought – hmmm, let’s compare an old version with a new version, which yielded a plot looking like the one below (the script for this is in the first comment). As you see, there are HUGE differences between the new Harry and the old Harry.
Steig didn’t archive the data as used. But it turns out that it is possible to show that Steig used the “new” Harry version. Steig gives lat-longs for 63 recon_aws series, which can be matched to lat-longs of READER AWS stations (which I’ve matched to GISS IDs). It turns out that, through the wonders of RegEM, the Steig AWS reconstruction does not merely approximate the target AWS series – it matches it perfectly. So you can use the Steig AWS as a type of fossil version to check whether Steig used the old Harry or new Harry versions. As shown below, he used new Harry. The 1979-1999 anomaly version of new Harry pretty much matches the Steig version perfectly. There is no possibility that Steig used the old Harry version.
The Harry AWS station is described here . It is located at 83.003S 121.393W 945 m (a different location than shown in Steig Table S2). It was installed in Nov 1994. Original AWS data at this site can be matched to READER data- for example, the file ftp://ice.ssec.wisc.edu/pub/aws/climate/1995/READER1355_1995.dat has a header line
Station Argos ID: 1355 Station Name: Harry Year: 1995
and the monthly values can be matched to READER Harry in 1995 (rounded at READER to one digit). Jan and Feb 1995 values are -10.6 and -14.1 respectively in both sources.
There was no particular pattern to the nomenclature in the Wisconsin data, so I ended up reading all the files, collating the header lines and making a details file, which is available at A/data/steig/details.wisc.tab (details). Using grep, I searched “Harry” and “HARRY” and got 5 files.
year id info
1355 1994 1355 Station Argos ID: 1355 Station Name: Harry Year: 1994
13551 1995 1355 Station Argos ID: 1355 Station Name: Harry Year: 1995
13552 1996 1355 Station Argos ID: 1355 Station Name: Harry Year: 1996
890012 1999 8900 Station Argos ID: 8900 Station Name: Harry Year: 1999
890013 2000 8900 Station Argos ID: 8900 Station Name: Harry Year: 2000
I collated the original data and compared it to new Harry (shown below only for years of overlap) and there was a match up to rounding between new Harry and Wisconsin Harry for 1994-1996 and 1999-2000, but the provenance of the other years was a mystery.
CA readers made a number of useful suggestions yesterday on data provenance. I was fairly determined to track down the provenance of the “other” Harry data and by this time, I’d downloaded all the Wisc data (collated at CA in data/steig/wisc.tab). In the end, I simply examined all the Jan-Feb data in selected years to see if any data matched “Harry”. And thus, we met Gill. The “other” Harry data is derived from Gill. I’d figured this out yesterday when I wrote the teaser – a CA reader also figured this out for 1987-1989 this morning. But it’s not just 1987-1989; it’s 1987-1993 and 1997-1998. Also the “old” Harry was actually “Gill”. The graphic below compares READER/GISS New Harry with original Gill. Values are identical from 1987 to July 1994 and in 1997-1998. Values are different from Dec 1994 to Dec 1996 and for 1999-2000 where “Harry” has been spliced in the READER/GISS version.
Figure 4. Compare original Gill data to READER New Harry.
Gill is located on the Ross Ice Shelf at 79.92S 178.59W 25M and is completely unrelated to Harry. The 2005 inspection report observes:
2 February 2005 – Site visited. Site was difficult to locate by air; was finally found by scanning the horizon with binoculars. Station moved 3.8 nautical miles from the previous GPS position. The lower delta temperature sensor was buried .63 meters in the snow. The boom sensor was raised to 3.84 m above the surface from 1.57 m above the surface. Station was found in good working condition.
I didn’t see any discussion in Steig et al on allowing for the effect of burying sensors in the snow on data homogeneity.
The difference between “old” Harry and “new” Harry can now be explained. “Old” Harry was actually “Gill”, but, at least, even if mis-identified, it was only one series. “New” Harry is a splice of Harry into Gill – when Harry met Gill, the two became one, as it were.
Considered by itself, Gill has a slightly negative trend from 1987 to 2002. The big trend in “New Harry” arises entirely from the impact of splicing the two data sets together. It’s a mess.
If you now turn to the READER information page, you’ll see that Harry is right underneath Gill. Did that contribute to the screw-up? Are there other corresponding errors? Dunno.
British Antarctic Survey Erases Harry Data
Yesterday, I observed that there was trouble with Harry, notifying readers that today’s post was on the way. Triggered by this teaser, a reader pointed out that Gill and Harry had been spliced (something that I’d noticed yesterday and was one of the reasons why I put out a teaser for today’s post). About 1 pm Eastern today, when I went to verify some information at BAS , I found (as a CA reader had also just observed) that BAS had erased the “New Harry” data and a correct Harry version suddenly appeared (different from either old Harry or new Harry) – which, in its way, is a pretty compelling verification of the points made above. The British Antarctic Survey issued only the following notice, notably omitting any credit to Climate Audit (following the example set by Hansen and Mann):
Note! The surface aws data are currently being re-proccessed after an error was reported in the values at Harry AWS (2/2/09)
They did not provide any description of the error. They erased the incorrect data without retaining any copy of the prior version. You can still get the incorrect data at GISS, who haven’t changed their data yet (and I’ve saved a copy which I’ll put online at A/data/steig.)
Does any of this bring back memories of a prior incident? Last June, we noticed data for the Southern Ocean shifting at GISS as we examined the derivation of Wellington NZ (an arbitrary example, but an interesting one in the present context.) See here. In the next comment, I noticed problems with Chatham Island (which is in the READER database, though we hadn’t turned our attention to it at the time); John Goetz identified the Chatham Island data as coming from the BAS site noting:
the record on the UK site has some errors,
We noted that GISS seemed to manually correct the BAS errors, see the next few comments 61-70, wondering what the basis of the edits was. Subsequently, we observed that GISS noted a few days earlier (without reference to Climate Audit) that changes had been made to Southern Ocean data due to problems with the READER collation:
June 9, 2008: Effective June 9, 2008, our analysis moved from a 15-year-old machine (soon to be decommissioned) to a newer machine. This will affect some results, though insignificantly. Some sorting routines were modified to minimize such machine dependence in the future. In addition, a typo was discovered and corrected in the program that dealt with a potential discontinuity in the Lihue station record. Finally, some errors were noticed on http://www.antarctica.ac.uk/met/READER/temperature.html (set of stations not included in Met READER) that were not present before August 2007. We replaced those outliers with the originally reported values. Those two changes had about the same impact on the results than switching machines (in each case the 1880-2007 change was affected by 0.002°C). See graph and maps.
Even though errors had been identified in one portion of READER, I guess they didn’t bother checking other READER data sets e.g. http://www.antarctica.ac.uk/met/READER/aws/awspt.html to see if maybe the problem that caused the Chatham Island problem occurred elsewhere. Had they done so, I’m sure that Steig et al would have appreciated it.
Postscript: See http://climateaudit.org/2009/02/03/gavins-mystery-man/