Tamino and the PC4

Mar 11, 2008 – 11:07 AM

As noted previously, Tamino did not quote, cite or discuss how our articles reported key issues in his post – an omission that results in our research not being accurately represented in the record at Tamino’s site. I’ll discuss a couple of examples. It’s unfortunate that time has to be spent on such matters prior to dealing with issues like short-centering.

The PC4
Tamino reports breathlessly that the hockey stick pattern can be observed in the PC4 of the North American tree ring network, because it’s “a pattern in the data” and not an “artifact”.

the hockey stick pattern is still there even with centered PC — which is no surprise, because it’s not an artifact of the analysis method, it’s a pattern in the data.

and later asks with this illustration,

Did MM really not get this? Did they really discard the relevant PCs just to copy the bare number of PCs used by MBH, without realizing that the different centering convention could move the relevant information up or down the PC list?

You betcha. When done properly on the actual data, using 5 PCs rather than just 2, the hockey stick pattern is still there even with centered PC — which is no surprise, because it’s not an artifact of the analysis method, it’s a pattern in the data. Here’s a comparison of PC#1 for the North American ITRDB (international tree ring data base) data using the MBH method (red), and PC#4 from using the MM method.

Tamino has misrepresented the research record, as both MM2005 (GRL) and MM2005 (EE) report the occurrence of the hockey stick pattern in the North American PC4, attributing it to the bristlecones.

In MM2005 (GRL), we stated:

Under the MBH98 data transformation, the distinctive contribution of the bristlecone pines is in the PC1, which has a spuriously high explained variance coefficient of 38% (without the transformation – 18%). Without the data transformation, the distinctive contribution of the bristlecones only appears in the PC4, which accounts for less than 8% of the total explained variance.

In MM2005(EE), we re-iterated the same point at a little more length:

In the MBH98 de-centered PC calculation, a small group of 20 primarily bristlecone pine sites, all but one of which were collected by Donald Graybill and which exhibit an unexplained 20th century growth spurt (see Section 5 below), dominate the PC1. Only 14 such chronologies account for over 93% of the variance in the PC1, effectively omitting the influence of the other 56 proxies in the network. The PC1 in turn accounts for 38% of the total variance. In a centered calculation on the same data, the influence of the bristlecone pines drops to the PC4 (pointed out in Mann et al., 2004b, 2004d). The PC4 in a centered calculation only accounts for only about 8% of the total variance, which can be seen in calculations by Mann et al. in Figure 1 of Mann et al. [2004d].

In MM2005 (EE), we further reported the effect of carrying out an MBH-type reconstruction under many permutations (most of which were re-stated by Wahl and Ammann without citing our findings), including the use of both 2 and 5 North American covariance PCs.

If a centered PC calculation on the North American network is carried out (as we advocate), then MM-type results occur if the first 2 NOAMER PCs are used in the AD1400 network (the number as used in MBH98), while MBH-type results occur if the NOAMER network is expanded to 5 PCs in the AD1400 segment (as proposed in Mann et al., 2004b, 2004d). Specifically, MBH-type results occur as long as the PC4 is retained, while MM-type results occur in any combination which excludes the PC4. Hence their conclusion about the uniqueness of the late 20th century climate hinges on the inclusion of a low-order PC series that only accounts for 8 percent of the variance of one proxy roster.

These are not exotic references; these points at issue in Tamino’s posts are specifically and clearly discussed in these articles.

Wegman also stated that the hockey stick from the bristlecone/foxtails occurred in the PC4 (see Question 10b):

Without attempting to describe the technical detail, the bottom line is that, in the MBH original, the hockey stick emerged in PC1 from the bristlecone/foxtail pines. If one centers the data properly the hockey stick does not emerge until PC4. Thus, a substantial change in strategy is required in the MBH reconstruction in order to achieve the hockey stick, a strategy which was specifically eschewed in MBH. In Wahl and Ammann’s own words, the centering does significantly affect the results

“In the Data”
Here’s a related Tamino straw man. Tamino states:

PCA (centered or not) doesn’t create patterns at all, they have to be there already even to “exhibit a larger variance.”

No one disagrees with this. We stated that the MBH algorithm “mined” for hockey stick patterns; we did not say that it “manufactured” them.

In effect, the MBH98 data transformation results in the PC algorithm mining the data for hockey stick patterns.

Wegman (see question 9b) expressed the point in similar terms:

If the variance is artificially increased by decentering, then the principal component methods will “data mine” for those shapes. In other words, the hockey stick shape must be in the data to start with or the CFR methodology would not pick it up… Most proxies do not contain the hockeystick signal. The MBH98 methodology puts undue emphasis on those proxies that do exhibit the hockey-stick shape and this is the fundamental flaw. Indeed, it is not clear that the hockey-stick shape is even a temperature signal because all the confounding variables have not been removed.

What is Tamino’s point of disagreement on this issue with either Wegman or ourselves?

As to the PC4, we stated clearly that the hockey stick shape was a distinct pattern in the North American tree ring data set, observable in the PC4 under a centered calculation (as Mann et al had done as well in their Nature submission placed online and in a post at realclimate). We reported that the pattern could be traced back to the bristlecones and spent a considerable amount of time analyzing bristlecones. For Tamino to present the PC4 without making any citation or reference to our comments on the matter – and to then snottily ask “Did MM not get this?” results in our research not being accurately represented in his posting.

By Stephen McIntyre | Posted in MBH98 | Tagged , | Comments (98)

"Mannian" PCA Revisited #1

Mar 10, 2008 – 12:27 PM

One of Hansen’s pit bulls, Tamino, has re-visited Mannian principal components. Tamino is a bright person, whose remarks are all too often marred by the pit bull persona that he has unfortunately chosen to adopt. His choice of topic is a curious one, as he ends up re-opening a few old scabs, none of which seem likely to me to improve things for the Team, but some of which are loose ends, and so I’m happy to re-visit them and perhaps resolve some of them. Tamino’s post and associated comments are marred by a number of untrue assertions and allegations about our work, a practice that is unfortunately all too common within the adversarial climate science community. When Tamino criticizes our work, he never quotes from it directly – a practice that seems to have originated at realclimate and adopted by other climate scientists. Thus his post ends up being replete with inaccuracies whenever it comes to our work.

Tamino’s main effort in this post is an attempt to claim that Mannian non-centered principal components methodology is a legitimate methodological choice. This would seem to be an uphill fight given the positions taken by the NAS Panel and the Wegman Panel – see also this account of a 2006 American Statistical Association session. However, Tamino makes a try, claiming that Mannian methodology is within an accepted literature, that it has desirable properties for climate reconstructions and that there were good reasons for its selection by MBH. I do not believe that he established any of these claims; I’ll do a post on this topic.

In the course of this exposition, Tamino discusses the vexed issue of PC retention for tree ring networks, claiming that MBH employed “objective criteria” for retaining PCs. I am unaware of any methodology which can successfully replicate the actual pattern of retained PCs in MBH98, a point that I made in an early CA post here. This inconsistency has annoyed me for some time and I’m glad that Tamino has revived the issue as perhaps he can finally explain how one actually derives MBH retention for any case other than the AD1400 NOAMER network. I’ll do a detailed post on this as well.

Whenever there is any discussion of principal components or some such multivariate methodology, readers should keep one thought firmly in their minds: at the end of the day – after the principal components, after the regression, after the re-scaling, after the expansion to gridcells and calculation of NH temperature – the entire procedure simply results in the assignment of weights to each proxy. This is a point that I chose to highlight and spend some time on in my Georgia Tech presentation. The results of any particular procedural option can be illustrated with the sort of map shown here – in which the weight of each site is indicated by the area of the dot on a world map. Variant MBH results largely depend on the weight assigned to Graybill bristlecone chronologies – which themselves have problems (e.g. Ababneh.)

pcsht44.gif
Figure 1. Each dot shows weights of AD1400 MBH98 proxies with area proportional to weight. Major weights are assigned to the NOAMER PC1 (bristlecones), Gaspe, Tornetrask and Tasmania. Issues have arisen in updating each of the first three sites.

Readers also need to keep in mind that two quite distinct principal component operations are carried out in MBH98 – one on gridded temperature data from CRU; and one on tree ring networks from ITRDB and elsewhere. The characteristics of the networks are very different. The gridded temperature networks result in a matrix of data with some effort at geographic organization, while there is no such attempt in the tree ring networks. The tree ring networks are more like the GHCN station networks than like a gridded network. For comparison, imagine a PC calculation on station data in the GHCN network going prior to 1930 with no attempt at geographic organization or balance. Followers of this discussion realize that long station data is overwhelmingly dominated by U.S. (USHCN) station information. Actually, the tree ring networks are even more disparate than GHCN station networks. Many of the tree ring sites are limited by precipitation. Thus, PC on the tree ring networks is more like doing a PC analysis on a pseudo-GHCN network consisting of mostly of precipitation measurements with some blends of temperature and precipitation, with a predominance of stations from the southwest U.S. There are obviously many issues in trying to transpose Preisendorfer methodology developed for very specific circumstances to this sort of information.

Before getting to either of the above larger topics, I wish to lead in with a discussion of the PC calculations in the gridded temperature network – something that hasn’t been on our radar screens very much, but which may shed some light on the thorny tree ring calculations. Here there is some new information since early 2005 – the source code provided to the House Energy and Commerce Committee shows the “Rule N” calculation for gridded temperature networks (though not for tree ring networks) and, in the absence of other information, can help illuminate obscure Mannian calculations.

In the course of this review, I also reconsidered Preisendorfer 1988, Mann’s cited authority for PCA. Preisendorfer (1988), entitled “Principal Component Analysis in Meteorology and Oceanography”, is an interesting and impressive opus, with many interesting by-ways and asides. Preisendorfer is clearly an experienced mathematician and his results are framed in mathematical terms, rather than ad hoc recipes. Many of his comments remain relevant to the present debate, . Continue reading →

By Stephen McIntyre | Posted in MBH98 | Tagged , , , , , | Comments (60)

Anthony Watts on Glenn Beck

Mar 8, 2008 – 2:25 PM

Anthony reports:

Here is a link to the audio and the transcript of the radio interview I did with Glenn Beck on Monday at the International Conference on Climate Change in New York.
http://www.glennbeck.com/content/articles/article/196/6727/?ck=1
For those that would like to see my slide show presented at ICCC that day, see this link:
http://gallery.surfacestations.org/watts-NYC-2008/index.html
UPDATE: The video interview I did on the Glenn Beck show is now online, see link below:

By Stephen McIntyre | Posted in General | Comments (79)

USHCN "Raw" – A Small Puzzle

Mar 5, 2008 – 9:55 AM

During the past few days, I’ve been assessing the GHCN-Daily dataset, which is a very large data set and plan to do a number of posts on this topic, including a description of the data set. It turns out that literally hundreds of stations that expire around 1989 or 1990 in the NASA data set are alive and thriving in the GHCN-Daily parallel universe. More on this over the next few days.

Before I get to this, I’d like to document a small puzzle in connection with the calculation of the USHCN “raw” monthly average that arose out of inspection of the GHCN-D data, a puzzle that takes us back to the Detroit Lakes MN station, the investigation of which led to the identification of Hansen’s Y2K error.

I’m not saying that these small puzzles necessarily or even probably “matter” in terms of world averages, but they are relevant in terms of craftsmanship and I presume the following: if one is going to the trouble of making these large temperature collations, the the craftsmanship should be as good as possible. By commenting on issues pertaining to craftsmanship, I am not imputing malfeasance, as some perpetually agitated commenters allege. However, as far as I can tell, no one – journal peer reviewer, NASA peer reviewer (if such existed), rival climate scientist, skeptic – ever seems to have gone to the trouble of parsing through the actual craftsmanship of the large temperature calculations and I see no harm and some benefit in doing so. It looks like NASA is paying some attention and has already implemented a couple of recommendations made at CA. (I’ll have some similar suggestions in the near future.)

The GHCN-D dataset contains daily max and mean information for nearly all the USHCN stations, as well as thousands of ROW stations. (I’ll discuss some discrepancies in the USHCN station lists on another occasion). The GHCN-D data set is available in a huge zipped file, but is also available on a station-by-station basis. Most of the identification codes are different than GHCN-M (and thus GISS), but I’ve managed to create a concordance of over 3300 station identifications – and do not preclude the possibility of further gains. I’ve created time series of monthly means for these 3300 or so GHCN-D series. As a first cut, I simply took a monthly average of available values, without requiring a minimum number of values to constitute an average (which I usually do and would probably do if I re-run the results.) I then calculated the monthly mean as the average of the mean monthly minimum and mean monthly maximum – in some cases, there would be different numbers of measurements.

The figure below shows the difference between the USHCN “raw” monthly mean and my calculation from daily information for a station (Kalispell MT) with an excellent match. There are rounding differences, but the two versions clearly reflect the same provenance. In this case, I presume that the small spike differences result from some procedural difference in calculation of monthly averages. While the differences appear attributable to rounding, the differences are not truly random: there are far more +0.1 differences than -0.1 differences, but this is unrelated to time.

ghcnd6.gif
Figure 1. USHCN Raw monthly (NOAA) minus monthly average calculated from GHCN-D. (I was at Kalispell airport once in the late 1970s and had an amusing experience there.)

Next here is the same plot for Detroit Lakes MN, a station which had a puzzling jump around 2000 in the original NASA version (a jump that could be attributed in part to the Y2K error.) This particular error has now been patched by NASA. In this case, the tracking looks very similar to the Kalispell tracking from 1950 to about 1980. But in the late 1990s-2000s, the USHCN Raw version (and thus the downstream versions) jumps up relative to the average calculated from GHCN-D daily information. Why is this?
ghcnd5.gif
Figure 2. As Figure 1, but for Detroit Lakes MN.

I parsed through about 40 such plots, most of which were in between Kalispell and Detroit Lakes in appearance. But there were a couple of oddballs: here’s one. It looks like the USHCN Raw version must be spliced from two different GHCN-D stations, with values after 1980 or so from the present station and earlier values from some other station.

ghcnd8.gif

Here’s a station (Dillon MT) which has a somewhat similar appearance of being spliced – only this time, it looks like the USHCN station is drawn from the GHCN-D data set prior to 1980 and perhaps some other related source after 1980.

ghcnd7.gif

The puzzle that needs to be resolved is the exact relationship between the USHCN “Raw” and GHCN-D data. If this can be sorted out, then NASA could make a substantial gain in the timeliness of their reporting.

GHCN-D versions of USHCN stations are current through early March 2008. Right now NASA’s USHCN data is only current to March 2006 – the date of the most recent GHCN update. Following a CA suggestion, NASA is moving to make its USHCN stations more current by adopting the USHCN (NOAA) source, which is more current than the versions at GHCN-M or CDIAC,

However, the GHCN-D data is truly current. NASA already uses “raw” USHCN data for its current results, using a patch to splice each station to the FILNET version used for historic values. If monthly averages calculated from GHCN-D data were used instead of GHCN-M data, then NASA could report USHCN stations right through to February 2008 (and keep current) instead of the current system of being up to two years out of date for USHCN stations. (A better system would be for NASA to write NOAA and ask them to update the USHCN data set on a monthly basis, which should be trivial to program and dispense with the patch altogether.)

Gaining two years in report timeliness for USHCN stations is a small thing but worth doing. In some forthcoming posts, I’ll discuss how NASA can gain nearly 20 years in reporting timeliness for many international stations.

By Stephen McIntyre | Posted in Surface Record | Tagged | Comments (51)

No Data For Some NASA Stations

Mar 4, 2008 – 10:06 AM

I’ve been parsing through the NASA data set in order to evaluate the prevalence of NASA using obsolete data versions e.g. a Rurrenabaque version ending in 1989 when data is available to 2008).

As I was doing so, I noticed that a surprising number (97) of NASA stations included in their inventory (http://data.giss.nasa.gov/gistemp/station_data/v2.temperature.inv.txt) appear to contain no information whatever in the actual NASA data set, even though some of these stations have easily available information. One notices the absence when you try to make inventories, which is how I happened to notice their absence. The absence of data from these stations won’t “matter” for any ultimate calculations, but it reflects poorly on the craftsmanship.

Here is a list of 97 stations identified by my algorithm as not having any dset=1 data. I’ve spot checked about 10 sites at the GISS website and verified the absence for these 10 sites. It’s possible that I’ve misallocated a few, but, even if I have, the issue is why NASA doesn’t have the data sets.

Some surprising series said to have been used, but not present in the data set, are Bern (Switzerland), a number of German sites (Braunschweig, Bremen etc.), Lund (Sweden). NASA has thanked regular CA reader Hans Erren here for drawing one data set to their attention. Perhaps other CA readers could communicate with NASA directly and help them locate temperature data for Bern, Switzerland (where WMO offices are located) and the other tricky spots mentioned in the list. OK, I’ve started with Bern:

Memo to: Reto Ruedy and James Hansen

Monthly temperature data for Bern, Switzerland is at http://www.meteoswiss.admin.ch/web/de/klima/klimaentwicklung/homogene_reihen.Par.0016.DownloadFile.ext.tmp/bern.txt

row id site
14 10160457000 MOSTAGANEM VI
203 11562464001 HURGHADA
401 13465201000 LAGOS/IKEJA
403 13465236000 WARRI
404 13465243000 LOKOJA
416 13761641001 GOREE SENEGAL
884 20554945001 JUXIAN
1199 20743333000 PORT BLAIR
1352 21047696001 YOKOSUKA
1567 21941560000 PARACHINAR
1748 22227995000 SAMARA (BEZEN
1749 22228028001 USSOLJE USSR
1750 22228044001 BOGOSLOWSK USSR
1754 22228240000 NIZHNYJ TAGIL
1761 22228630000 ZLATOUST
2048 30382400000 FERNANDO DE N
2270 31480444000 CIUDAD BOLIVA
2692 40371730004 RUEL,ON
3009 40371913001 CHURCHILL FACTORY
3224 41476220001 CIUDAD GUERRERO, CHIHUAHUA
3235 41476258002 QUIRIEGO, SONORA
3248 41476373000 TEPEHUANES,DG
3314 41476695001 CHAMPOTON, CAMPECHE
3327 41476775003 CANTON, OAXACA
4196 42572491003 HOLLISTER USA
5323 44078526001 ANNAS HOPE, ST. CROIX VIRG
5690 50194788000 KEMPSEY
6172 60311020001 HOHENFURTH AUSTRIA
6182 60311320001 WILTEN AUSTRIA
6184 60437851000 BAKU (DENDROP
6228 61111438001 SCHOSSL CZECH
6230 61111502001 TETSCHEN CZECH
6232 61111518001 SMETSCHNA CZECH
6234 61111643001 NEURODE CZECH
6235 61111643002 HOHENELB CZECH
6236 61111659001 DEUTSCHBROD CZECH
6237 61111679001 LANDSKRON CZECH
6239 61111735001 LEOBSCHUTZ CZECH
6252 61326038001 BALTISCHPORT USSR
6254 61326231001 DORPAT USSR
6255 61326233001 FELLIN USSR
6258 61402844001 TORNEO FINLAND
6261 61402874001 CARLO FINLAND
6265 61402912001 WORO FINLAND
6279 61507055001 MONTDIDIER FRANCE
6281 61507090000 METZ/FRESCATY
6288 61507169001 CHALONS FRANCE
6299 61507460000 CLERMONT-FERR
6301 61507471001 LE PUY FRANCE
6303 61507491001 SAINT BERNHARD FRANCE
6322 61710091001 PUTBUS E.GERMANY
6325 61710152001 EUTIN W.GERMANY
6326 61710156000 LUEBECK-BLANK
6328 61710170001 ROSTOCK E.GERMANY
6330 61710180001 SYLT E.GERMANY
6331 61710181001 STRALSUND E.GERMANY
6334 61710224000 BREMEN
6335 61710249001 LUNEBERG W.GERMANY
6337 61710315001 MUNSTER W.GERMANY
6340 61710348000 BRAUNSCHWEIG
6346 61710396001 FRANKFURT AMODER E.GERMANY
6348 61710413001 BOCHUM W.GERMANY
6349 61710425001 ARNSBERG W.GERMANY
6351 61710444000 GOETTINGEN
6353 61710453001 KLAUSTHAL W.GERMANY
6355 61710466001 HALLE E.GERMANY
6356 61710466002 KOTHEN E.GERMANY
6359 61710476001 TORGAU E.GERMANY
6363 61710513001 KOLN W.GERMANY
6364 61710515001 KOBLENZ W.GERMANY
6368 61710555001 ARNSTADT E.GERMANY
6390 61710776000 REGENSBURG
6392 61710838001 AUGSBURG W.GERMANY
6454 62316022001 TRENTO ITALY
6466 62316095000 PADOVA
6532 62316410002 PALERMO ITALY
6562 62626238001 IDWEN USSR
6565 62626425001 MITAU USSR
6583 63306380000 MAASTRICHT AP
6591 63401052001 HAMMERFEST NORWAY
6638 63512280001 ARYS POLAND
6662 63512520001 NYSA POLAND
6708 63715235001 STANISLAV RUMANIA
6731 63826063001 KRONSTADT USSR
6746 63827453001 BALACHNA USSR
6763 63834560001 SSAREPTA USSR
6773 63834949000 STAVROPOL’
6857 64502456001 KREUZBURG SWEDEN
6866 64502627001 LUND SWEDEN
6868 64606630001 BERN SWITZERLAN
7161 65033945001 SSEVASTOPOL USSR
7163 65033966001 ENISSALA USSR
7168 65034523001 LUGAN USSR
7271 70089060000 Primavera
7313 70089612000 Casey_New_Airstrip
7333 70089820800 Dome_F
7336 70089833900 D_17

By Stephen McIntyre | Posted in General | Comments (58)

NASA Follows CA Recommendation

Mar 4, 2008 – 8:01 AM

On Feb 20, 2008, I wrote a post reviewing the provenance of various versions of an individual USHCN station (Lampasas), observing that a much more recent version was available at NOAA than at CDIAC) (the source used by NASA. I made the following recommendation:

Regardless of whether these station histories “matter”, surely there’s no harm in NASA (and GHCN) adopting rational approaches to their handling of the USHCN network. To that end, I would make several recommendations to NASA:
1. Use the NOAA USHCN version rather than the stale CDIAC and/or GHCN versions.
2. Lose the splice and the patch.
3. Use USHCN interpolations rather than Hansenizing interpolations.
4. Use TOBS or perhaps MMTS, and if MMTS is used, ensure that NOAA places this online.

A reader observed that, on March 1, 2008, NASA announced the implementation of the first recommendation.

March 1, 2008: Starting with our next update, USHCN data will be taken from NOAA’s ftp site rather than from CDIAC’s web site. The file will be uploaded each time a new full year is made available. These updates will also automatically include changes to data from previous years that were made by NOAA since our last upload. The publicly available source codes were modified to automatically deal with additional years.

No mention was made of the CA recommendation (although they seemed to be aware of the CA discussions, as they altered the legend on one of their station inventory webpages inserting a caption to a series then under discussion here.)

They didn’t mention anything about the patch – recommendation 2. It will be worth checking to see how they implemented the new version. The most logical approach (recommended by CA) would have been to use the NOAA USHCN Filnet version, which did away with any need for a patch (this patch arose out of the Y2K correction – they calculated patches for all 1221 USHCN stations so that there was no Y2K step between the GHCN Raw version and CDIAC SHAP/Filnet versions). If they use the NOAA USHCN Filnet version consistently, then there is no need for the calculation of a patch; plus, their series will be traceable back to its sources.

It is, of course, possible that they’ve continued to pull post-2005 data from USHCN Raw (NOAA up to date version) and pre-2005 version from USHCN Filnet and continued to estimate a patch between the two. It would be pretty silly if they did and I hope that they don’t. (The new method is not yet implemented in the online database).

They’ve slightly edited their methodology page to reflect the changed procedure. As a source, instead of CDIAC, they now show:

For US: USHCN – ftp://ftp.ncdc.noaa.gov/pub/data/ushcn
hcn_doe_mean_data.Z
station_inventory

This doesn’t make clear which USHCN version they use – something that should be shown in this page. They continue to describe a splicing step. This may simply be an oversight or they may plan to continue splicing.

Replacing USHCN-unmodified by USHCN-corrected data:
The reports were converted from F to C and reformatted; data marked as being
filled in using interpolation methods were removed. USHCN-IDs were replaced
by the corresponding GHCN-ID. The latest common 10 years for each station
were used to compare corrected and uncorrected data. The offset obtained in
way was subtracted from the corrected USHCN reports to match any new incoming
GHCN reports for that station (GHCN reports are updated monthly; in the past,
USHCN data used to lag by 1-5 years).

As to the headline – NASA did not credit CA in announcing the changed procedure. Perhaps the timing was simply coincidence. In any event, the net result is a slight improvement in NASA’s methodology and so we can all take some comfort in that.

By Stephen McIntyre | Posted in General, GISTEMP, GISTEMP Replication, Surface Record | Comments (35)

Atmoz Agrees on USHCN Adjustment Defect

Mar 2, 2008 – 2:16 PM

A theme in many recent posts has been whether the USHCN and NASA adjustments are successful in achieving their goals.

On a number of occasions, we’ve observed that the USHCN station history (SHAP) adjustment appears to be an odd statistical procedure and can be objectively seen to be unsuccessful in picking up recorded station moves. This issue was re-visited in connection with Lampasas TX, where a 2000 move to a non-compliant location was not identified and corrected for by the USHCN adjustment algorithm. In a post on Lampasas, I observed that the SHAP/FILNET algorithm seemed to have the effect of blending stations, in the following terms (though similar observations have been made on other occasions):

My impression of the impact of the SHAP/Filnet adjustments is that, whatever their stated intention, they end up merely creating a blend of good and bad sites, diluting the “good” sites with lower quality information from sites that are “bad” in some (objective) sense. When this version gets passed to Hansen, even his “unlit” sites no longer reflect original information, but are “adjusted” versions of unlit sites, in which it looks to me like there is blending from the very sites which are supposed to be excluded in the calculation.

Atmoz has now analyzed an arbitrary USHCN site (Saguache), mentioning Watts Up (but not CA), concluding:

In this post, I examined one surface station record to determine the effects of microsite bias. In doing so, I found that the SHAP adjustment as applied by NOAA does not account for all the station moves in the station history. A simple and tractable correction method is outlined in this case study which uses regional anomalies to correct stations for local effects.

This is the third occasion [Lampasas, TX; Miami, AZ] where SHAP corrections have been documented to not fully account for station moves. Furthermore, this analysis was done on a random station in the USHCN; it was not cherry-picked to prove a point. This suggests the SHAP algorithm does not correct for all microsite issues related to station moves. People using the SHAP-corrected data should be aware that not all microsite biases have been removed, and they should attempt to account for these issues themselves.

At this point, I’ve not evaluated whether Atmoz’ proposed method is more successful than the USHCN method. However, it’s nice that a third party has confirmed that the USCHN adjustment algorithm has failed to identify station moves – a point made here and at Watt’s Up, occasioning some unwarranted derision elsewhere.

By Stephen McIntyre | Posted in General | Tagged | Comments (47)

Hansen and "False Local Adjustments"

Mar 1, 2008 – 4:59 PM

Over the last few days, I’ve shown that Hansen et al 1999 illustrated and discussed the effect of the NASA adjustment for two stations (Phoenix, Tokyo) where the NASA urban adjustment yielded the expected adjustment (denoted in these posts as a “positive” adjustment). In an earlier post, I’d observed that negative urban adjustments (i.e. for nonclimatec urban cooling) had occurred in some Peruvian stations, followed by a post carrying out an inventory of all NASA adjustments – the number of negative adjustments in the ROW proved to be only slightly lower than the number of positive (expected) adjustments.

Unfortunately Hansen et al 1999, the primary reference, did not contain a systematic discussion of any sites with negative adjustments. However, Hansen et al were aware of the existence of negative urban adjustments and it will now be useful to review the original account of the present-day NASA adjustments.

Here’s an important paragraph from Hansen et al 1999:

Examination of this urban adjustment at many locations which can be done readily via our website shows that the adjustment is quite variable from place to place and can be of either sign. In some cases the adjustment is probably more an effect of small-scale natural variability of temperature (or errors) at the rural neighbors rather than a true urban effect. Also the nonclimatic component of the urban temperature change can encompass many factors with irregular time dependence such as station relocations and changes of the thermometer’s environment, which will not be well represented by our linear adjustment. Such false local adjustments will be of both signs and thus the effects may tend to average out in global temperature analyses but it is difficult to have confidence in the use of urban records for estimating climate change. We recommend that the adjusted data be used with great caution, especially for local studies.

Later in the paper, Hansen added the following:

local inhomogeneities are variable; some urban stations show little or no warming, even a slight cooling relative to rural neighbors. Such results can be a real systematic effect e.g. cooling by planted vegetation or the movement of a thermometer away from the urban center or a random effect of unforced regional variability and measurement errors. Another consideration is that even rural locations may contain some anthropogenic influence.

I didn’t notice any other relevant discussions, but will amend this post if any other relevant quotes are brought to my attention.

Let’s review the negative adjustment of 3.3 deg C at Puerto Maldonado in the context of these reviews.

First, I submit that a negative adjustment of 3.3 deg C rises above a “slight cooling relative to rural neighbors”, In an engineering-quality assessment, such results would require specific investigation and explanation.

Second, while “cooling by planted vegetation” can be a feasible mechanism for a type of negative urban heat island in desert settings e.g.here , it seems implausible that this has affected Puerto Maldonado, which is located on an Amazon tributary, or that this is relevant to the vast majority of sites receiving negative urban adjustments.

Third, while we know virtually nothing of the metadata for Puerto Maldonado, it seems unlikely that the cooling relative to “rural” neighbors is due to the “movement of a thermometer away from the urban center”.

Fourth, while I know nothing of the local particulars of Puerto Maldonado climate relative to (say) nearby Cobija, I’d be amazed if there was a 3.3 deg C swing due to “unforced regional variability”. One could certainly not just assume this without some kind of proof.

It seems by far the most likely that the inconsistency between Puerto Maldonado and Cobija is due to something in the staiton histories – some change in instrumentation at Puerto Maldonado or, perhaps in the Bolivian sites, or perhaps both. In the US, where extensive metadata is available, step changes for station moves or instrument changes are included in the various USHCN adjustments (TOBS, MMTS, SHAP, FILNET), none of which are done for the Peruvian and Bolivian stations and is not a “true urban” adjustment.

In Hansens’s terminology, this adjustment would be a “false local adjustment”.

“Averaging Out”

Hansen postulated in very guarded language that these “false” local adjustments would cancel out:

Such false local adjustments will be of both signs and thus the effects may tend to average out in global temperature analyses.

In an engineering-quality study (as opposed to an exploratory scientific article), it would obviously be unacceptable to leave matters in such a state. An engineer would have been obliged to determine whether the effects actually did average out and would not have been permitted to simply leave the matter hanging.

Note that Hansen did not limit the concept of “false local adjustments” to negative adjustments. He clearly contemplated and stated that false local adjustments (i.e. adjustments that did not adjust for urban effect but for local station history issues) would be “of both signs” and, as noted above, that false positive and false negative local adjustments would “average” out.

In my previous post, I calculated the total number of positive and negative NASA adjustments. Based on present information, I see no basis on which anything other than a very small proportion of negative urban adjustments can be assigned to anything other than “false local adjustments”. Perhaps there are a few incidents of vegetative cooling resulting in a true physically-based urban cooling event, but surely this would need to be proved by NASA, if that’s their position. Right now, as a first cut, let’s estimate that 95% of all negative urban adjustments in the ROW are not due to “true urban” effects i.e. about 1052 out of 1108 are due to “false local adjustments”.

On the reasonable assumption that there will be an equal number of positive “false local adjustments” as negative “false local adjustments”, this will yield a total of approximately 2100 “false local adjustments” out of a total population of 2341 adjustments (disregarding bipolar adjustments.) In other words, there is a valid case that about 90% of all NASA adjustments are “false local adjustments”.

If the purpose of NASA adjustments was to do station history homogenizations (a la USHCN), then this wouldn’t matter. But the purpose of the NASA adjustments was to adjust for the “true urban” effect”. On this basis, one can only conclude that the NASA adjustment method is likely to be completely ineffective in achieving its stated goal. As other readers have observed (and anticipated), it appears highly likely that, instead of accomplishing an adjustment for the “true urban effect”, in many, if not most cases, the NASA adjustment does little except coerce the results of one poorly documented station to results from other equally poorly documented stations, with negligible improvement to the quality of whatever “signal” may be in the data.

This does not imply that the NASA adjustment introduces trends into the data – it doesn’t. The criticism is more that any expectation of using this methodology to adjust for urban effect appears to be compromised by the overwhelming noise in station histories. Needless to say, the problems are exacerbated by what appears to be poor craftsmanship on NASA’s part – pervasive use of obsolete station versions, many of which have not been updated since 1989 or 1990(!), and use of population data that is obsolete (perhaps 1980 vintage) and known to be inaccurate.

Some readers have wondered why Hansen even bothered with the entire NASA adjustment project. That seems a very reasonable question. There are a lot of stations where there is no adjustment – wouldn’t it make sense just to use these stations? This takes you into the metadata problem – right now Hansen shows a lot of ROW “rural” stations, but how many of them are actually tropical towns and cities?

I’ll discuss this further tomorrow.

By Stephen McIntyre | Posted in Hansen, Surface Record | Tagged , | Comments (70)

Positive and Negative Urban Adjustments

Mar 1, 2008 – 10:33 AM

A few days ago, I commented on the surprisingly large negative urban adjustments made by NASA at several Peruvian stations. I’ve now calculated the maximum negative and maximum positive urban adjustments at all NASA stations – something that I was able to do only because of my scraping of NASA data from their website (something that caused some controversy last year.) The results of these calculations have been added into the giss.info.dat (ASCII) data set at CA/giss data directory (The core of this is taken from the GISS station.inv file.)

While Hansen et al 1999 notes in passing that urban adjustments can be “of either sign”, neither it nor any other survey actually carries out a simple inventory. Indeed, a notable feature of the Hansen urban adjustment a a statistical method is that its efficacy is not actually demonstrated on a statistical data set of known properties, but is merely asserted and then implemented in an important practical setting – a practice that we’ve seen elsewhere in climate articles.

I’ve classified the 7364 stations in the GISS network into the following categories (with a cross-stratification to U.S. and ROW):
1. Positive urban adjustment (this is the Phoenix, Tokyo “expected” case, illustrated in Hansen et al 1999)
2. Negative urban adjustments (this is the counterintuitive case encountered at Puerto Maldonado and discussed recently)
3. “Bipolar” adjustments (this is an adjustment which is negative in one part and positive in another. These arise from the operation of the two-legged adjustment, but the interpretation of such cases is not discussed in Hansen et al 1999, 2001. In the inventory below, these occur in both the Positive and Negative stations and are deducted in making a total.)
4. Not used. (These are stations in the data base for which no adjusted version is calculated. In the U.S., their exclusion seems to result from the record being too short (under 20 years). I haven’t canvassed ROW exclusions in detail, but, in a quick look, shortness seems to be the predominant factor, but the methodology also provides for exclusion if there are not 3 rural comparanda – this doesn’t seem to be a big hurdle since 1000 km is a large radius and “rural” is, to say the least, loosely interpreted as discussed elsewhere.)
5. Unadjusted. (These are stations that are Code 1 in the U.S. and R in the ROW.)

I remind readers that there is a noticeable difference between U.S. and ROW temperature histories, which received much publicity last summer in connection with Hansen’s “Y2K” error – in which Hansen and NASA observed that the U.S. accounted for only 2.5% of the earth’s surface and re-ordering or errors in the U.S. data were immaterial to the world results.

First, here is a table showing the inventories – take a quick look at the table and I’ll comment below.

   U.S.  ROW  Total
 Negative  740 (39%)  1108 (20%)  1848 (25%)
 Positive  1003 (52%)  1233 (23%)  2236 (30%)
 “Bipolar”l  324 (17%)  335 (6%)  659 (9%)
 Subtotal: Adjusted  1419 (74%)  2006 (37%)  3425 (47%)
 No Adjustment  353 (18%)  2220 (41%)  2573 (35%)
 Not Used  149 (8%)  1217 (22%)  1366 (19%)
 Total  1921 (100%)  5443 (100%)  7364 (100%)

There are many striking aspects to the adjustment inventory.

First, 74% of all U.S. stations are adjusted, while only 37% of ROW stations are adjusted. This is a statistically significant difference by any measure. Is this because the ROW stations are, on average, located in more rural settings than in the US? Or is it because of a difference in methodology (or metadata)? While no one to my knowledge has carried out the engineering-quality investigations necessary to resolve the matter, my impression is that the US has made a fairly concerted effort to maintain weather stations in rural settings (Orland, Miles City etc.) and that many ROW stations are in cities and small towns (especially airports). Using a consistent apples-and-apples population classification, I would be very surprised if this very large difference between U.S. and ROW classifications held up.

Second, negative urban adjustments are not an exotic situation. In the ROW, there are almost the same number of negative adjustments as positive adjustments. In the U.S., there are about 50% more positive adjustments as negative adjustments – again a noticeable difference to the ROW. Some commenters on my Peruvian post seemed to think that negative urban adjustments were an oddball and very anomalous situation. In fact, that’s not the case, negative adjustments are nearly as common as positive adjustments. As such, extreme cases (such as Puerto Maldonado) need to be analyzed and explained.

Bipolar adjustments account for 17% of U.S. stations, but only 6% in the ROW. These can be either “cup” or “cap” shaped, one of each being shown below:
adjust96.gifadjust97.gif

In my next post, I’ll discuss some very interesting obiter dicta in Hansen et al 1999 about negative urban adjustments.

In the meantime, I’ll illustrate the above tables by showing the locations of several classes of stations on a world map (apologies to Hu McCulloch for not implementing a Mollweide projection on this occasion.)

First, here is a location map for all GISS stations with negative urban adjustments. The color coding is: red: greater than 2 deg C; salmon – 1 to 2 deg C; pink – 0 to 1 deg C. Peru definitely has a noticeable number of extreme examples (Pucallpa, Puerto Maldonado, Cuzco plues Piura, which I’d not mentioned), so I guess I do sometimes have sharp eyes for picking anomalies out of large data sets. Other ROW cities with negative urban adjustments exceeding 2 deg C are : Asmara, Nouadhibou, Darbhanga, Krasnovodsk, Sancti Spiritu, Praha/Ruzyne (which will no doubt intrigue Luboš Motl) and Beirut/Beyrouth Airport.

adjust89.gif

Next here is a corresponding map for all stations with positive urban adjustments.
adjust90.gif

Third, here is a corresponding map for all (used) stations with no adjustments. This is an important map to consider, because readers need to keep squarely in mind that there are a LOT of stations which aren’t adjusted and that, even if you didn’t use any of the adjusted stations, there still would be a lot of stations in the index. Some readers have wondered why Hansen doesn’t simply use the “good” stations in his index – and I must say that this question certainly crosses my mind as well.

adjust91.gif

Here is a histogram of maximum positive and maximum negative adjustments for all adjusted ROW stations, showing somewhat similar distributions, though there is a very slight balance towards positive adjustments.

adjust48.gif


“Long” Stations
In discussions last summer about comparing the 1930s to the 2000s, NASA made the unarguable observation that the U.S. was only 2,5% of the world’s surface (6% land surface). I thought that it would be interesting to illustrate the population of stations which were present in 1930 and 2000 or later, stratifying the results by whether the stations were positive, negative and unadjusted. First here are the stations with negative adjustments:
adjust2.gif

Next here are the long stations with positive urban adjustments.

adjust3.gif

Finally, as before, here are the long stations with zero adjustments:
adjust4.gif

Obviously, while the U.S. is only a small fraction of the world’s land surface, it provides the vast majority of the long station records.

By Stephen McIntyre | Posted in Hansen | Comments (133)

Phoenix and Tokyo: "Traditional" UHI

Feb 28, 2008 – 11:25 PM

In my discussion of Peruvian stations, I noted several examples of negative urban adjustments. A couple of readers inquired as to whether there were any examples of the opposite effect. In fact, Hansen et al 1999, the primary reference for the present-day GISS adjustment methodology provides using two such examples (the only ones discussed): Phoenix and Tokyo. Here is their Figure 3 showing the adjustment process for both sites. Hansen provided the following discussion of these sites:

The measured and adjusted temperature records for Tokyo and Phoenix are shown in Figure 3. These are among the most extreme examples of urban warming but they illustrate a human influence that can be expected to exist to some degree in all population centers. Tokyo warmed relative to its rural neighbors in both the first and second halves of the century. The true nonclimatic warming in Tokyo may be even somewhat larger that suggested in Figure 3 because some “urban” effect is known to occur even in small towns and rural locations [Mitchell 1953; Landsburg 1981]. The urban effect in Phoenix occurs mainly in the second-half of the century. The urban-adjusted Phoenix record shows little temperature change.

adjust49.gif
Hansen et al 1999 Figure 3. (a, b) Measured time series of temperature for Tokyo, Japan, and for Phoenix, Arizona; (c, d) adjustments required for linear trends of measured temperatures to match rural neighbors for the periods before and after 1950; and (e, f) adjusted (homogenized) temperatures.

As an exercise, I updated Hansen et al 1999 Figure 3 using current GISS values for Phoenix and Tokyo, shown in the figure below, drawn after the style of the earlier figure. The Phoenix adjustment has been reduced slightly in the earlier portion of the series by about 0.3 deg C, resulting in a slightly increased adjusted Phoenix trend relative to 1999. My guess is that the increased Phoenix adjustment results from changes in the USHCN adjustments between 1999 and 2001. These changes ended up increasing the trend in the USHCN network – and presumably the trend in Code 1 stations used as a comparandum for Phoenix was also increased.

There has been a slightly larger reduction in the Tokyo adjustment – the current adjustment at 1905 is about 1.2 deg C, while the 1905 adjustment in the 1999 version looks to be about 1.7 deg C – for an reduction in UHI adjustment of about 0.5 deg C between 1999 and 2007. In addition, the adjusted series has been shortened from an 1885 start to a start around 1905. In both cities, the adjusted trend is higher in 2007 than it was in 1999.

adjust85.gif

I’m not suggesting any malfeasance – I presume that both these effects occur from the operation of the GISS algorithm; but exactly why the algorithm has produced such changes seems to be a relevant question and one that is not discussed anywhere by GISS.

I also checked the rankings of Tokyo and Phoenix among all UHI adjustments – ranking by the largest total adjustment for each station. In the present network, Phoenix ranks 3rd overall and, is, as Hansen says, an “extreme” case. Tokyo is in the top 80 or so, but is not in the top 10.

In my earlier post on Peruvian stations, I identified some high negative UHI adjustments and have been criticized for selecting extreme cases to discuss. I disagree with this comment from the point of view of data analysis, something that I pride myself on: analysis of extreme cases helps groundtruth any algorithm to see whether it makes sense. It’s something that I routinely do this on many data sets. In this case, Hansen had already done analysis of extreme positive cases and was able to develop a highly plausible interpretation of the results.

Interpreting the negative urban adjustments is more challenging – I’ll return to this tomorrow.

By Stephen McIntyre | Posted in Hansen, Surface Record | Comments (117)