Before proceeding, I needed to compare the Steve M breakpoint method from R with those described and graphed for global NOAA/NCDC temperature anomalies (1880-2006) in the paper titled, “Abrupt global temperature change and the instrumental record”, by
Matthew J. Menne, NOAA/NESDIS/NCDC, Asheville, NC and linked at:

http://ams.confex.com/ams/Annual2006/techprogram/paper_100694.htm

Menne was the main authority referenced for determining the breakpoints used in the USHCN temperature series adjustment for Version 2. The NOAA global data is linked here:

http://www.ncdc.noaa.gov/oa/climate/research/anomalies/index.php#means

The NOAA/NCDC Global Temperature Anomaly Breakpoints using the library(strucchange) method in R were at the years 1906 1945 1973 which are very close to those determined in Menne. The graph of this series with the breakpoints is presented below.

Secondly, before looking at breakpoints for V1 and V2 versions of USHCN, I wanted to look at the breakpoints for the V2 series for the period 1895-2006 for the contiguous US. For that calculation I found a single breakpoint at the year 1963 and the graph for it is listed below.

Thirdly, before proceeding I calculated the breakpoints for the differences series of USHCN V2 – GISS and USHCN V2 – USHCN V1. Those breakpoints are shown graphically below and the years were as follows:

V2-GISS: 1912, 1929, 1986

V2-V1: 1939, 1992

Finally, I calculated the breakpoints for the V1 and V2 versions for individual USHCN stations for the periods 1895-2006 and 1920-2006 and tabled the results as shown below. The stations were subdivided by the trend differences between the V1 and V2 versions into the largest differences and the smallest differences. All stations with small V1 to V2 trend differences, which were used in the analysis, had significantly positive trends.

The results of these breakpoint calculations points to perhaps some significant differences between temperature series for the US between USHCN V1 and 2 and between V2 and GISS. Without any a prior evidence or inklings for assigning the breakpoints to specific differences, at least at this point in the analysis, the breakpoints and time of occurrences in the series is just an interesting observation.

The analysis of breakpoints for the individual stations V1 and V2 series could perhaps reveal whether the methods of adjustment for the V2 series under or over adjusted. Over adjusting would imply that real climate caused breakpoints are improperly being adjusted out of existence. Under adjusting would be indicated by breakpoints remaining substantially unchanged for both versions V1 and V2. Those stations with the largest trend differences V1 to V2 could indicate larger adjustments of breakpoints while the smaller trend differences V1 to V2 could be evidence of less breakpoints needing adjustment.

The stations with the larger trend differences did tend to have more breakpoints, but it does not appear, from my somewhat meager numbers of stations studied, that the V2 version for those large trend difference stations has reduced numbers of breakpoints. In some cases the adjustment evidently only changed the timing of the breakpoint(s). In the stations that I analyzed, the group with the smallest trend differences actually showed a larger reduction, on going from V1 to V2, in breakpoints than the group with the largest trend differences.

I did my analysis using two time periods the longer 1895-2006 one and one for the period 1920-2006. The shorter period contains much less missing data and in my mind is probably more reliable. The question to be answered is whether the time period selection changes the breakpoints significantly. Overall the changes are not dramatic but there are changes in breakpoint numbers and time of occurrence.

Again I am not sure how to interpret these differences, but I do think that the authors of the USHCN V1 to V2 change would have done and reported some of the breakpoint analysis since their adjustment depend strongly on locating breakpoints.

As you have discovered connecting satellite data from different satellite sensors and integrating them is a very challenging effort. But what is important to note is satellite sensors are well characterized and can have internal calibration and be calibrated against known targets or against well calibrated ground sensors. The point is the satellite sensor is a single known entity with measurable or defendable precision/errors. Therefore their global measurements are self consistent to the highest degree.

As you noted the challenge of integrating these systems when they are updated or age, now contemplate the same problem with thousands of different ground sensors with disparate calibration or state information that may be completely outdated. Sensors that have been moved, replaced, deleted, etc. If you thought getting a defendable temperature profile out of a sensor that samples the globe for years and can be reference calibrated, think of the mess we have with ground sensors.

This logically leads me to conclude (without the need to do it mathematically) that the ground based sensor network that produces a global temperature cannot produce a more accurate measurement. It is impossible for a ill defined network of various sensors or varying quality to create a more accurate number than one sensor used over many years across the globe.

I may be over simplifying it, but that is an engineer’s back of the envelope quick test that usually gives the direction of the answer, if not also a reasonable magnitude.

I would bet that if someone used one of these satellite sensors to assess the ground sensor network used by NOAA and GISS they would not only discover the precision (or lack of it) for each sensor site, but could prove how poorly the ground network performs when compared to the satellite sensor.

Cheers, AJStrata

The Wilcoxon test is non-parametric, and so would not require correction for serial correlation of the residuals.

Surprises me a bit; transformation into ranks smoothes the outliers away, but it should not remove the autocorrelation structure ?

I agree. What I meant was that if you suspect a satellite sensor for temperature is degrading, you might get useful supplementary information if you look at other data gathered by other instruments to see if there is a non-temperature climate change at the same time. If quite a few diverse climate measures change, it lessens the likelihood of a satellite instrument problem. Sorry for my poor wording. The several station changes that I have studied give remarkably large temperature break points even though the change might be a few km at best. Importantly, they do not give greak points solely because the microclimate is different. In some datasets they give break points because adjusters assign them different trends before and after. There might be less inclination to adjust (say) rainfall records in hindsight. It’s messy.

Steve, In some ways it might have helped. If you look at the photos of tidy Australian BOM high quality stations in Australia, on BOM Web sites, you might feel that Anthony’s work highlighted the need for site cleanup. I do not know if it was aleady under way before then, but it continues. OTOH, how much adjustment of past results has been made is less clear.

Here’s another spaghetti graph to show that even relatively clean sites have adjustment problems. There was a station site change about 1942. There is guesswork in-filling of a small % of the daily data.

Would you be doing mathematical hara-kiri by assuming no change of any significance in 130 years?

You might see a similar frustration in my post 24 above. I can only suggest that we keep probing.

At the moment, methods like break points and other deconvolutions seem popular. There is scope to widen these analyses to include other climate data like rainfall, evaporation, Tmax and Tmin separately, rather than Tmean, etc, to try to distinguish iunstrument variables from climate variability. Sooner or later there will be better confidence in both hypotheses and data.

The correspondence of temperature data between different agencies has to continue under the microscope because the agreement is still not good enough for some purposes. Thank you, Dr Christy, for your openness there.

You make a good point that authors might look at their older papers and either issue a caution about findings or nullify them. Few authors seem to be doing this, which adds to the confusion, but it is a sustainable proposition that some past papers are no longer relevant because their temperature data basis was wrong. e.g there were past hypotheses based on Tmax and Tmin converging over decades, but that pattern might not have been so reliable everywhere.

Or do these scientists whose investigations depend strongly on these series being valid within the reported limits band together in an attempt to get an independent measure of the series? Also how leery must a scientist be of his conclusions changing when these series are periodically corrected? And does that concern tend to inhibit significant changes from being made or proposed?

10. Since 2003 UAH has used the AMSU on AQUA which has on-board propulsion, and thus rigorous station-keeping. As a result, no diurnal drifting occurs, so UAH needs no diurnal correction for that period – which gives evidence to our hypothesis that the RSS diurnal corrections are a bit too strong.

Yet the years for which AQUA has been used, 2003-8, show a pronounced minimum in the anomaly in May/June of ~0.2ºC which isn’t apparent in RSS and suggests to me that the UAH drift correction prior to AQUA might not be right.