I am in the process of comparing by station the USHCN temperature series Version 2 adjusted, Urban adjusted, Filenet adjusted, TOBS adjusted, Areal adjusted and GISS temperature series. I found that the default GISS temperature series that I used in the comparison in the previous post on this thread is comparable to the USHCN Filenet and that another series listed as Homogenous adjusted is probably closer to the USHCN Version 2. To show a more complete comparison I have listed the previous regressions and the new one for the GISS Homogenous adjusted series below. The trends are in degrees C per decade.

I also have included a plot of the difference between the USHCN Version 2 temperature series and the GISS Homogenous for the period 1895-2006. The differences are in degrees F. While the regressions of these two time series would seem to indicate that there are similarities in the trends over time, the difference plots shows some substantial differences from year to year.

Regression values from regressing annual temperatures over the time period 1895-2006:

Ver2 Adjusted Temperature:
Adj. R^2 = 0.42; Trend = 0.117; Stdev Trend = 0.013; Lag1 r = 0.072

Urban Adjusted Temperature:
Adj. R^2 = 0.43; Trend = 0.144; Stdev Trend = 0.016; Lag1 r = 0.263

Filenet Adjusted Temperature:
Adj. R^2 = 0.64; Trend = 0.221; Stdev Trend = 0.016; Lag1 r = 0.262

GISS Temperatures:
Adj. R^2 = 0.62; Trend = 0.220; Stdev Trend = 0.016; Lag1 r = 0.185

GISS Homogenous Adjusted Temperatures:
Adj. R^2 = 0.28; Trend = 0.11; Stdev Trend = 0.016; Lag1 r = 0.180

David Smith, I would suggest in doing your flow chart as noted in your post above that you review what Steve M has previously provided at Station Data at this link: http://www.climateaudit.org/?page_id=1686

I wanted to comment in more detail in this post on my findings from above with regard to the trends that are calculated from the various USHCN temperature versions and now also including the GISS temperature data for the University of AZ station.

What I find most interesting and worthy of further discussion and analysis is the relatively large adjustment in the USCHN temperature data on going from filenet to urban adjusted. It amounts to approximately a 0.7 degree C per century adjustment or 1/3 of the temperature trend prior to the UHI adjustment. I would think that such large adjustments could be subject to large errors and/or uncertainties. Remembing also that the Karl adjustments for UHI effects for the USHCN temperatures are based on populations and considering that simplistic approach in light of what we have found for the Watts and team CRN quality findings, regardless of population considerations, and the correlations with temperature trends, one has to have some major doubts about the uncertainty of those adjustments over time.

When we compare the two different UHI adjusted USHCN temperature series, i.e. the urban adjusted and Version 2 adjusted, we see further uncertainties in the comparison of the 1.17 degrees C per century for Ver2 and 1.44 degrees C per century for the Urban adjusted. Also note the lower Stdev of the trend for the Ver 2 series and its lower lag 1 auto correlation of the regression residuals when compared to the comparable values for the urban adjusted series.

When the GISS series is thrown into the mix it appears to be approximating the filenet adjusted USHCN series and shows no adjustments for UHI. The Ver 2 USHCN minus GISS series is shown in the plot below. I find this difference in temperature series most curious and worth some investigation with other stations. The GISS series was extracted from this link:

http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=425722740020&data_set=1&num_neighbors=1

Regression values from regressing annual temperatures over the time period 1895-2006:

Ver2 Adjusted Temperature:
Adj. R^2 = 0.42; Trend = 0.117; Stdev Trend = 0.013; Lag1 r = 0.072

Urban Adjusted Temperature:
Adj. R^2 = 0.43; Trend = 0.144; Stdev Trend = 0.016; Lag1 r = 0.263

Filenet Adjusted Temperature:
Adj. R^2 = 0.64; Trend = 0.221; Stdev Trend = 0.016; Lag1 r = 0.262

GISS Temperatures:
Adj. R^2 = 0.62; Trend = 0.220; Stdev Trend = 0.016; Lag1 r = 0.185

Now that I have the latest and greatest version of the Tucson final adjusted temperature (Version 2), I thought it would be appropriate to compare it to the urban and filenet versions from earlier times. The graph below shows that comparison.

Additionally I did regressions for these three temperature series for the time period 1895-2006 and report a summary of those calculations below. Lag 1 r is the lag 1 autocorrelation for the regression residuals. The trends are in degrees C per decade.

Ver2 Adjusted Temperature:
Adj. R^2 = 0.42; Trend = 0.117; Stdev Trend = 0.013; Lag1 r = 0.072

Urban Adjusted Temperature:
Adj. R^2 = 0.43; Trend = 0.144; Stdev Trend = 0.016; Lag1 r = 0.263

Filenet Adjusted Temperature:
Adj. R^2 = 0.64; Trend = 0.221; Stdev Trend = 0.016; Lag1 r = 0.262

David, I think we should not bother with the urban adjusted differences between (UTmin+UTmax)/2 and UTave and here is why.

USHCN Version 2 data became available in recent months which has replaced the Karl urbanization adjustment and siting adjustments with a ‘change point detection algorithm’ that NCDC` believes will better identify previously undocumented inhomogeneities.

http://icecap.us/images/uploads/US_Temperatures_and_Climate_Factors_since_1895.pdf

The USHCN Version 2 Serial Monthly Dataset
In the original HCN, the regression-based approach of Karl et al. (1988) was employed to account for urban heat islands. In contrast, no specific urban correction is applied in HCN version 2 because the change-point detection algorithm effectively accounts for any “local” trend at any individual station. In other words, the impact of urbanization and other changes in land use is likely small in HCN version 2.

http://www.ncdc.noaa.gov/oa/climate/research/ushcn/#urbanization

The link below gives the adjusted version 2 average, minimum and maximum temperatures for the 1221 USHCN stations. I extracted these data for Tucson, AZ station number 28815 and compared the Tave to (Tmin + Tmax)/2 and they agree completely. David, they have moved on and I suspect that we should follow suit.

ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/

So far, I have only been able to find an abstract of the Karl et al. (1988) paper on the urban adjustment. I have also noted that this adjustment leads only to a 0.06 degree change in the US temperature trend over the 20th century.

The USHCN data base contains urban adjusted monthly maximum (urban.max.Z), minimum (urban.min.Z),and mean (urban.mean.Z) temperature data (in hundredths of degrees fahrenheit) for the 1221 USHCN stations. There is also an urban mean data set (urban.mean.calc.Z) that is the mean monthly temperature calculated from the urban adjusted maximum temperature (urban.max.Z) and the urban adjusted minimum (urban.min.Z) data sets.

http://ingrid.ldgo.columbia.edu/(plotaxislength)540+def/SOURCES/.NOAA/.NCDC/.USHCN/detailed_documentation.html

I think I can rationalize the differences between the (Tmin+Tmax)/2 and Tave temperatures for the progressively more adjusted USHCN series prior to the urban adjusted, but given the readme statements from USHCN below, I cannot do the same for the urban adjustment.

I base that thought on the proposition that the urban adjustments are made to the maximum and minimum temperatures as noted by the evidence shown in the graphs in Re: David Smith (#66), for the Tmin and Tmax between the filenet and urban adjusted data series. I think that the answer may be in the Karl paper on the urban adjustment.
Perhaps an adjustment made to the maximum and minimum temperatures is also applied independently to the average temperatures.

Finally, I think it is worth noting that the urban adjustments appear large for Tucson in light of some of the data sets where these adjustments are minimized in importance. It is also interesting to see the inflection points in the adjustments and the large difference and direction in the adjustment to the minimum versus maximum temperatures – as David has noted in this same graph.

FILNET (Fill Missing Original Data in the Network):

Estimates for missing data are provided using a procedure similar to that used in SHAP. This adjustment uses the debiased data from the SHAP and fills in missing original data when needed (i.e. calculates estimated data) based on a “network” of the best correlated nearby stations. The FILNET program also completed the data adjustment process for stations that moved too often for SHAP to estimate the adjustments needed to debias the data.

Urban (Urban Warming Adjustment):

The final adjustment is for an urban warming bias which uses the regression approach outlined in Karl et al. (1988). The result of this adjustment is the “final” version of the data.

While I do that I think I’ll try to develop a step by step flowsheet to illuminate all the steps and decision points. The lowly flowchart can be quite helpful at times.

I need to recheck this observation but as I recall there was an additional adjustment to the urban adjusted temperature series beyond that for the UHI and it involved adjustments for missing and questionable data. That, however, does not make sense of the differences that we see going forward. I do know that when you work with the individual data series using the raw data some of these differences become more apparent. Too bad it is not explained in detail in the readme files.

If you are like me, you will want to attempt to find this difference yourself before going to the source.

I’ll try a graph:

The graph shows the difference between mean and (max+min)/2 for the UHI-adjusted data (red line) and also for the non-UHI adjusted data (blue line). As you note, both the red line and the blue line move towards zero as time progresses.

My question has to do with the difference between the blue line and the red line. That difference (blue minus red) becomes greater as time progresses. I don’t understand why. There is probably a simple and obvious explanation, but it escapes me at the moment.