In order to assess the 2006 anomaly in John V’s 2006 calculation reported previously, I’ve calculated a gridded anomaly for the U.S. using standard spatial smoothing software in R. This enables the calculation to be done in a few lines as explained below. I’ll also show results which reconcile most of the results to date. I’ve shown the details of calculation step so that there will be no confusion. I also want to show how simple it is to produce gridded data from station data. It only takes a couple of lines of code.
CRN Ratings
First I downloaded the current ratings from Anthony Watts from http://www.surfacestations.org/downloads/USHCN_stationlist.xls and saved it as a tab-separated ASCII file. I first deleted the comment columns and changed the apostrophes to _. This is important for R programs. I then added these ratings to my USHCN info file details as follows:
url=”d:/climate/data/station/watts”
#downloaded from http://www.surfacestations.org/downloads/USHCN_stationlist.xls
#some columns deletedl also replace ‘ with _
loc=file.path(url, “stationlist.txt”)
watts=read.table(loc,sep=”\t”,fill=TRUE,nrow=1221,header=TRUE)
details$watts=NA
test=match(details$id,watts[,”USHCN.ID”]);temp=!is.na(test); sum(temp) #1221
details$watts[temp]=watts[ test[temp],5]
details$watts[(details$watts==0)&!is.na(details$watts)]=NA
I then loaded my collation of USHCN station data and extracted the stations with CRN ratings of 1 or 2 as follows:
load(“d:/climate/data/station/ushcn/ushcn.collation.tab”)
temp=(details$watts< =2)&!is.na(details$watts);sum(temp)
index_good=(1:1221)[temp]
good=ushcn.collation[[2]][,index_good] #2544 53
M=ncol(good)
Anomaly Calculation
I then converted this to an anomaly basis and calculated annual averages. I did a separate calculation for 2006 using available months only. The functions anom and ts.annavg are available in scripts/gridcell/collation.functions.txt.
good.anom=good
for(i in 1:M) good.anom[,i]=anom(good[,i])
good.ann=ts( array(NA,dim=c(nrow(good)/12,M)),start=tsp(good)[1])
for(i in 1:M){ good.ann[,i]=ts.annavg(good.anom[,i])}
temp=(c(time(good.ann))>=1890)
good.ann=ts(good.ann[temp,],start=1890)
time0=c(time(good.ann)) #1890:2006
for(i in 1:M) good.ann[N,i]=mean(good.anom[(nrow(good.anom)-11):nrow(good.anom),i],na.rm=T)
Gridding
Gridding is a type of spatial smoothing for which there are a variety of standard functions and no need for people to ponder over triangles or hexagons (which don’t matter much) or to write new code. If one wants to vary smoothing parameters (in this perspective, GISS smooths much more than CRU), then one should do so in the context of standard software where there is working experience. In this case, I used the function interp from the R-package akima (which I had previously installed.) I used three “masks” to screen out values which otherwise would get interpolated into the ocean or into Canada (tho not much turns on this.) Because each gridpoint represents an equal area, I simply calculated the mean of the values.
library(akima)
N=length(time0)
annual=rep(NA,N)
for (j in 1:N){
X=data.frame(details[index_good,c(“long”,”lat”)],good.ann[j,]) #locations of “good” stations
names(X)=c(“long”,”lat”,”anom”)
temp=!is.na(X$anom)
gridded < – interp(X$long[temp], X$lat[temp], X$anom[temp])
gridded$z=gulfmask(gridded$z)
gridded$z=atlmask(gridded$z)
gridded$z=canadamask(gridded$z)
annual[j]=mean(gridded$z,na.rm=T)
}
annual.tobs=ts(annual,start=1890)
The mask functions were not particularly elegant. The akima package with this data yields a 40×40 grid and I simply set values at inappropriate gridpoints to NA as follows.
gulfmask=function(Z){
Z[21:31,1:2]=NA;Z[21:30,3]=NA;Z[22:30,4]=NA;
Z[23:30,5]=NA;Z[c(23:24,27:30),6]=NA; Z[27:28,7]=NA; Z}atlmask=function(Z){
Z[40,c(1:26,29:31)]=NA; Z[38:39,1:26]=NA; Z[37,1:23]=NA;
Z[36,1:16]=NA;Z[35,1:15]=NA; Z[34,1:13]=NA;Z[33,1:11]=NA;
Z}canadamask=function(Z){
Z[40,37:40]=NA; Z[37:39,35:40]=NA; Z[36,34:40]=NA;
Z[34:35,32:40]=NA;Z[32:33,29:40]=NA; Z[31,c(28:29,33:40)]=NA;
Z[30,36:40]=NA;Z}
Here’s an illustration of the points that are masked out by these masks. I didn’t bother with masks in other places because the algorithm didn’t interpolate over there, but it could be esily done.
Here’s an example of the type of annual map that is generated by the smoothing function. This was generated by doing a single step and plotting from the object gridded produced in the above routine as follows (using the R package fields). You’ll note that this smoothing function does not calculate gridded values at the edges which are an extrapolation from the stations. (This map also shows the stations used in the calculation.)
par(mar=c(3,3,2,1))
breaks0=seq(-3,3,.25)
image.plot (gridded$x,gridded$y,gridded$z, col=tim.colors(length(breaks0)-1),breaks=breaks0,ylab=””)
contour(gridded, add=TRUE,ylab=””)
points (X$long[temp],X$lat[temp], pch = 19,cex=.7)
US(add=TRUE,lwd=2)
title(main=paste(“USA “, time0[j]))
So it’s pretty easy to do a gridding calculation using standard software. Hansen’s program has very ad hoc statistical methods in which stations are adjusted from other stations and gridcells calculated. At the end of the day, the best approach to this aspect of his program will probably be to see what smoothing parameter best emulates his method and thereby restore it to a known statistical methodology.
COMPARISONS
First here is a comparison of my calculation using CRN1-2 data to NOAA results from 1890-2007. (I re-iterate for the N-th time that analyses on U.S. data are not representative of the ROW.) NOAA has values to Aug 2007 and I’ve estimated the last 4 months as the same as the corresponding 2006 months for now. NOAA has run warmer by about 0.2 deg for the past decade and was about 0.1 deg C cooler in the 1930s.
Second here is a comparison of CRN1-2 data to GISS pre-Y2K results. As you see the GISS with the Y2K error was running warm by about the same as NOAA – which may be one reason why they didn’t spot the error. GISS runs a bit warmer than NOAA in the early and mid portions of the graph due to their greater efforts to do an urban adjustment.
Third, here is a comparison of CRN1-2 data to GISS post-Y2K results (but not including the September change from SHAP to FILNET, which seems much too opportunistic.) Relative to CRN1-2, GISS tends to warm the cool periods (1960s, 1900s) and to cool the warm periods (1930s), reducing the size of the mid-20th century cooling. For the U.S., as noted before, by using the trends from “unlit” stations to establish trends – in a context where there are a lot of decent rural stations with long records – the GISS methodology for USHCN stations seems more appropriate than the NOAA (and probably the CRU methods). Unfortunately, this method is not used in the ROW, where there is nothing equivalent to the USHCN.
Finally, here is a reconciliation to John V’s annual results sent ot me yesterday. His 2006 results run much colder than the ones that I calculate- which is presumably an artifact of the 3 months stub period that he used in his calculation. So the anomaly with his 2006 results can be traced definitely to this (which shows up a “cool” difference in the graph below.)
So what does my calculation using CRN1-2 data look like: see below, which includes a 2007 estimate (using the 2006 difference between the CRN1-2 and NOAA to estimate 2007 CRN1-2 from the YTD NOAA results.) In this calculation, the warmest year remains 1934 by a margin of nearly 0.2 (enough to be “statistically significant” according to things that GISS has said, then 1921, then 1998, then 2006 in fourth place. Values since 1998 have been at uninterrupted high levels. Looking at the graph, it’s definitely noticeable – but is it the sort of change that’s inconsistent with the sort of stochastic behavior in the early part of the series – hard to tell here. If you cut the series off in 1997, you certainly wouldn’t have noticed anything odd about the 1990s. You can sort of see in retrospect why MBH98-99 were so excited about how warm 1998 and in such a rush to include it in their article. The main verification issues right now is why the ROW history is so different from the US – is it climatic or nonclimatic? And can one tell?
Climate Audit 
106 Comments
THanks Steve. What does CRN1,2 TOBS – CRN 1,2 NO TOBS look like?
That’s an interesting analysis. A few points come to mind:
I realized that my 2006 data included 10 months, not 3 months as I previously stated. I made the correction and apologized in the previous thread.
I think that trendlines would be informative. Eyeballing it, it looks like GISTEMP shows less warming over the century than CRN12. That is consistent with my results. Some sort of smoothing over the single-year data would also make it easier to identify any trends.
Where did you find USHCN TOBS data for the rest of 2006? I’ve been looking for it.
My results were for rural CRN12 stations (CRN12R) rather than all CRN12 stations. Your comparison of my results vs your results is therefore skewed. If I recall correctly, CRN12R shows slightly less warming than CRN12 in my analysis.
By the way, I’d prefer if you used “John V” or “OpenTemp” in showing my results. If you prefer to use my last name, it’s “Van Vliet”.
SteveMc said:
You’re right. Like most software developers I have been suffering from a bit of a not-invented-here syndrome. I will look around for an open-source toolkit for gridding or smoothing. Can you recommend some search terms to get me started?
#2. John, I have USHCN TOBS only up to Oct 2006 for most of these stations and used the 10-month average of anomalies.
If you icluded 10 months of 2006 data, I’m really puzzled why your 2006 is so low. Something must have gone awry – maybe with missing data somewhere – leading to your too low 2006.
Shouldn’t the corner in the southwest be removed from the grid? Without removing it, the stations in that part of the USA lower 48 will affect more land than they should.
John V, the open source code in R that I’ve posted here is, I think, exactly what you’re trying to do. The R package is akima; you might look at its manual. But I think that this post already does it.
#4 Steve McIntyre:
There was a bug in OpenTemp where it would generate yearly results based on incomplete months, but still divide by 12. It has been fixed. It now excludes any years without data for all 12 months. (There are more details in the previous thread).
Re #4
John V.,
You said
There are entire texts on the subject and no shortage of code developed. Some quick googling didn’t point me to any particular open-source toolkit but I did find a nice write-up that breaks down some of the issues at:
http://www.epa.gov/oar/oaqps/pm25/docs/dsisurfaces.pdf .
It may provide a good starting point for terminology to use in more focused searches.
Regards,
Earle
For irregular sampled data, I recommend kriging (well known in geochemical sampling), this gives you also an error estimate.
http://en.wikipedia.org/wiki/Kriging
Re my #5, I see now that the Akima interpolation did not extend into the Pacific in the southwest (as you said). Never mind.
be also aware that a rectangular lat long grid is not equal area
http://en.wikipedia.org/wiki/Map_projection#Equal-area
As with the many excellent plots rendered earlier by JohnV, this post once again brings me back to the question – just what in the heck happened between the mid/late 1910s and the early 1920s? Yes, it could be nothing more than an odd stochastic jump. Or, it could be something truly wierd. Either something wierd with the weather/climate, or, wierd with how data from prior to the 1920 were gathered, recorded, handled or adjusted. I look at really neat charts every day as part of running a business. If I say something like that on one of my charts, my hackles would be raised.
say s/b saw. Doh!
I can highly recommend generic mapping tools (GMT, freeware!) for better handles on cartography and mapprojections
http://www.soest.hawaii.edu/gmt/
take a look at these!
http://www.soest.hawaii.edu/gmt/gmt/gmt_examples.html
re 2
Wouldn’t the provision of a trend line be consistent with the assumption that there is a linear trend. What is the justification for this assumption? As someone has pointed out, there appears to a be a step-wise discontinuity around 1920 with the additional appearance of a step down in the 70s and a current step up in 1998. If this appearance has any validity a linear trend would not be of much use.
How does this correlate with population changes from, say 1930 to 2006. Has not the US population moved from the NE to the SW?
OK, here is the update for CRN 1,2R stations that shows the most recent move or change in equipment according to MMS (remember its NCDC so expect some problems like Woodstock, VA that Douglas Hoyt talked about in the last thread). But we can verify that when we get the B44 forms. I think that no data should be used in its current CRN rating prior to moves or equipment changes. I understand John V has been, ummmm, engaged all day so I dont know if he had time to make the changes. I havent gotten back the same for CRN 5 but a volunteer is working on it.
John V says he programmed opentemp to adjust geographic distribution each month to account for missing months, I think that was a good thing. I do not think it is necessary to eliminate a whole year of data just because one month is missing.
I understand that this is a small amount of data but given time we will be able to get enough data to fill each grid cell.
John V here is a link to satellite data:
http://vortex.nsstc.uah.edu/public/msu/t2lt/uahncdc.lt
There is data from a second satellite called RSS which usually runs about a tenth warmer.
Rural CRN12 Stations:
42572383001,34.7,-118.43 Fairmont CA (Not Surveyed, aerial photographs only but looks good) / No changes in location or equipment / records back to 1931
42572694004,44.63,-123.2 Corvallis State University OR / no change in location or equipment / records back to 1948
42572376006,35.27,-111.73 Fort Valley AZ / Equipment: 2004 current other temperature equipment / 1986 2004 MMTS (LIG backup) / before 1986 LIG / Location same throughout / records back to 1948
42572273004,31.7,-110.05 Tombstone AZ Equipment: MMTS 2002 current / LIG 1933 2002 / Location: same throughout / records back to 1933
42574501001,38.33,-120.67 Electra ph CA / Station closed 1997 / Equipment LIG throughout / Location: same throughout / records back to 1931
42572489002,39.97,-120.95 Quincy CA / Equipment: MMTS 1999 present / LIG before 1999 Location: last change 2004 / records back to 1931
42572786006,48.35,-116.83 Priest River Experimental Station ID / no location or equipment changes / records back to 1911
42572533004,41.63,-84.98 Angola IN / Location: Location move reflected in elevation change 1977 / Equipment 1985 Present: MMTS / LIG before 1985 / records back to 1887
42572743002,46.33,-86.92 Chatham experimental farm MI / Location: last move (short distance) probably not a problem / Equipment: LIG throughout / 1987: STN ESTABLISHED TO COMPARE WEATHER READING WITH CHATHAM EXP FARM 20-1484-2, 1.1 MI TO THE NNW / This station is a problem, the correct name is Chatham Exp Farm 2. The actual station is Chatham Exp Farm, which was active from 1948 to 1988 with no recorded station moves and LIG equipment throughout station history.
42574341005,45.13,-95.93 Milan 1 nw MN / Location same throughout / Equipment Nimbus 2006 to current / LIG before that / records back to 1948
42574341007,45.58,-95.88 Morris wc Experiment Station MN / Location: same throughout / Equip: LIG throughout / records back to 1948
42572235004,31.97,-91 Port Gibson 1 ne MS / Possible problem with either MMS or Surfacestations.org / MMS exposure & obstruction field do not match photos, MMS says: MMTS 260/10 F&P 90/7 TREE 100/75/29 140/100/25 HOUSE 160-220/90-120/14-7 TREES LINES 165-200/175-125/24-24 200-280/120-200/30-15, photos do not show these obstructions / Location changed 2001 / Equipment: MMTS 1988 to current / LIG before 1988 / records back to 1948
42572677004,45.92,-108.25 Huntley Experimental Station MT / Location and Equipment (LIG) unchanged / records back to 1951
42572488003,39.45,-118.78 Fallon Experimental Station NV / Location changed in 1992 / equipment the same (LIG) throughout
42572662003,43.97,-101.87 Cottonwood 2e SD / Location unchanged / Equipment: MMTS 1986 current / LIG before 1986 / records to 1948
42572572005,40.55,-111.5 Snake Creek PH UT / Location: no changes / Equipment: Thermograph 1997 to current / LIG 1997 and before / records to 1948
42572403002,38.9,-78.47 Woodstock 2 NE VA / Location change 1994 / Equipment MMTS 1993 to current / LIG before 1993
I just thought trend lines would be more consistent with some of Steve McIntyre’s previous articles.
http://www.climateaudit.org/?p=2069
http://www.climateaudit.org/?p=2107
http://www.climateaudit.org/?p=2092
If not trendlines, then maybe a direct comparison of temperature anomalies (vs differences in temperature anomalies), a smoothed plot, trailing 20yr trends, or trends for key periods would be useful. (Those are the plots I typically provide in my results).
Hans (#11, 14) and Steve M — there’s a very simple way to achieve equal-area map output.
X = longitude as usual
Y = sin(latitude)
This adjustment to Y creates an authalic (equal-area) latitude coordinate system.
Handy, accurately equal-area, and very fast.
(Map projections is one of my long-ago specialties…)
The last graph (USA Calculated from CRN 1-2 TOBS) illustrates what I have pointed out for many years and is evident in all plots of USA temperatures for this period. The variablity is high from 1850 to 1940, low from 1940 to 1980 (except for the anomalous 1950 spike), then high from 1980 to the present. I believe this pattern is most evident in the middle latitudes (30° to 65°) especially in the US with the most extensive record. It is likely a result of the general pattern of waves in the circumpolar vortex that dominate the weather pattern in these latitudes changing from zonal flow to meridional flow. It suggest zonal flow dominates when global temperatures are declining and meridional when they increase.
>> Wouldnt the provision of a trend line be consistent with the assumption that there is a linear trend. What is the justification for this assumption? As someone has pointed out, there appears to a be a step-wise discontinuity around 1920 with the additional appearance of a step down in the 70s and a current step up in 1998. If this appearance has any validity a linear trend would not be of much use.
Thank you Stan. That is a huge assumption. In fact, it is assuming AGW a-priori.
One candidate for what happened in and around 1920 is expansion and mechanization of agriculture, i.e. large scale land use changes. In 1920, US wholesale agricultural prices dropped nearly 50%, indicating a large surge in agricultural supply.
http://eh.net/encyclopedia/article/Smiley.1920s.final
#20. PEte and Hans, sure, but within the lower 48, the difference is negligible.
#20:
The OpenTemp grid cells are area-weighted for this reason.
>> One candidate for what happened in and around 1920 is expansion and mechanization of agriculture
Why do you think of man first, when he is dwarfed by his environment, including the sun.
RE: #21 – Tim, that is definitely something to look at. It may be a proxy for PDO phase or some higher order harmonic.
RE: #26 – Another thought that I had was that there is some sort of systemic error in TOBS adjustments. For example, there may be something badly wrong with how TOBS was dealt with for the time prior to 1920. There may be other systemic errors above and beyond this.
JohnV addresses some of these these issue in his inline comments of the code.
Take no offense at my implication that you have not read the code. hehe.
(#24 — I mostly tend to make this adjustment so things “look correct” to people :-D)
here are JohnVs comments in the code, relative to this issue. He is well aware of the issue.
And plans an improvement. RTFC!
// TODO: Improve the area calculation for a cell
// The current calculation works well for a small cell, but will break down for larger cells (> ~1000km on a side)
float sinLatitude = (float)Math.Sin(latitude * Math.PI / 180.0);
float degLength = (float)(2.0 * Math.PI * _earthRadius / 360.0); // Length of a degree at the equator
_area = (width * sinLatitude * degLength) * (height * degLength);
}
///
/// Get the latitude (in degrees) of the centre of this cell.
///
internal float Latitude
{
get { return _latitude; }
}
///
/// Get the longitude (in degrees) of the centre of this cell.
///
internal float Longitude
{
get { return _longitude; }
}
///
/// Get the area of this cell.
///
/// The area of the cell is calculated from the width and height. Currently the area calculation assumes
/// small values for width and height. That is, it assumes that the cell is square in cartesian coordinates.
/// The error for a 1000km cell is approximately 0.2%, increasing to 0.8% for a 2000km cell.
If one line in my code above is changed to:
annual[j]=weighted.mean(gridded$z,rep(cos(gridded$y*pi/180),40),na.rm=T)
this allows for latitude change in weights. You don’t need to worry about the radius of the earth since everything is proportional.
The following implements a weighted calculation as a function for an index list of USHCN stations:
This yields an annual series and plot in a couple of lines:
Or for rural stations:
Simple as pie.
Why do you think of man first, when he is dwarfed by his environment, including the sun.
I could equally well ask, why posit a global scale process to explain what could well be local effects.
MrPete on Post # 75 from the closed thread “2006 and CRN1-2” said:
I have seen these alluded to statements also, but never seen the links validating it. All I have found is a calibration process whereby the sensing devices from satellites are calibrated using the deep space temperature and a known calibrated temperature standard on the earths surface. I have not been able to find the surface temperature station data and satellite connection. Admittedly the satellite measurements have errors and measurements appear to be algorithm related.
I think this is an important point in these ongoing discussions that needs to be put to bed once and for all.
Steve McIntyre:
I just emailed you with my results for all CRN12 stations, rather than just the rural stations. I would appreciate it if you would update the “Vliet minus CRN1-2 (SM)” plot to use these results to make the comparison valid.
As I mentioned before, I would also appreciate my results being labelled as “John V”, “OpenTemp”, or at least “Van Vliet”.
Thanks.
RE 37. JohnV I think I’m gunna start to list.
1. The version of OpenTemp I Used.
2. The files and date I used.
Is there a simple metadata format we could adopt. *cvs is convient.
.
Ideally, ( in my perfect world ) clicking on a data point in a chart would magically find the
code, load the data, and rerun… Hmm. Neat idea.. and so every piece of data would come encoded with
its own DNA for replicating it… Ok way too too much coffee
One thing that stands out to me but I can’t see why it exists is the periodicity in the differences between the CRN1-2 and the other data. I would have thought there would be a random difference in any one year with maybe a superimposed trend (assuming UHI issues) but the sign of the differences appear in clusters. If I’m actually seeing this and not imagining a pattern (quite possible) then what would induce a sinusoidal pattern in one or the other data sets but not the other? Could it be that the pattern is due to the non uniform geographic coverage of the CRN1-2 with a climatic shift over the US on a decadal timeframe? Would it go away if you reduced the NOAA data set to those areas where there is equivalent coverage in CRN1-2?
#38 steven mosher:
I have not changed any of the OpenTemp processing code since the first release (on or around Sept 18). I have only been adding new features and fixing output bugs. To answer your question from earlier today, your old batch files will continue to work.
I’m going to add a few more features and then I’ll do a stable versioned release:
– file prefix for output (to reduce the number of folders required)
– per-station date filtering
– random selection of N stations from the list
The gridding and averaging code could use a little work, but changes there will change the output so they can wait for the next version.
Steve McIntyre:
I think there may be a problem with the reference period for the anomalies in your GISS comparison plots above. You did not state which reference period you were using, but from your code it looks like all dates after 1890 (I don’t know R so that’s just a guess). As I understand the anomaly method, the average of difference between two anomaly plots with the same reference period should be zero. The GISS comparison plots seem to be biased positive.
RE 40.. I should be able to make a “batcave” a master.bat with all my other *bats included.
I’ve sent you a bunch of station lists, perhaps we should compile a test script. So when you
have a new release you can check against old versions.
#32 Steve McIntyre:
Shouldn’t that be cosine?
annual[j]=weighted.mean(gridded$z,rep(cos(gridded$y*pi/180),40),na.rm=T)
Or maybe my brain’s just not working yet. It’s early here.
Steve: Yes. MY bad. In the US 48, the areal weighting really isn’t a big consideration as the rane is only from 0.87 to 0.65.
Where does one find the new USHCN V2 monthly (or daily) station data? V1 is available from CDIAC, either daily or monthly, but only through 2005. I’d like to see how the 2006 spike looks at my local stations, Circleville and Delaware OH. Delaware has been defunct since 1/30/01, but that hasn’t prevented it from providing phantom evidence for AGW up through 2005, according to CDIAC! NCDC says the new V2 was to be released in July of 2007, but I can’t find a link there to the actual data.
>> I have not been able to find the surface temperature station data and satellite connection.
You are correct. I understand that there is on board calibration with a known sample of oxygen, and readings are also calibrated to “known calibrated temperature standard on the earths surface”. Like all instrumentation, there are always error, and all instrumentation requires calibration, including thermometers, thermocouples, etc. We don’t even use thermometers for measuring kids body temperature for fever anymore. It’s archaic.
#43 Hu McCulloch:
USHCN v2 is not available yet. USHCN v1 is available here (to October 2006):
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn
Re: #44
The link and excerpt below is the closest I have come to any details of the calibration of the microwave instrumentation on satellites used to measurement tropospheric temperatures. Further reading on the subject leads me to conclude that the warm targets are onboard the space craft.
http://iridl.ldeo.columbia.edu/descriptions/.msuprecip.html
Steve M
Hu McCulloch (#43) is a top-notch econometrician, if you have questions about that. (Hi, Hu, long time!)
Hello from the UK.
This is my first attempt to comment although I have lurked with great interest for a couple of years or more.
I am a mature (in age – 60+) student with the Open University studying Earth Sciences. I am way out of my depth on scraping data etc. but there is a technique I have used in my previous employment for plotting spatial data of this sort i.e. Semi-Variograms and Krigging.
This has been used in the past in geostatistics, for example in Uranium prospecting and in presenting data for air pollution by SO2 in the UK. As I understand it, the semi-variogram takes all the spatial data on an irregularly spaced grid and determines how far away one has to be before the results from one station are no longer correlated with another. The Kiigging then uses this information to carry out the interpolation to produce a contour map of the results.
The advantage (if I understand it) is that the process is well accepted and uses standard computer packages.
best wishes
Alan
Earle Williams and Hans Erren, thanks for the links. I will probably stick with inverse-distance-weighting (IDW) in the near-term and move to one of the more robust methods for the next version of OpenTemp.
Alan Bates, thanks for the additional info. I was leaning towards Kriging based on my reading yesterday — it’s useful to get an independent recommendation for it.
Now thats better!
I can understand most of this, unlike the previous thread, which seemed to consist of “my Dick’s bigger than yours”.type statistics
Please keep up the good work, out here the world has gone enviro”mental”, and you are one of the few places keeping me sane.
Thanks Alan!
Regarding Steve’s comment
“So what does my calculation using CRN1-2 data look like: see below, which includes a 2007 estimate (using the 2006 difference between the CRN1-2 and NOAA to estimate 2007 CRN1-2 from the YTD NOAA results.) In this calculation, the warmest year remains 1934 by a margin of nearly 0.2 (enough to be statistically significant according to things that GISS has said, then 1921, then 1998, then 2006 in fourth place. Values since 1998 have been at uninterrupted high levels. Looking at the graph, its definitely noticeable – but is it the sort of change thats inconsistent with the sort of stochastic behavior in the early part of the series – hard to tell here. If you cut the series off in 1997, you certainly wouldnt have noticed anything odd about the 1990s. You can sort of see in retrospect why MBH98-99 were so excited about how warm 1998 and in such a rush to include it in their article. The main verification issues right now is why the ROW history is so different from the US – is it climatic or nonclimatic? And can one tell?”
It looks like from 2000 on, there is a 0.5 degree step function, notice none of the temperatures swings go below 0.5 on the graph. Is this another Y2K problem?
Leon Palmer,
Or it could have to do with all the stations being asked to change time of observation to 7 AM in the last year or two.
RE 53. ALL the stations? Even the ASOS. Interesting, can you site the reference
Steve McIntyre:
I am interested in looking at the scale of the differences between CRN12 results and GISTEMP/NOAA results. I would appreciate it if you would post the annual temperatures you calculated from CRN12 as well as a link to the NOAA monthly or annual tempertures. Thanks.
I am also having trouble reconciling a few of your plots. Maybe you can help:
The “NOAA minus CRN1-2 TOBS (SM)” plot above shows a small warming trend (~0.1C/century by eye).
The “GISS Post-Y2K minus CRN1-2 TOBS (SN)” shows a small cooling trend (~0.1C/century by eye).
Based on the this I would guesstimate NOAA minus GISS to be ~0.2C/century.
The “US48: NOAA – NASA” plot in the Second Look at USHCN article shows a trend of 0.66C/century.
Why the difference? Are my estimates for the trends above wrong?
Steve: good question. It doesn’t look like the graphics are consistent. I’ll try to reconcile them this weekend (which is Thanksgiving and I’ll be away for much of it.)
RE #21
Tim,
Wouldn’t you expect it to be the other way round?
Aren’t cooler temperatures associated with a wider, more southerly track for the jet stream and a tendency to block, creating meridional flow?
Don’t warmer temperatures have the reverse effect?
Also, the variability is just as high 1890-1920 as in 1920-1940 even though the first period is consistently cooler than the latter. If your argument was correct, shouldn’t the degree of variability be different?
I do not understand why everyone is analyzing such an obviously biased data set to death. Things may look quite different when the rest of the map is filled in. The part which is missing, the interior plains, is an area particularly susceptible to meridional flow and temperature and precipitation extremes under cooler temperature regimes and vice versa under warmer regimes. That is, it is where a great deal of the variability for US temperatures might be expected to be on decadal time scales.
I find it suspicious that this data set does not show the commonly accepted 1900 to 1940 warming trend, supposedly stronger than the reputed one since 1975. It may be that this trend does not exist, but until Anthony has a more complete data set, it would be unsafe to assume that the sample we have to hand proves that. It could just as easily point to the unacceptable bias in the current sample.
Based on the meridional versus zonal flow argument as I have laid it out, as applied to this data set, one could argue that, contrary to the accepted opinion, that 1900 to 1940 and after 1975 were cool periods. Moreover, the severe African and Great Plains droughts of those periods are consistent with persistent meridional flow and jet streams closer to the equator. These droughts are considerable counter facts to the global warming hypothesis.
I think we have a long way to go before we really know what has been going on with surface temperatures.
Mike
#17: The whole point of CRN1,2 stations is that they are of high quality and are not in need of a UHI adjustment; population moves shouldn’t affect the data at all.
Carl Gullans:
The CRN1 and CRN2 stations are relatively free of microsite issues, but the CRN rating says nothing about UHI.
Steve McIntyre:
I realize it’s a long weekend, but I would appreciate it if you could find a few minutes to update the comparison plot above (my results vs your results). I would like to do some new work with OpenTemp and would prefer to wait until a comparison with your results is available.
If it’s easier, you could also just send me your yearly CRN12 averages and update the graph later.
Thanks.
RE 58. UHI
causes of UHI: from Johnsen
1. Street Canyons and screening the skyline.
2. Increased heat storage in buildings.
3. Anthro heat: animal metabolism.
4. Urban Greenhouse from urban pollution.
5. Canyon radiation geometry- Basically multipath IR
6. Evapotranspiration impairment. ( artificial ground cover)
7. Wind Shelter: Reduced turbulant transfer.
When you look at the microsite issues you see the same causes ( save urban pollution)
Very simply. You can have a site in an Urban place (pop 1M) that doesnt have a lot of wind shelter (#7)
like an airport. AND you can have a site in the Wilderness ( pop ZERO) that does have wind shelter.
NOAA and GISS use population as a proxy for the details of the site. Since we dont have a parametric
model of microclimate, as Karl notes, he uses population to stand in for the 7 causes.
NAmely, if you have a high population chance are the site will be impaired by these factors.
If you have a LOW population chances are you won’t. We know this to be untrue. The point of visiting
sites was to dtermine the following
1. Street Canyons and screening the skyline. Is the skyline screened at the site?
are there trees that screen it, is the ground sloped?, buildings, etc etc. MICROSITE
can happen in the rural arena.
2. Increased heat storage in buildings. Are their building nearby this rural sites.
A waste water treatment plant. A ranger station with 22 air conditioners. etc etc
3. Anthro heat: Not such a big problem in rural sites, unless you consider sites located
at power plants, damns, waste water treatment.
4. Urban Greenhouse from urban pollution. Aerosols cool.
5. Canyon radiation geometry. happens in both urban and rural settings.
6. Evapotranspiration: non natural ground cover. Happens in both urban and rural.
7. Shelter: Reduced turbulant tansfer. Trees, buildings, all reduce turbulant transfer.
Lets put this another way. MICROSITE bias is AHI. Anthropomorphic Heat Island. And AHI
could be more prevalent in urban settings.
That’s testable,perhaps, Compare CRN12345 ratings versus population.
Population might amplifiy certain aspects of it.
#60 steven mosher:
I have read (on this site) about UHI being caused by thermal inversion over large urban centres.
Is that also a recognized cause of UHI?
Thermal Inversion is probably more readily construed as an effect. Simply Windshelter
prevents turbulent transfer, But it is also referred to cause.
The causes I listed are in Johnsen. I can get you a proper cite.
For now, here is a nice little description
Click to access George_v51n2.pdf
Could the thermal inversion over a city could be present and affect station readings even if the station has no immediately local influences? For example, a station in a small neighbourhood park could be 100m from trees, buildings, etc (and have a CRN1 rating) but still be under the influence of the larger UHI effects of the city (such as thermal inversion).
I really should read the references, but you’re making it too easy by just answering me directly.
The need for external temperature data for satellite atmospheric temperature profile reconstructions is a separate issue from the calibration of absolute sensitivity of the microwave sounding units. There are more variables than there are measurements, i.e. the problem is ill-posed. That means there are an infinite number of temperature profiles that will match the measurements. So you have to start the calculation with a “best guess” and hope the process will converge to the correct profile. Represent all possible temperature profiles as a three dimensional surface with the z axis representing goodness of fit. A well-posed problem will have only one peak so you could start anywhere on the surface and find the correct solution. An ill-posed problem has multiple peaks, so different starting points lead to different solutions. A good best guess would be a radiosonde sounding. See this post for more detail.
Re: #64
DeWitt Payne, I may have made an incorrect interpretation of what your comment reference here actually said, but here is my view of it (from Post #303 in the thread Unthreaded #21):
Firstly, it states that IR (and not microwave sounding units [MSU]) instrumentation needs to be calibrated in various ways, one of which is with reliable surface station measurements.
Secondly, it references Grant Pettys comments on the problems of finding correct tropospheric (and stratospheric) temperature profiles due to the vertical overlap that the MSU instruments senses.
In effect the subject changed from IR calibration to the measuring problems from vertical overlap that MSU instruments have, without addressing the issue of MSU calibration and the use of surface temperature measurements to accomplish this.
RE 63.
I have a copy of Oke I can send you.
you asked:
“Could the thermal inversion over a city could be present and affect station readings even if the station has no immediately local influences? For example, a station in a small neighbourhood park could be 100m from trees, buildings, etc (and have a CRN1 rating) but still be under the influence of the larger UHI effects of the city (such as thermal inversion).”
I really should read the references, but youre making it too easy by just answering me directly.
That’s ok. I find this much more enjoyable than the other crap. I really must stay away from those threads.
To answer your question It would be possible to have a CRN1 site infected by UHI, primarily through
the urban canopy effect. It would be modulated by wind speed and wind direction and surface
roughness. So, you might have 100feet of pristine globe, but if your have a 1000 ft high structure
upwind, the boundary layer wil change. There are lots of cool studies on this including
a neat 3D simulation done of a city in the bubble study. One issue is this. On one side
of the climate argument you have people arguing for green cities. They Cite the Horrible UHI.
Then, on the other hand they claim that the stations in urban areas ar not biased.
This is a fascinating question. and there are only two decent studies of the problem.
Peterson ( which has flaws ) and Parker ( which has flaws ) I look at the claims being made
A. UHI is horrible. Our cities are baking
B. UHI doesnt bias the temp record.
And I think. Dang. That needs explaining. I don’t think the result impinges on AGW theory in
a significant way ( hey the ice is melting) but the current explanation of A&B is
unsatisfying in my estimation
There is also the phenomena of “cool parks” In UHI studies studies they find areas that are cooler
than rest. Although the distribution of temp differenetial is skewed.
Stupid example UHI_bias = ACTUAL +5C, -2.6C Skewed figures like that.
There seems to be this belief that UHI affects only a few tens or hundreds of feet vertically and horizontally from urbanization structures and artifacts. I question this. Certainly, while this may be true of IR reradiation from surfaces which are incident to insolation, it is not true for direct thermal flux due to electrical dissipation, climate control, and other work performed by humans and their machines.
RE 67. This disconnect is fascinating.
I’ll illustrate the insanity by grabbing the other Horn.
UHI is not real. Why are you envirowackos talking about green roofs, and urban parks,
and measures to control the UHI that doesnt exist?
RE: #68 – Yup. Also, consider this. IR radiometric images show “cool” parks, lawns, etc. But that tells me nothing about the air temperature. Of course the emissivity and H2O content of lawns, shrubs, pools, well watered dirt, etc in the IR band is much lower than concrete. BFD. All the emitted IR from the stuff that does emit highly, plus all the other thermal flux, warms the general air parcel over a city (or even over a place like Happy Camp) and if you measure that air’s temperature, it’s going to be higher than what it would be in the true bona fide boonies. This is not rocket science and frankly, I am tiring of the fact that it is debated at all. Talk about denialism …. those who seek to discredit UHI and other anthropogenically delivered or influenced direct thermal flux effects are the real denialists.
Re: #65
Ken,
I’ll to be clearer this time.
The original comment I replied to, #296 in UT #21, referred to the calibration of IR sensors on the satellites. While there are IR cameras and spectrometers on satellites, the atmospheric temperature (LS, MT and LT) from UAH and RSS are determined by analyzing the data from multi-channel microwave sounding units. The absolute sensitivity of those units is checked by a two point calibration, empty space for the low point and a controlled temperature warm panel on the satellite for the high point. So far no need for a ground based calibration.
The question then is how do you convert observed microwave intensity to temperature? That is not a trivial problem. There is no unique solution. You have to start out with a guess and the guess has to be close. Where does the guess come from? Independent temperature measurements, i.e. radiosonde soundings.
Overlap increases the difficulty of the solution by effectively reducing the number of independent observations, it is not the source of the problem. The problem would still be ill-posed even if there were no overlap. In a sense it’s like climate modeling. The surface temperature record is not an input to the models, but the models must be tuned to reproduce the surface record (even if the modelers won’t admit it in public). It’s the same with the temperature profiles derived from the MSU data except the data being tuned to are radiosonde soundings.
You can’t just choose any profile. There must be some reference to independent measurements.
Is that better?
That’s try to be clearer… An edit function sure would be nice.
Re: #70
I believe it is. If I may summarize your point, you are saying that since the MSU calibration/profile measurements have many choices for the “retrival” process that the surface temperatures are taken into account, however indirectly that may occur, in selecting a “realistic” profile. I follow your thinking on this issue, but can you link me to a specific paper or other scientific publication that makes this case?
Re: #72
Unfortunately, Petty doesn’t cite much. The entire bibliography consist of 8 books. Since he and Gerald Browning seem to agree, I assumed they were correct. I think you will have to go to the original literature. Here’s an abstract of a recent paper published by Christy, Norris, Spencer and Hnilo in J.Geophys.Res. Here’s the paper cited at the beginning (bottom) of the readme file below. It contains a discussion on the calibration of the sounding unit in the lab before launch. From what I could tell, it doesn’t seem to agree with Petty and Browning. I could easily be missing a lot, though. They do spend a lot of effort comparing their data to the radiosonde data.
Or you could just ask Christy. His email address is at the bottom of the readme file.
Steve McIntyre:
I realize you’re busy but I hope you can find some time to clean up some loose ends that I have mentioned above. I will repeat them here for convenience:
1. Update the “Vliet minus CRN1-2” (see post #36 above);
2. Post or email your yearly results used for the plots above (monthly results would be nice too if you have them);
3. Confirm the reference period used for your difference plots (see post #40 above)
4. Reconcile the “NOAA minus CRN1-2 TOBS (SM)” and “GISS Post-Y2K minus CRN1-2 TOBS (SN)” plots from above with the “US48: NOAA – NASA” plot in the Second Look at USHCN (see post #55 above);
5. Consider adding trendlines (see post #19 above)
Thanks.
Re: #73
DeWitt, I have done a rather extensive online search for information along these lines and have many of these same references. What I wanted was a reference(s) that uses something similar to your scenario in linking (or not) surface temperatures to the MSU satellite calibration. An email to Christy may be the best choice at this time.
Anyway, I appreciate your thoughts on this subject as I tend also to look at these indirect connections in climate science as more important than they often appear on the surface of things. I do think that an important first phase of our discussions here of the surface station temperature trends and data should be whether we can independently compare it to other measurement methods — with independence being the key consideration.
Re:#75
Ken, This may be apples and oranges. The primary purpose of the satellite observations is weather forecasting. They had been up ten years before Christy and Spencer first published. So the question is, what algorithm is used to determine temperatures on a daily basis in more or less real time? That may be what Petty and Browning are talking about. I haven’t tried looking for that yet.
Re: #76
That the satellite temperature measurements were for short term forecasting is often stated in papers attempting to make adjustments for using the data for calculating long term temperature trends — the same problem that exists with the surface temperature measurements.
Re:#77
I think it is safe to say that the long term satellite anomaly measurements are not completely independent of the radiosonde measurements. There seems to have been considerable tweaking of the satellite data extraction method to get them to agree. The question then is: are the radiosonde measurements independent of surface temperature measurements? I don’t know. If radiosondes are calibrated to the local temperature before launch, then they aren’t completely independent. If they are just cross-checked without adjustment to see if they are working correctly, then they still might be considered independent because the trace back to NIST could be a separate path. Of course, the next question is: Are the radiosonde measurements traceable to NIST? I don’t know that either.
Re: #78
The recent articles that I have read on the radiosonde and MSU satellite measurements use the MSU measurements to correct radiosonde data sets due to the radiosondes being considered less stable and error prone than MSU. I could not find a Christy article that did a reanalysis of the radiosonde data based on breaks in the time series that corresponded to instrument and procedural changes in the radiosonde measurements and I thus link to the article below.
http://www.gfdl.noaa.gov/aboutus/milestones/JRL_FY01_Q2_Milestone.html
I think the meant for use in short term weather predictions and the limitations this places on use for longer term temperature trends is even more of problem for radiosondes than it is for MSU and surface temperature measurements.
Re: #79
Nice work by the NOAA guys. I’m assuming that’s the paper describing the RATPAC radiosonde database. If only Jones and/or Hansen were that open.
I don’t see anywhere in the Christy et.al. 2000 paper linked above anything that would imply the use of a ‘best guess’ approach to calculating atmospheric temperatures from the microwave intensities. However, given the huge overlap in the T2 and T2LT channel weighting functions, and the significant overlap of T4 and T2, something like that had to be done at least once. Maybe it was done in the original laboratory calibration. You don’t by any chance have a copy of or have access to Mo, T., 1995: A study of the Microwave Sounding Unit on the NOAA-12 satellite. IEEE Trans. Geosci. Remote Sens., 33, 11411152. $35 is more than I want to spend when I’m not sure the article is what I’m looking for.
OTOH, maybe Grody et.al. is the better choice. It’s only $9 and is the most up-to-date. If you already have it, I don’t see the need to buy it myself.
Re: #75
I found it (I think)! Better yet, it’s free. Conrath, B.J., Vertical Resolution of Temperature Profiles Obtained From Remote Radiation Measurements, J.Atmos.Sci., 1972.
He then proceeds to finesse this. I’m not sure I can (or want to) follow the math all the way.
Steve McIntyre:
I assume you have just been busy, but I have to ask you again to clean up some of the loose ends regarding this article. See my post #74 above.
He’s been very busy with the Juckes topic. I can only hope I’ve lessened the workload and not added to it.
I realize he’s busy with Juckes. That’s not an area that interests me so I’ve been trying to wait patiently.
SteveMc, the loose ends (ie errors) in this article really should be fixed.
Give it a rest, John. It’s been barely a week, and Steve was on vacation for some of that time.
The only “error” that you’ve referred to is the spelling of your name. The other requests you’ve made are suggestions on how the analysis should be performed, which Steve may or may not think are appropriate.
Mike B:
Thanks for your input. The spelling of my name is not an error. The errors are:
1. The final difference plot compares SteveMc’s results with *all* CRN12 stations versus mine with *only rural* CRN12 stations;
2. The reference periods do not appear to be consistent since the average of the differences is not zero for the apparent reference period of 1890-2006;
3. The NOAA minus CRN1-2 TOBS (SM) and GISS Post-Y2K minus CRN1-2 TOBS (SN) plots from above do not reconcile with the US48: NOAA – NASA plot in the Second Look at USHCN thread;
I also made a simple request for his data last Friday. It should only take a minute to send the data file.
Steve McIntyre — any chance of getting that data file soon?
Steve
In your Google targeted ad thingy on the lower right of your homepage there is this offer for a 100 year archive of weather data!
http://weather-warehouse.com/?gclid=CN2wl5apiI8CFQacggodaGXOvA
Just thought it was funny.
He’s apparently too busy demanding data, corrections, and explanations to provide data, corrections, or explanations.
cce, I’m actually quite happy that Steve M. runs on his own schedule, and not yours. Instead of doing what you want him to do, he’s been updating the Graybill bristlecone series, and simultaneously verifying the Starbucks Hypothesis. You don’t like the speed with which he’s fulfilling your requests? Live with it. John V. has made it clear what he thinks the errata are, and they will get addressed when Steve gets around to it. Until then, I offer you three options:
1) Lead
2) Follow
3) Get out of the way
Note that “Get snarky and whine because the world doesn’t run on cce’s schedule” is not on the list of options. If I were Steve, all your comment would do is make me go off and work on something else for a while … but I’m a reformed cowboy, and Steve is a gentleman, so I suspect your comment won’t affect him either way.
w.
I like option 4:
1) Don’t care one way or the other since nothing will come of this anyway.
I just like pointing out irony. I wish him luck finding Waldo. Be sure to check Starbucks.
Steve McIntyre:
Thanks for updating the “NOAA minus NASA” plot in A Second Look at USHCN Classification.
I have to ask you again to update your “Vliet minus CRN12 TOBS (SM)” plot above. I sent you the data on October 5th. I have two reasons to ask for the change:
1. It is not a valid comparison (rural CRN12 versus all CRN12) of my results to yours and therefore leaves an incorrect impression about the match;
2. Clayton B looked at the effect of OpenTemp’s averaging radius (here) — I would like to see how my OpenTemp results with a 1000km radius compare to your results using Akima for gridding
If you can’t update the “Vliet” graph above, please take it down and send me your data so I can do the comparison myself. Thanks.
Steve: Fair enough. Will do some time soon.
I download the excellent “GetData Graph Digitizer” today:
http://getdata-graph-digitizer.com/index.php
I tried it out on Steve McIntyre’s “USA Calculated from CRN1-2 TOBS” to compare against my OpenTemp CRN12 TOBS results. The digitization was not perfect and any digitization errors are my own. Here’s a graphical comparison:
I overlaid a five-year trailing average and a linear trend on the complete data set. Unforunately, I could not easily do a year-by-year difference because my digitized years did not always line up with the actual years. (Misalignment is visible in the graph above).
Overall I’d say the correlation is quite good. It doesn’t raise any concerns about the OpenTemp algorithm in my mind.
#93. John, a digital version (which looks similar to this graphic and should be the same) is at CA/data/station/watts/recon.crn12.dat
The point in this post is that there is perfectly good smoothing software readily available in standard packages. There’s nothing particular magical about any of these smooths, which simply produce varying weights of the underlying proxies, and, as shown in the script above, the calculations can be done in a few lines. I don’t understand why you wouldn’t just use this sort of standard software where it is available.
As I’ve said endlessly, my concern is with the ROW data and methods where one does not see the same relationship between the 1930s and 2000s. IT seems to me that your time would be spent more fruitfully trying to come to grips with some of the ROW issues.
Steve McIntyre:
Thanks.
Absolutely. The only problem is the overhead and technical skill required to install and learn these packages. My goal is to make a very simple program that anyone can use.
I do want to get to the ROW data, but not prematurely. There is a wealth of information in the USA48 data and much knowledge can be gained from that. The site quality ratings permit a number of new analyses with one variable (micro-site issues) at least partially constrained. It is possible to differentiate and quantify micro-site and UHI issues. Using stations that are known to be of high quality, the regional variation can be estimated for areas with few stations. Tmin and Tmax trends can be investigated. Etc.
I should add that I also wanted some independent verification of OpenTemp before going much further with it. A comparison against your results will help.
Installation of the akima package took about 3 seconds and one command. Yes, there is a little overhead involved in learning R, but it is a very common statistical language that is designed for use by anyone for multiple purposes.
I’d like to support Steve M’s call for the use of the R computer language. It is fairly fast to learn, immensely powerful for dealing with large datasets, and specifically designed for statistics (it’s built on the core of “S”, a language made for statistics from the bottom up). I resisted learning it for a long time, but I’m glad I finally put forth the effort.
Perhaps the best part of it is that you rarely need to re-invent any wheels. There are standard, tested functions for virtually any statistical manipulation you want to undertake. It gives very good control over the graphical representation of data. I’m not very good with it yet, but it’s a jewel. I still use Excel for lots of quick-and-dirty stuff, but when it gets big and complex, I turn to R.
Finally, if you want to develop something like OpenTemp, you can write it up and test it as an R function, and its principles, strengths, and weaknesses will be totally transparent.
Just my $0.02 worth …
w.
Could someone summarize how the problem of geographical distribution of CRN12 sites was addressed? Thanks
#99 David Smith:
The problem with a small number of stations (and thus poor geographic coverage) was primarily an issue with *rural* CRN12 stations. There are quite a few total CRN12 stations.
I did some work with rural CRN12 vs rural CRN123 here:
http://www.climateaudit.org/?p=2124#comment-147568
The coverage is much better when the CRN3 stations are included.
FWIW, a discussion of grid box sizes, and shapes, may be found here.
re 93:
I’ve always wondered what the trend line in such diagrams is supposed to mean. What phsyical interpretation should it be given? My own prejudice is that a trend line is useless. It presupposes a linear trend without justification.
If you go to the online script for this, I simplified the above calculations into a function that calculates the geog weighted average (using the akima smooth) from a set of USHCN identifications index_good and a USHCN collated version. Doesn’t get any simpler. You can adapt the script to get the contoured maps for individual years if you want.
Please explain how this is different than other code?
re: 94
Like JohnV I was interested in the Hansen reverse-engineering project and thought that this area (software development) might be where I could contribute. In this vein I have the following questions:
1) Is there work than can be done by the non-statistician to enable the statistician to focus more on the design and interpretation of the analysis. If there is this type of work, and there is a will to support it, then software “friendlier” than R may have advantages.
2) Are there tasks that occupy the time of principal researchers where purpose built, or integrated software can play a significant role? For example I have seen many requests for data cut this way and that, data sets to be correlated, etc.
3) JohnV has certainly demonstrated that there is sufficient good will and expertise that can be harnessed towards these ends.
c
#105. Analysis of ROW station data has been barely scratched.
2 Trackbacks
[…] McIntyre, Steve (October 4, 2007). “Gridding from CRN1-2″. Climate Audit. http://www.climateaudit.org/?p=2145. Retrieved […]
[…] I had done my own initial assessment of this a few years ago, I had used TOBS versions and am annoyed with myself for missing this. […]