Quality Control – Jones and Hansen Style

Anthony Watts has an excellent post showing the calibre of the quality control carried out by Phil Jones and Jim Hansen and the quality of Phil Jones’ “proof” that the “overall urban bias …is greater than 0.05 deg in the 20th century”. Marysville CA (425745000030), GISS population 12,000, is in the USHCN network and is used in the GISS, CRU and NOAA calculations of global temperature. Here’s one of Anthony’s pictures; go to his site for more sickening pictures.


Marysville CA weather station. Cars with heated radiators park inches from the thermometer (among other things)

Here is a plot of the temperature history at Marysville as downloaded by Anthony from GISS

Anthony contrasted this with results from nearby Orland CA (425725910040), GISS rural area, where the temperature history is shown below:

One small tweak on Anthony’s interpretation. His GISS graphic shows the “raw” not the “adjusted” GISS version. But as far as I can tell, the NOAA calculation uses the “raw” version according to their the readme which says:

This data set contains gridded temperature anomalies calculated by the “anomaly method”. Gridpoint temperatures are calculated by averaging the unweighted raw data from all stations within the grid box, and then the anomaly is the difference from 1961-90 mean.

So this is worth bearing in mind when NOAA tells you that March 2007 was the warmest in a milllllll-yun years or was it a billlll-yun years. The “adjusted” Marysville data from GISS is shown below:

marysv4.gif

The UHI warming here isn’t as bad, but it’s still arbitrary and biased relative to Orland. (And they reduced “warm” 19th century Orland temperatures no doubt for a “good” reason.) They also reduced Orland temperatures in the 1930s; the adjusted Orland temperatures are warmer now than in the 1930s; the opposite is the case with unadjusted. I wonder how they adjusted the Marysville temperatures. But what to do with the adjustments to the Dawson, Yukon data which Rob Wilson rejected. How does one go about sometimes using their adjustments and dometimes not.

But let’s say that they got all their adjustments exactly right. What does that say about the quantum of UHI? Here’s a town of 12,000 which qualifies as “rural” in all the UHI studies. The non-climatic effect here is at least 3 deg C. In this case, we can estimate it because there’s a less bad station nearby. What does one do in China or Indonesia where nearly all the stations seem to be in large cities (even if they started out small, they’re large now)?

[UPDATE: Eli Rabett has claimed that I “blew” it when I said that this “town of 12,000 qualifies as ‘rural’ in all the UHI studies”, because it is not used as a GISS unlit site. The phrase “UHI studies” is not especially clear, but is intended to cover studies like Jones et al 1990, Karl et al 1998 purporting to show that there is negligible UHI effect in global composites; the GISS composite is not a “UHI study”. Jones et al 1990 used Chinese cities with populations under 100,000 as “rural” comparanda. Karl et al 1988 said:

Due to the large number of stations located in sparsely populated areas [over 85% (70% of all stations had a 1980 population of less than 25 000 (10 000)], the impact of urbanization is not large in relation to decadal changes of temperature in the United States.

Marysville falls squarely into Karl’s definition of a “sparsely populated” area. The salient point is that the GISS adjsutment estimated a substantial UHI effect at Marysville requiring adjustment; if this adjustment is justified, then GHCN and CRU (and IPCC relying on CRU) should have used it. Eli can’t have it both ways. [End UPDATE Jul 30, 2007]

And what can one say about quality control as practiced by Phil Jones and Jim Hansen if they let this sort of stuff into their network? Maybe they should spend a little less time going to conferences and a little more time doing quality control on the temperature data that they’re being paid to report on.

Here are plots of the GISS adjustments for Marysville and Orland. I haven’t tried to wade through GISS adjustments yet. At a first impression, the adjustments look to implemented in 5-year steps. This is obviously something different than Time-of-Observation adjustments or step adjustments. Any thoughts on what they’re doing would be welcome.

marysv7.gif

Update 2:
Here is the difference between the GHCN raw data and the GISS raw data:
marysv14.gif

Here is the difference between the GHCN raw monthly data and the USHCN raw (“Areal”) data.

marysv16.gif

In each case, the adjustments have the effect of increasing the trend.marysv16.gif

A GISS Rural Cooling Adjustment?
Here’s another odd plot, which I’ll have to triplecheck. The first diagram below shows 3 calculation stages: GHCN Raw- GISS Raw (which should deal with all the time of observation and station history stuff); GISS Raw – GISS Adjusted (the GISS urban heat adjustment); the combined GHCN Raw – GISS Adjusted. I’ve done this first for Marysville GISS population 12,000, periurban lighting; and Orland GISS rural; unlit. We’ve seen a strong UHI effect at Marysville and GISS makes a substantial adjustment in the early portion of this series – up to 2 deg C, notwithstanding Jones view that UHI does not exceed 0.05 deg per century.

marysv29.gif
Marysville CA Adjustment Stages

Now the same things with Orland CA. This time the GISS adjustment goes the other way – I presume that the rationale for this adjustment is the well-known “rural cooling” effect, but I’ll ocntinue investigating.

marysv30.gif

113 Comments

  1. Richard deSousa
    Posted May 26, 2007 at 2:04 PM | Permalink

    How in hell does Jones figure that only 0.05 deg per century adjustment is accurate? I’ve stood in the middle of an asphalt covered parking lot in summer and it’s damn hot. If I move over to a field with natural ground cover the temperature is a hell of a lot less.

    From that very revealing photo they might as well have placed the Marysville weather monitoring station in the middle of the asphalt. LOL

  2. JerryB
    Posted May 26, 2007 at 2:26 PM | Permalink

    The MMTS sensor at Marysville was installed in 1984 after
    which even the GISS homogeneity adjusted version jumps.

    The graphic that Anthony showed did include USHCN adjustments,
    but not GISS adjustments, other than estimates for missing data.

  3. Richard deSousa
    Posted May 26, 2007 at 2:34 PM | Permalink

    On second thought I have a better idea. Separate the urban temperatures from the non-urban temps and show both results. It should be readily apparent that UHI is definitely a factor in global warming. It doesn’t make sense to “average” the temperatures because the people in the cities don’t live in “average” temperature environment.

  4. bender
    Posted May 26, 2007 at 2:54 PM | Permalink

    Re #1 I can’t LOL. I feel a bit sick, actually.

  5. Reid
    Posted May 26, 2007 at 3:02 PM | Permalink

    Bender says “I feel a bit sick”

    Sounds like you are suffering fron heat exhaustion.

  6. trevor
    Posted May 26, 2007 at 3:21 PM | Permalink

    Re #3:

    On second thought I have a better idea. Separate the urban temperatures from the non-urban temps and show both results. It should be readily apparent that UHI is definitely a factor in global warming. It doesn’t make sense to “average” the temperatures because the people in the cities don’t live in “average” temperature environment.

    Can I suggest that just as you have rendered the word average as “average”, so we should render the words global warming as “global warming”.

  7. Steve McIntyre
    Posted May 26, 2007 at 3:36 PM | Permalink

    I’ve added a plot showing the amount of the GISS adjustments for each site. The adjustments are in different directions. Any thoughts on what GISS is doing would be welcome.

  8. Barclay E. MacDonald
    Posted May 26, 2007 at 3:51 PM | Permalink

    How do we find out where our local temp gauges are? How about we all do digital photos of our local stations and post them at some common site on a common thread? There’s a lot of us now. Perhaps we could start something useful. In the beginning of course it would be completely annecdotal, but if we collected appropriate info for each site, perhaps at some point it would become more than just annecdotal. But maybe we should first send Bender some Dramamine or something:)

  9. Steve McIntyre
    Posted May 26, 2007 at 3:56 PM | Permalink

    The most useful would actually be for foreign readers to take pictures of the weather stations at the airports that constitute the post-1991 temperature record assuming that the local police allow it.

  10. Chris Wright
    Posted May 26, 2007 at 4:00 PM | Permalink

    I wrote a letter to New Scientist about their recent article. I pointed out that their graph seemed exaggerated (it showed a recent rise of about 0.7 degrees). Of course they didn’t print the letter, but to their credit they are willing to enter into some discussion. The editor informed me that the graph was based on the GISS data, which does indeed show a very steep rise in recent years. By choosing GISS and northern hemisphere data New Scientist had chosen the most extreme combination which almost certainly exaggerates the true state of global warming.

    I was already familiar with the Hadley graphs. Clearly there is a big difference between Hadley and GISS. Hadley shows that global warming has effectively come to a halt, even if temporarily. The question obviously arises: which data set is more reliable and why? The example above is appalling. Is this typical of GISS or is it an exception? Overall, how reliable are Hadley and GISS?

    Regards, Chris

  11. Anthony Watts
    Posted May 26, 2007 at 4:03 PM | Permalink

    Thanks for the comments, you may also enjoy this post I made on a spot test of latex paint compared to whitewash as was originally specified for Stevenson Screens, but appears to have been discontinued around 1979.

    I have a running experiment right now in addition to this spot test with a 4 channel datalogger with NIST calibrated temp probes measuring the temperature of boreholes in the painted, whitewashed, and bare wood (for control) I can tell you that preliminary results show whitewash consistently 2-3 degrees cooler through the day once solar heating commences. I’m also logging aspirated air temperature within 2 feet of the experiment and I’m making wind and solar insolation logs also.

    I’m running this wood borehole experiment as a precursor to getting three new Stevenson Screens delivered. They had to be special ordered, without paint, so it takes time.

    I will make all the logger data available publicly and will provide experiment details so that anyone whom wishes to reproduce it may do so.

    And I have an even bigger project that I will announce soon. As they say in television, stay tuned.

    In the meantime, it appears that measuring temperatures in parking lots here in California is becoming a trend. Here is a picture of the Yolo County Cooperative Extension Service Building which also houses the County office of weights and measures. This is a Cooperative Weather Observer station with an MMTS.

  12. Barclay E. MacDonald
    Posted May 26, 2007 at 4:05 PM | Permalink

    Although at the USHCN Version one site here, http://www.ncdc.noaa.gov/oa/climate/research/ushcn/ushcn.html, it does state that adjustments are made and that “The cumulative effect of all adjustments is approximately a one-half degree Fahrenheit warming in the annual time series over a 50-year period from the 1940’s until the last decade of the century.”

  13. Demesure
    Posted May 26, 2007 at 4:12 PM | Permalink

    Hasn’t anybody noticed the automatic adjustment provided by the National Weather Service for the asphalt’s heat at Marysland ? Just look at the green plants impeccably trimmed (& copiously watered ?) just under the station.

  14. Steve McIntyre
    Posted May 26, 2007 at 4:22 PM | Permalink

    Anthony, such fun. A small point, I think that it would be helpful to include the ID numbers of the stations in your posts.

  15. Paul Linsay
    Posted May 26, 2007 at 4:26 PM | Permalink

    There’s also the issue of what the true errors are for these measurements. It’s amazing how there’s never an error bar in sight in climate data.

  16. Steve McIntyre
    Posted May 26, 2007 at 4:34 PM | Permalink

    #15. Jones has purported to put error bars on his reconstruction – see Brohan et al for the most recent effort. Whether the error bars mean anything is a different question. They don’t analyze errors like we’re discussing here.

  17. Anthony Watts
    Posted May 26, 2007 at 4:40 PM | Permalink

    re:14, will do, FYI the station photo above is Woodland, CA, which is not in GISS, but does report to NCDC

  18. Anthony Watts
    Posted May 26, 2007 at 4:54 PM | Permalink

    Steve, see my private email to you

  19. samoore
    Posted May 26, 2007 at 5:09 PM | Permalink

    RE #8:

    Instead of digital photos of the stations, I’ve found it interesting to use the station coordinates and find them on GoogleEarth.
    Gives a good idea of the surroundings, albeit with the occasional jaw-dropping moment.

  20. Posted May 26, 2007 at 5:26 PM | Permalink

    Steve,
    Are you sure CRU use Marysville ?
    I can not see it in my Jones 1994 station list.
    I am not aware of any published list after that.
    Just wondering if you have a list or map that specifies CRU use Marysville.

    I am impressed with Anthony Watts work documenting these poor quality sites and have put up a post pointing to some pages on Californian trends.

    Replication #10: NH Temperature from Archived RPCs


    I have also made a series of graphics from Climate Explorer for the LA and San Francisco grid cells showing the often disparate trends for the various global databases.
    Off topic, the “Errors in IPCC climate science” Blog has been upgraded after we realised the effect of the tidal wave of spam. Commenting should be easier now.

  21. Barclay E. MacDonald
    Posted May 26, 2007 at 5:30 PM | Permalink

    #19

    Good idea! Along with the digital photo, the station ID no., the station coordinates, and a specific focus on foreign stations that are at airports post 1991. When Hans wakes up, we can send him to the airport with a camera:)

  22. Robert van Haaren
    Posted May 26, 2007 at 5:34 PM | Permalink

    Wen the NWS replaced the MaxMin alcohol/mercury thermometers with the digital thermometers the engineers referred to the MMTS sensors as the Mickey Mouse Temperature System. So there! Now you know.

    Bob van Haaren

  23. e.o.
    Posted May 26, 2007 at 7:08 PM | Permalink

    Roger Peilke Sr. has a number of papers and review on the poor location of the temperature stations at his website

    http://climatesci.colorado.edu/2007/05/16/request-for-photographs-of-ghcn-sites-a-need-for-documentation/

  24. steven mosher
    Posted May 26, 2007 at 7:35 PM | Permalink

    Well Anthony I’ll post a comment over on your site, when I get a chance. Orland is
    definately more rural than Marysville, but neither are trurely Rural and both areas
    have undergone significant land use change ( agri culture and development)

    Now, I found a source that you might use to double check or compare to the USHCN data.

    http://www.ipm.ucdavis.edu/WEATHER/wxretrieve.html

    It is state of california agriculture data. Daily records back to 1951, min, max, precip. Solar
    Intensity in some cases. The cool thing is that Geographiclly you get coverage for areas that are
    more rural. So, you could check Marysville data againts cities that are closer than Orland,
    like willows or williams or Oroville, I think beale AFB is close by as well. So with Williams
    a stones throw to the west ( and very rural) and baeal AFB a few miles to the east of marysville
    It might be enlightening.

  25. Steve McIntyre
    Posted May 26, 2007 at 8:06 PM | Permalink

    Are you sure CRU use Marysville ?

    I guess we’ll find out soon in response to Willis’ FOI.

  26. Posted May 26, 2007 at 8:15 PM | Permalink

    I am working with Anthony Watts in collecting Sierra USCHN site data, taking digital pictures and conducting site surveys. The results of two surveys are here and here.

    I am still working on additional stations. After automation, the Spaulding station data was not added to the GISS data base, but I could not find an explaination. Colfax is still a manual station.

    Pacific Gas and Electric is the major hydro electric provider in California and has a strong interest in climate change. I found this in a Pacific Gas and Electric Report on the impact climate change might have on hydro operations:

    8. CLIMATE STATIONS
    In years prior to the changeover of high elevation, manually read climate stations that were utilized to gather precipitation and air temperature data to remote automated, non-visited stations, station data was cooperatively shared with the National Weather Service (NWS). An NWS cooperator visited the stations daily and standards for data collection in terms of both equipment and data collection quality were for the most part assured with regularly scheduled visits by the NWS station network specialists. Today that situation is changed with the removal of most lake tenders and powerhouse personnel from many of PG&E’s mountain climate station sites. Automation and the ease of satellite telemetry have changed how climate data is collected at many mountain station sites. This change in methodology has contributed to additional uncertainty as to what is believable in terms of having significance for identifying and tracking climate change.

    The report concludes that: At this time PG&E’s water management team has not observed any significant change in hydroelectric production that can be directly attributed to global warming or climate change.

    On some of the other sites I am surveying, after automation large chunks of data are missing, in some case whole years. More on this as I survey more stations and check the data tables. These preliminary findings seem to justify PG&E concerns about the automated stations.

  27. Paul Linsay
    Posted May 26, 2007 at 8:16 PM | Permalink

    Douglas Hoyt has found a very complete site that has official photographs of weather stations. I looked up KBOS in Boston, which turns out to be located in a rectangle formed by four runways at Boston’s Logan International Airport. You can see a jet taking off in one photo. It should give excellent unbiased temperature readings.

    “http://weather.gladstonefamily.net/cgi-bin/wxphoto.pl?station=1” (link thing wouldn’t work)

    The late John Daly posted a picture of a weather station that rivals Maryville. It’s sniffing the exhaust of a Toyota pickup truck(scroll down).

  28. Posted May 26, 2007 at 10:09 PM | Permalink

    Re 22, I recall in the early 1990’s there was a policy in Australia to de-person lighthouses. Some mad Govt Dept trying to reduce their wages bill with no thought for say, the pluses of having remote area rangers, coastwatch, keeping world heritage buildings intact and in our case, world heritage weather / climate time series. This lead to installation of AWS (automatic weather stations, I think) at long term rural sites, resulting in inhomogeneities, data gaps as these new devices broke down at the far away localities and repairs took time to arrive. I am sure it is not only lighthouses and the intro of AWS was staggered over many years, I bet that the effect runs like a jagged (timewise) fault line thru Australian data. I recall I could pick the start of AWS data by a change in style of daily readings.
    Re 23, yes most of us were aware of the Pielkes good efforts for many years but 15 to 20 years on into the AGW panic and in one of the wealthiest and most advanced societies on Earth, just up the road from Silicon Valley, it is still a shock to see such new, probably expensive installations that are next to totally useless for climate purposes. Without being disparaging one has to ask the question, what could be found in the Third World.
    Anyway, it has got me interested to take my camera to a few places around here.

  29. bender
    Posted May 26, 2007 at 10:19 PM | Permalink

    installations that are next to totally useless for climate purposes

    Well, the network WAS designed primarily for local weather forecasting, wasn’t it? So go figure. Who cares about inhomogeneities if you’re trying to forecast 3 days for a single point, as opposed to 300 years over the entire globe?

  30. steven Mosher
    Posted May 26, 2007 at 10:29 PM | Permalink

    Anthony,

    The NCDC station at Colusa, CA ( station 1948) might be a good comparison to
    Marysville. just to reinforce your point. Its 20-25 miles away, a little northwest.

    It is on the UC IPM site

    http://www.ipm.ucdavis.edu/calludt.cgi/WXSTATIONDATA?MAP=colusa.html&STN=COLUSA.C

    I had a very quick look at the Pest management station weather station data (CIMIS) in Colusa on the UC IPM site:
    http://www.ipm.ucdavis.edu/calludt.cgi/WXSTATIONDATA?MAP=colusa.html&STN=COLUSA.A

    Its record is shorter than the NCDC site ( mentioned above)
    Daily data from 1983 onward. No apparent upward trend since 1983.
    Anyways, that station is located the the middle of a field.

    http://169.237.140.1/calludt.cgi/WXDESCRIPTION?MAP=colusa.html&STN=COLUSA.A

    So on one hand you have
    The marysville station, essentially in a parking lot, showing an upward trend. and 25 miles
    away in the middle of a grass feild you have no upward tend since 83.

  31. Anthony Watts
    Posted May 26, 2007 at 11:32 PM | Permalink

    RE 24, 30

    Steve, I did look at Williams, CA but I don’t know what to make of it because my site survey there uncovered nothing, meaning I can’t find the weather station. The police dept doesn’t know anything about it, as they were listed as the observer. A visit to the poliuce station at the coordinates listed revealed nothing, no sensors of any kind. I have been using the UC Davis IPM reference site to find other stations.

    I visited Colusa yesterday, right after Marysville. Its at the sewage treatment plant, 2 miles SW of town, and here are photos looking north and south, resepctively. It truly is in the middle of nowhere, and despite some questionable mounting, seems to read well enough. Lots of wind there helps exposure and mixing.

  32. bernie
    Posted May 26, 2007 at 11:41 PM | Permalink

    There are two UHI effects right? One comes into play when you are averaging stations across a geography that contains both cities and rural areas. The second occurs for each station as its immediate environment changes through population growth and development. The other issue is the siting of the station. Here there are also two conditions: Smart and not so smart!

  33. Barclay E. MacDonald
    Posted May 27, 2007 at 12:20 AM | Permalink

    #31

    But, arguably, even this site will get reflected heat from those buildings all day long. (One building wall is facing East and the other West, according to your description.) They may be fairly recent, being of good condition except for some thinning paint.

  34. Paul
    Posted May 27, 2007 at 12:21 AM | Permalink

    What would be really useful is hourly temperature readings as well as rural/urban. This way we could tell if cities are acting as heat sinks; giving higher night temperature results urban areas.

  35. Barclay E. MacDonald
    Posted May 27, 2007 at 12:29 AM | Permalink

    Of course you are right Bender. These station locations should not be a major problem for recording and predicting, local day to day weather. But to use them to conclude that over the last 100 years there has been global warming of .6C, that it is now the hottest in the last 1000/1,000,000 years and to further to conclude what portion may be anthropogenic, well, to say the least, one must be careful. I guess I’m not saying anything that isn’t already well understood.

  36. Bob Weber
    Posted May 27, 2007 at 1:04 AM | Permalink

    #27 Here is a working link to stations with photos.
    http://weather.gladstonefamily.net/cgi-bin/wxphoto.pl

    Bob

  37. Matt
    Posted May 27, 2007 at 1:40 AM | Permalink

    Very interesting.

    This doc here http://www.srh.noaa.gov/ohx/dad/coop/EQUIPMENT.pdf notes two things that might be of interest:

    1) Page 11 notes that calibration isn’t possible on the MMTS, and gives a procedure for checking accuracy. If MMTS and thermometer agree within a degree, it’s considered OK
    2) Towards the bottom of page 11, note that observers are only to record temps to the nearest degree.

    On a correctly functioning unit, the actual temp could be 71.5, with a recorded temp of 70 and that would be considered within acceptable limits (measures one degree low, plus half degree rounding). Or actual temp could be 71.5, and recorded temp could be 73 and that would be within acceptable limits.

    These are thermistor sensors, so in practice the error would be a constant offset from reading to reading, closing that 3 degree window down to +/- 1 degree.

    But, if a unit is ever replaced, it’s possible the replacement unit could have a +1 bias while the unit it replace had a -1 degree bias. Both would be within limits.

  38. Barclay E. MacDonald
    Posted May 27, 2007 at 1:56 AM | Permalink

    #36

    Your link works. I happened to check the entry for Dubai. Interestingly, it says, “The location information for this site was gathered from 7 data sources. The possible locations for this site are up to 2.2 miles away. This makes it possible that the location chosen is incorrect. The source which most closely matches the consensus is chosen. Many of these sources only report the location to the nearest minute (or hundredth of degree). This can lead to the marker being placed noticeably incorrectly. Please see Discussion of Location Errors for more information.” They indicate they don’t have a picture of the station, but would like one.

  39. EW
    Posted May 27, 2007 at 3:49 AM | Permalink

    The observatory in Prague Klementinum is in the very center of the city – there is a placemark at Google Earth (coordinates:W 14.415801,N 50.086528). Therefore it was a subject of some studies also including UHI effect:

    Article about heat waves in 20th century – in 40-50’s it was hot (also observed in Basel, Switzerland), but small interannual differences, whereas in 90’s, there were big differences)
    Article about differences in UHI under various weather “The increase in the heat island intensity is steeper under anticyclonic than cyclonic conditions in all seasons except for spring.”

    Heat waves and circulation – where the heat waves in 1990’s are ascribed to “an enhanced persistence of the atmospheric circulation over Europe in the summer season because all groups of weather types have considerably increased residence times in 1988’€”97 compared with long-term means.”

  40. John A
    Posted May 27, 2007 at 5:29 AM | Permalink

    These are the shockers. How does anyone get sensible results from this?

  41. Bob Koss
    Posted May 27, 2007 at 6:16 AM | Permalink

    This site is as close as you can get to the Marysville site. Seems to be about two miles east, if both sets of coordinates are correct. Not as long a series, unfortunately.

    The NCDC database I have gives the mean for all readings for the day. Also, tmin and tmax for which I’ve calculated the average. It has 24 readings per day about 28% of the time.

    I’ve done maybe a dozen plots this way.(mostly airports) This is the first time I’ve seen the average drop below the mean for any extended time. Usually the mean stays about 3-6 tenths lower than min max average. At this site they swap and there is up to a full degree difference. Very curious.

    Here’s Beale AFB. About nine miles east of Marysville.

  42. DocMartyn
    Posted May 27, 2007 at 8:15 AM | Permalink

    Dear Anthony Watts, I look forward to the results of your expirement with the threeS tevenson Screens boxes.
    I am just wondering if you are going to make air pressure recordings at the same time as you are measuring temperature? I wonder about the total energy in the system, which will depend on both the temperature, water vapor content and the air pressure. It is possible that there may exist air pressure gradients as well as temperature gradients in the different boxes.

    With regard to your previous photographic series, I wander what a childrens sized paddling pool at the base of one of the boxes will do to the recorded temperature.

    Good Luck.

  43. Posted May 27, 2007 at 8:43 AM | Permalink

    Re:31,

    Anthony, the Willam’s record goes from July 1928 to Nov 1988. Maybe the station has been removed..

  44. Anthony Watts
    Posted May 27, 2007 at 9:02 AM | Permalink

    RE42 Doc,

    I have a complete weather station on the premises, which will record and log pressure, wind, solar radiation, air temp, and DP simultaneously with the running Stevenson Screen experiment. When I’m ready, I will have real-time data posted to the net in graphical form so everybody can watch.

    I also have been in contact with the chief chemist for the National Lime Corporation and the National Lime Institute to acquire the correct hydrated lime formula that meets historical formulas for whitewash. They were gracious enough to send me the supplies I need, gratis.

    BTW an interesting side-note to all of this is that by preliminary experiments with wood slats and thermistors indicate that whitewash appears to be a superior coating when it comes to energy efficiency for a/c of buildings. With all the rage being sustainability these days, whitewash will keep a building 2-3 degrees cooler it appears. Old is new again.

  45. Anthony Watts
    Posted May 27, 2007 at 9:18 AM | Permalink

    RE 37 Matt,

    There’s a bit of a smoking gun in the document for CWO’s you reference. It says on page 13 “The CRS should kept painted with a semi-gloss exterior white paint at all times, not only for looks but also to reflect the suns rays.”

    and then again on page 14 under maintenance:

    “Maintenance is minimal for the most part. The CRS should be kept clean with a good quality semigloss white painted surface.”

    This contrasts the earlier USWB documents which specify whitewash. It appears that the NWS simply caved to availability, and stopped requiring whitewash somewhere around 1979, which is when I first start noticing the “semi-gloss” paint reference. Note they don’t say what kind, so the variability may now include latex, enamel, oil based, or expoxy based “semi-gloss”.

    Score one for entropy.

  46. Steve McIntyre
    Posted May 27, 2007 at 9:21 AM | Permalink

    The USHCN readme says:

    The U.S. Historical Climatology Network (USHCN) is a high-quality, moderate-sized data set of monthly averaged maximum, minimum, and mean temperature and total monthly precipitation. The USHCN was developed to assist in the detection of regional climate change; it is comprised of 1221 high-quality stations from the U.S. Cooperative Observing Network within the 48 contiguous United States.

    I’ve double-checked that Marysville is one of the “high-quality” stations in the station list.

  47. Anthony Watts
    Posted May 27, 2007 at 9:22 AM | Permalink

    Ya know what, it just hit me, I need to do a parallel test with the MMTS screen at the same time I do the Stevenson Screen tests, since it is the newest shelter. Since I’m comparing shelters, it seems only fitting. Does anybody know where I can buy the standard MMTS stacked plate shelter?

  48. Steven mosher
    Posted May 27, 2007 at 9:33 AM | Permalink

    Anthony and Steve.

    I did a more extensive check on Colusa and did a Annual average. for 1951 to 2006.
    Eyeballing it, maybe a slight upward trend, nothing like Marysville.
    As mentioned above Yuba is closer, albeit a more urban area than Colusa.

    Anyways, I decided to take another look at the Marysville record.

    I went here

    http://cdiac.ornl.gov/cgi-bin/broker?_PROGRAM=prog.climsite_monthly.sas&_SERVICE=default&id=045385

    Station 045385

    And downloaded the Monthly data for 1870 to the present.

    Selected Monthy Mean and Urban Adjusted Monthly mean for download.

    Load the file in Excell for a quick look and subtract Urban monthly Mean from
    the unadjusted data.

    Interesting, Most months the unadjsuted values exceed the urban adjusted.
    So back in 1870 they are adjusting the monthly mean in Marysville ( down .12 F)
    for Urban effects. Gold fever I guess.

    Then I spot some outliers. Jan 1914. The adjusted file is 6F cooler.
    also in the file some of the adjustment go Negative. So, in June 1895
    the (unadjusted-urban adjust) = -2.3

    Strange, maybe they are making adjustments at long time scales ( like steve
    notes) and then propgating those changes down to monthly averages. Not
    sure how this would result in a one month 6 degree F adjustment down
    or a 2.3F adjustment up?

    Anyways, have a nice memorial day

  49. Anthony Watts
    Posted May 27, 2007 at 10:17 AM | Permalink

    Steven,

    FYI many thermometers were exposed unsheltered until about 1893-1895 when Stevenson screens started to be deployed. There was an extensive paper published in BAMS on this issue.

    You can see that effect here in the early data from station 425725910030 Chico, CA (about 50 miles northwest) seen here

    http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=425725910030&data_set=1&num_neighbors=1

    Note the wide swings during the period from 1880 to just before 1900.

    Odd that they would include this portion of the record knowing this exposure issue, but then again based on what we’ve seen, quality control doesn’t seem to weigh heavily on the “quality” USCHN sites

  50. Steve McIntyre
    Posted May 27, 2007 at 10:43 AM | Permalink

    [deleted – I’ve amended this – see below]

  51. bender
    Posted May 27, 2007 at 10:53 AM | Permalink

    This one, I would like to see explained.

  52. EW
    Posted May 27, 2007 at 10:55 AM | Permalink

    Now if someone would undertake a series of buckets-inlet-hull thermometer measurement comparisons as a sea experiment complementing the land ones of Anthony Watts…

  53. bender
    Posted May 27, 2007 at 10:59 AM | Permalink

    The decadal cycle is the solar cycle, is it not? Then the lower-frequency fluctuation, the one modulating the amplitude of the decadal cycle, with cooling in the 1960s-70s, is that the presumed effects of aerosols? Then these data are not “raw”, correct?

  54. Posted May 27, 2007 at 11:13 AM | Permalink

    Hmmm… A few obvious things.

    1) CO2 effect on global temperature is global. It should not be greater in urban areas.
    2) Urban and Suburban areas make up a very small part of the actual global surface and data should be weighted by the percentage of earth surface area they represent. Common f***ing sense analysis.
    3) Urban areas are much more likely to show solar effects (such decreased cloud cover) due to very low albedo of the surface and should show the exact same temperature rise due to GHG effects as similar rural areas.
    4) Urban areas contribution should be further reduces by identifying heat sources and realizing that they ultimated will dissipate throughout the volume of the atmosphere.

  55. JerryB
    Posted May 27, 2007 at 11:15 AM | Permalink

    GISS does not use “raw” data for USHCN stations. It uses
    data with some USHCN adjustments (excluding adjustments for
    missing data and also excluding UHI adjustments).

  56. steven mosher
    Posted May 27, 2007 at 11:15 AM | Permalink

    anthony,

    I suspected that there would have to be adjustments for station changes etc.
    Yet the issue ( to second what steve says in #50) is the adjustment in the GHCN
    data appears to have been done on a continuous basis. Essentially, subtracting
    a small amount ( .12 F to .17)from the monthly mean. But then every once in a while
    you have individual months ( like jan 1914) that have a 6F degree adjust. and periodicly
    through the data you have negative adjustments. On an annula basis these odd months
    don’t have a big effect, which led me to believe that the adjustemnt may have be made
    on a annual scale or multi annual scale and then massaged into the monthly data,
    automatically, without anyone making a check on the monthly results.
    I dont thnk an individual went over this site and in the middle of 1895 decided that
    june needed a -2.3F adjustment. If that were the case, one could expect to find this kind
    of adjustment decision documented. Same with Jan 1914. But, I could be wrong.

  57. steven mosher
    Posted May 27, 2007 at 11:27 AM | Permalink

    SteveM,

    Dont know if this helps explain how USHCN adjusts data.

    2. Descriptions and Formats of USHCN Data Files and Supporting Files
    The section describes the various USHCN temperature and precipitation data files and several supporting files that contain information about the USHCN stations. Temperature and precipitation data files will be described first, followed by descriptions of all other files.
    The data for each station in the USHCN are subjected to several quality control tests, homogeneity tests, and adjustment procedures. These steps are applied so as to sequentially produce six different types of data records, with each successive record type using the preceding record type as input. Each data record type is listed below with a description of the tests and/or adjustments that go into making each of them.

    Areal Edited (Raw)
    A quality control procedure is performed that uses trimmed means and standard deviations in comparison with surrounding stations to identify suspects (> 3.5 standard deviations away from the mean) and outliers (> 5.0 standard deviations). Until recently these suspects and outliers were hand-verified with the original records. However, with the development of more sophisticated QC procedures at NCDC, this has been found to be unnecessary.

    TOBS (Time of Observation)
    The temperature data are adjusted for the time-of-observation bias (Karl, et al. 1986), which occurs when observing times are changed from midnight to some time earlier in the day. The ending time of the 24-h climatological day varies from station to station and/or over a period of years at a given station. The time of observation (TOB) introduces a non-climatic bias into the monthly means. The TOB software is an empirical model used to estimate the TOB biases associated with different observation schedules and the routine computes the TOB with respect to daily readings taken at midnight.

    MMTS (Maximum/Minimum Temperature System)
    Temperature data at stations that have the Maximum/Minimum Temperature System (MMTS) are adjusted for the bias introduced when the liquid-in-glass thermometers were replaced with the MMTS (Quayle et al. 1991). The MMTS program debiases the data obtained from stations with MMTS sensors. The NWS has replaced a majority of the liquid-in-glass thermometers in wooden Cotton-Region shelters with thermistor based maximum-minimum temperature systems (MMTS) housed in smaller plastic shelters. This adjustment removes the MMTS bias for stations so equipped with this type of sensor. The adjustment factors are most appropriate for use when time series of states or larger areas are required.

    SHAP (Station History Adjustment Program)
    The homogeneity adjustment scheme described in Karl and Williams (1987) is performed using the station history metadata file to account for time series discontinuities due to random station moves and other station changes. The debiased data from the MMTS adjustment are then entered into the Station History Adjustment Program or SHAP. The SHAP allows a climatological time series of temperature and precipitation adjustment for station inhomogeneities using station history information. The adjusted data retain their original scale and are not anomaly series. The methodology uses the concepts of relative homogeneity and standard parametric (temperature) and non parametric (precipitation) statistics to adjust the data. In addition, this technique provides an estimate of the confidence interval associated with each adjustment. The SHAP program debiases the data with respect to changes other than the MMTS conversion to produced the “adjusted data”. Specific details on the procedures used are given by Karl and Williams (1987).

    FILNET (Fill Missing Original Data in the Network)
    Estimates for missing data are provided using a procedure similar to that used in SHAP. This adjustment uses the debiased data from the SHAP and fills in missing original data when needed (i.e. calculates estimated data) based on a “network” of the best correlated nearby stations. The FILNET program also completed the data adjustment process for stations that moved too often for SHAP to estimate the adjustments needed to debias the data.

    Urban (Urban Warming Adjustment)
    The final adjustment is for an urban warming bias which uses the regression approach outlined in Karl et al. (1988). The result of this adjustment is the “final” version of the data.

    The Areal Edited, TOBS, FILNET, and Urban data files are available in this version of the USHCN. The MMTS and SHAP iterations of the data are also available (email D. Kaiser at CDIAC), but are used less often in favor of the FILNET data files, which incorporate both the MMTS and SHAP adjustments.

    Currently all data adjustments in the USHCN are based on the use of metadata. However, station histories are often incomplete, or changes that can cause a time series discontinuity (such as replacing a broken thermometer with one that is calibrated differently) are not routinely entered into station history files. Because of this, NCDC is currently developing another step in the processing that will apply a time series discontinuity adjustment scheme described in Peterson and Easterling (1994) and Easterling and Peterson (1995). This methodology does not use station histories and identifies discontinuities in a station’s time series using a homogeneous reference series developed from surrounding stations.

  58. JerryB
    Posted May 27, 2007 at 11:28 AM | Permalink

    For USHCN stations, the GHCN adjusted version will usually
    be very close to the numbers used by GISS. The differences
    would be instances in which USHCN adjustments included
    estimates for missing data.

    USHCN adjustments are calculated, and applied, on a monthly
    basis, varying with the time of year.

    Re #48,

    Steven, the data that you get at that link are said to be
    “filnet” and urban adjusted. The filnet data include
    other adjustments. See http://www.ncdc.noaa.gov/oa/climate/research/ushcn/ushcn.html
    for brief descriptions thereof.

    Re #49,

    The link in that comment is to a temporary file which
    will be deleted within days, possibly within hours.

    BTW, the GISS server seems to be crawling this weekend. I
    do not recall seeing be as slow in the past.

  59. steven mosher
    Posted May 27, 2007 at 12:26 PM | Permalink

    Thanks JerryB!

    When I read through the entire link I was a lot more well informed.
    It’s a good piece of work. Still, I’m puzzling on these one time “oddball”
    adjustments. Since Anthony mentioned chico I looked at that. from 1870
    on the adjustments are all in the same range lets say .18 F to .22 F
    Then around 1949, the adjustments start to smoothly ramp up ( So, One assumes some kind of station
    change or shelter change or whatever) and the adjustment ramps up to around
    .8F to the year 2000. All well and good.But then here and there are these “oddball”
    months. adjustment of 1.54F in August of 1980. Is this an artifact of the adjustment
    routines or an adjustment made for a specific reason that month. done by hand?

  60. Steve McIntyre
    Posted May 27, 2007 at 12:26 PM | Permalink

    Here is an amended calculation of the difference between the GHCN raw data and the GISS “raw” (their dset=0) data.

    Here is the difference between the GHCN raw monthly data and the USHCN “raw” (“Areal”) data [correction – this is TOBS version]

    In each case, the adjustments at this stage have the effect of increasing the trend.

    It’s amazing the amount of energy they spend on adjusting for the Time-of-Observation bias which increases 20th century trend, as compared to lack of attention to the Parking Lot bias.

  61. steven mosher
    Posted May 27, 2007 at 12:49 PM | Permalink

    Hey Anthony,

    RE #47.. USHCN had this to say about adjustments for new type shelters.

    http://cdiac.ornl.gov/epubs/ndp/ushcn/ndp070.html

    “Another factor users should be aware of pertains to thermometers used at the HCN/D stations. In 1984, the NWS introduced a new Maximum/Minimum Temperature System (MMTS) at cooperative network observing stations. Through 1994, 645 out of 1062 (about 60%) of the HCN/D stations had installed an MMTS (the station history file identifies these stations). Concerns have arisen about the calibration of this system as compared to that of the earlier thermometric system. The new system is thermistor-based with a “beehive like” instrument shelter, whereas the older systems consisted of liquid-in-glass thermometers, mounted inside a Cotton Region Shelter (Stevenson Screen). Quayle et al. (1991) looked into performance differences of the two systems and found that the new system produces maximum temperatures about 0.3 deg C lower and minimum temperatures about 0.4 deg C higher than the old system. Unfortunately, because large samples of side-by-side overlapping measurements are not available, site-specific corrections cannot yet be derived and only large-scale temperature changes can be corrected. Furthermore, daily biases, which are likely to be dependent on synoptic conditions, are unlikely to be the same from day to day. Thus, to date there has been no attempt to adjust the daily temperature data from the HCN/D for these instrument-induced biases”

  62. JerryB
    Posted May 27, 2007 at 1:16 PM | Permalink

    Steven,

    The README.TXT file at
    ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/
    describes codes for adjustments by the filnet program,
    but not for the urban adjustments, as far as I can see.

    SteveM,

    “Here is the difference between the GHCN raw monthly data and the USHCN
    ‘raw’ (‘Areal’) data.”

    When I make that comparison, the annual differences that I get are 0.03 C
    or less, and usually less, which I attribute to rounding differences
    in the conversions of each month’s numbers from F to C.

  63. Steve McIntyre
    Posted May 27, 2007 at 1:27 PM | Permalink

    Jerry, here are differences for Marysville USHCN Area minus Filnet. The adjustments seem to be determined by season. While recent adjustments are not large, older adjustments can be significant as shown below and cannot be attributed to rounding.

    1911 -0.599 -0.599 0.000 0.000 0.000 -0.342 -0.343 -0.343 -0.120 -0.118 -0.117 -0.598
    1912 -0.598 -0.598 0.000 0.000 0.000 -0.342 -0.343 -0.342 -0.120 -0.118 -0.117 -0.599
    1913 -0.599 -0.599 0.000 0.000 0.000 -0.342 -0.343 -0.343 -0.121 -0.118 -0.118 -0.599

    1992 0.046 0.047 0.030 0.031 0.031 0.016 0.016 0.015 0.015 0.016 0.015 0.047
    1993 0.047 0.046 0.031 0.031 0.031 0.015 0.016 0.016 0.016 0.015 0.015 0.047
    1994 0.046 0.047 0.031 0.031 0.031 0.016 0.016 0.015 0.016 0.016 0.016 0.046

  64. JerryB
    Posted May 27, 2007 at 1:34 PM | Permalink

    R #63 SteveM,

    Yes, but in your comment #60 you indicated that you were
    using areal, i.e. raw, not filnet, which includes several
    adjustments.

  65. Posted May 27, 2007 at 1:41 PM | Permalink

    Here is a map of the CRU stations as of 1986 plus the 1994 Jones update.
    If anyone wants the data I can email an xls file ~ 1.8 MB, or whatever.

  66. Steve McIntyre
    Posted May 27, 2007 at 2:58 PM | Permalink

    Jerry, as you pointed out to me, the USHCN code “A” doesn’t mean Area, but Filnet. However, if one looks at the USHCN adjustment stages, the Area-TOBS delta is different for each month, but the TOBS-Filnet adjustmnet (which is what I actually have above) is different by season for some reason. GISS raw is close to USHCN Filnet, but not exactly.

    #AREA- TOBS
    1910 0.372 0.239 0.278 0.217 0.144 0.166 0.139 0.194 0.272 0.289 0.428 0.322
    1911 0.372 0.238 0.278 0.217 0.145 0.167 0.139 0.194 0.272 0.289 0.428 0.322
    1912 0.372 0.239 0.277 0.217 0.145 0.167 0.139 0.194 0.272 0.289 0.428 0.322
    1913 0.372 0.238 0.277 0.217 0.145 0.167 0.139 0.194 0.272 0.289 0.428 0.322

    ##TOBS – FILNET
    1911 1.056 1.056 0.500 0.500 0.500 0.833 0.833 0.834 0.517 0.517 0.516 1.056
    1912 1.056 1.055 0.500 0.500 0.500 0.833 0.833 0.833 0.517 0.517 0.517 1.056
    1913 1.055 1.056 0.500 0.500 0.500 0.833 0.833 0.833 0.517 0.517 0.517 1.056

  67. JerryB
    Posted May 27, 2007 at 3:17 PM | Permalink

    Just for review:

    For USHCN stations only,

    USHCN “raw” should be within rounding of GHCN “raw”.
    GHCN adjusted should be within rounding of USHCN filnet.
    GISS “raw GHCN data+USHCN corrections” should be close to
    GHCN adjusted minus filnet adjustments for missing data.

  68. Demesure
    Posted May 27, 2007 at 3:23 PM | Permalink

    #63, adjustement of 0.000 can’t be adjustment but tweaking based on “expert” judgment.

  69. Fred Harwood
    Posted May 27, 2007 at 3:29 PM | Permalink

    I may have missed the discussion, but my experience with thermistors (metal juctions) is that they fail open, not shorted, which means that cumulatively they must progress to higher impedences. Might that mean higher temperatures as they age?

  70. Steve McIntyre
    Posted May 27, 2007 at 3:44 PM | Permalink

    Jerry, here’s what I got for Marysville. I think that I’m on the same page here. For the most part, USHCN Area = GHCN Raw, but there are some differences, perhaps the editions are different.



    Same with USHCN Filnet and GHCN Adj. Some changes in one period.

  71. Steve McIntyre
    Posted May 27, 2007 at 3:56 PM | Permalink

    A GISS Rural Cooling Adjustment?

    Here’s another odd plot, which I’ll have to triplecheck. The first diagram below shows 3 calculation stages: GHCN Raw- GISS Raw (which should deal with all the time of observation and station history stuff); GISS Raw – GISS Adjusted (the GISS urban heat adjustment); the combined GHCN Raw – GISS Adjusted. I’ve done this first for Marysville GISS population 12,000, periurban lighting; and Orland GISS rural; unlit. We’ve seen a strong UHI effect at Marysville and GISS makes a substantial adjustment in the early portion of this series – up to 2 deg C, notwithstanding Jones view that UHI does not exceed 0.05 deg per century.


    Marysville CA Adjustment Stages

    Now the same things with Orland CA. This time the GISS adjustment goes the other way – I presume that the rationale for this adjustment is the well-known “rural cooling” effect, but I’ll ocntinue investigating.

  72. JerryB
    Posted May 27, 2007 at 4:01 PM | Permalink

    Steve,

    With apologies: I forgot to warn you that the USHCN version
    currently at ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/
    is not used either in GHCN, or in GISS, and it has different
    adjustments than the previous version that was posted there.

    A previous USHCN version, which was not posted at the USHCN ftp
    site, is what is being used both by GHCN and GISS. I do not
    have a copy of that version, but I do have copies of earlier
    versions. Let me check the filnet vs GHCN adjusted from
    an earlier version and get back to you.

  73. Anthony Watts
    Posted May 27, 2007 at 4:08 PM | Permalink

    I decided I’d drop some more fun with entropy your way. Here is the USHCN station in Redding, CA # 425725920010 now operated by the US Forest Service at their HQ located at the Redding Airport it used to be operated by the NWS, but that WSFO closed in the mid 90’s.

    Like Marysville, the site is surrounded by asphalt, and the surface is unnatural – its wood chips over weedmat, and I’ll have to say it was hot as hot to walk on.

    But the kicker is the “accessories” they’ve
    added for convenience of running the hygrometer and for night observations. Yes is another fine high-quality USHCN site. I winder how many times they forgot to turn off the light? Its a 65 watt bulb by the way.

  74. Anthony Watts
    Posted May 27, 2007 at 4:12 PM | Permalink

    sorry, lets try that again:

  75. Steve McIntyre
    Posted May 27, 2007 at 4:23 PM | Permalink

    How many climate modelers does it take to change a light bulb?

  76. Steve McIntyre
    Posted May 27, 2007 at 4:31 PM | Permalink

    I wonder what a “low quality” weather station would be like.

    I guess the ISO is going to come and take lessons from Jones and Hansen on QC.

  77. JerryB
    Posted May 27, 2007 at 4:35 PM | Permalink

    Steve,

    I’ve gone back a ways and am not seeing what I expected to be there.

    In the GHCN overview paper, a PDF of which can be found at:
    http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php?name=temperature
    I had read the following:

    “Approximately 1,000 homogeneity-adjusted stations in the United States were
    computed using a different technique. These are high quality rural stations
    taken directly from the U.S. Historical Climatology Network (U.S. HCN,
    Easterling et al. 1996a) a sister project to GHCN. These data were adjusted
    using a metadata approach as part of the creation of the U.S. HCN and their
    adjusted time series were directly incorporated into GHCN.”

    I do not clearly recall what led me to conclude that those adjustments
    were up to, and including, the USHCN filnet adjustments. In any case, I
    must withdraw my earlier comment that they are, and go back and figure out
    just what they do include, a task which will not get done tonight.

  78. Steve McIntyre
    Posted May 27, 2007 at 4:41 PM | Permalink

    Google “jones hansen climate”: CA is first, ahead of IPCC or realclimate

  79. steven mosher
    Posted May 27, 2007 at 5:37 PM | Permalink

    Hi SteveM.

    Well, I took a look at Maryville data from USHCN. Very little difference in the urban adjustment.
    Then I compared 3 GISS files

    1. what they term “GHCN + UShcn adjustments)
    2. Comibined sites.
    3. Homegenity.

    GISS #1 and #2 track the USHCN very well. throughout the whole series.

    Homegenity, as you note, diverges substantially from USHCN. For grins I compared two records
    from Colusa : one with records from 1951 on, the other with records from 1983 on. ( by the
    way both series were remarkable similair in shape to each other and to the marysville record.)
    Colusa rural sites were 1-1.5C lower than Marysville over the period.

    I suppose Hansen does not publish the slope of his adjustment curve for Marysville?
    He just describes it:

    The nonrandom inhomogeneity of most concern is anthropogenic influence on the air sampled
    by the thermometers. Urban heat can produce a large local bias toward warming [Mitchell, 1953;
    Landsberg, 1981] as cities are built up and energy use increases. Anthropogenic effects can also
    cause a nonclimatic cooling, for example, as a result of irrigation and planting of vegetation, but these
    effects are usually outweighed by urban warming.
    We take advantage of the metadata accompanying the GHCN records, which includes
    classification of each station as rural (population less than 10,000), small town (10,000 to 50,000),
    and urban (more than 50,000), to calculate a bilinear adjustment for urban stations. The adjustment
    is based on the assumption that human effects are smaller in rural locations. We retain the
    unadjusted record and make available results for both adjusted and unadjusted time series (section
    10). The homogeneity adjustment for a given city is defined to change linearly with time between
    1950 and the final year of data and to change linearly with a possibly different slope between 1950
    and the beginning of the record. The slopes of the two straight line segments are chosen to minimize
    the weighted-mean root-mean-square difference of the urban station time series with the time series
    of nearby rural stations. An adjusted urban record is defined only if there are at least three rural
    neighbors for at least two thirds of the period being adjusted. All rural stations within 1000 km are
    used to calculate the adjustment, with a weight that decreases linearly to zero at distance 1000 km.
    The function of the urban adjustment is to allow the local urban measurements to define short-term
    variations of the adjusted temperature while rural neighbors define the long-term change. The break
    in the adjustment line at 1950 allows some time dependence in the rate of growth of the urban
    influence.”

  80. bender
    Posted May 27, 2007 at 8:58 PM | Permalink

    The adjustment is based on the assumption that human effects are smaller in rural locations.

    So Hansen here is conflating city-wide UHI effects vs. micro-site effects? Are you sure you aren’t making this up? What is the source? I can not LOL. This is serious business.

  81. Posted May 27, 2007 at 10:10 PM | Permalink

    This reminds me of a water resources study we were doing in S. Iran in 1969. At the town of Jask, at the entrance to the gulf, the mayor was also the local weatherman. We stayed at his place overnight, and I noticed the rain gage and thermometer were indoors on his mantle; when I asked why they were there, he said he had to bring them in at night so no one would steel them. So the night half of the record is blank.

    I would think this or similar bias might be a fairly common occurrence, especially in the earlier 3rd world records. However the colonial records – pre WWII – may be of somewhat better quality. Not sure to what extent things may have improved; probably very spotty. War and revolutions have left their mark in terms of long periods of no data in many countries between 30N and 30S.

  82. Posted May 27, 2007 at 11:29 PM | Permalink

    # 70, 71, …

    Brohan et al gives a good overview how this kind of adjustments are taken into account when computing uncertainties. I’m not sure if I understood it correctly, but it seems to be a humoristic approach

    1. Make a histogram of known adjustments. You’ll lose time information, but stochastic processes are not invented in climate science yet, so there are no other options.

    2. This histogram is bimodal, but assume that true required adjustment histogram is Gaussian -shaped. So you’ll need to fit a Gaussian distribution to that bimodal histogram.

    3.

    The homogenisation uncertainty is then given by this missing component (dotted line in figure 4), which has a standard deviation of 0.4 C.

    4. Tell everyone that you understand that adjustments are not symmetric, but add a sentence

    Although a real effect, this asymmetry is small compared with the typical adjustment, and is difficult to quantify; so the homogenisation adjustment uncertainties are treated as being symmetric about zero.

    5.

    So the homogenisation adjustment uncertainty for any station is a random value taken from a normal distribution with a standard deviation of 0.4 C.

    6. Finally, tell everyone that it is important to make adjustments

    Figure 4 also demonstrates the value of making homogenisation adjustments. The dashed line is an estimate of the uncertainties in the unadjusted data, and the dotted line an estimate of the uncertainties remaining after adjustment. The adjustments made have reduced the uncertainties considerably.

    Quotes are from the linked paper.

  83. bender
    Posted May 28, 2007 at 6:38 AM | Permalink

    In this paper Jones, like Hansen in the previous papaer, is not differentiating between “urbanization effect” and microsite effects due to changing circumstances around the sensor. The UHI effect may or may not be as marginal as he claims. But the microsite effects, which are ignored, may be significantly larger.

  84. JerryB
    Posted May 28, 2007 at 7:13 AM | Permalink

    SteveM,

    At the USHCN ftp site, there is an old set of files showing
    intermediate USHCN adjustment results. They are in:

    ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/OtherIntermediates/

    Of those files, the file hcn_shap_ac_mean.Z has numbers that include
    TOB, MMTS, and SHAP adjustments, but not FILNET adjustments. Based on
    comarisons using Marysville data, it appears to me that the GHCN adjusted
    data for USHCN stations uses that level of USHCN adjustments: SHAP, not
    FILNET.

  85. bender
    Posted May 28, 2007 at 7:22 AM | Permalink

    Whoops, posted a little too quickly there. In the next paragraph they write:

    Bias correction uncertainties are estimated following [Folland et al., 2001] who considered two biases in the land data: urbanisation effects [Jones et al., 1990] and thermometer exposure changes [Parker, 1994].

    So they are well aware of the two scales of effect.

    But later still:

    Thermometer exposure changes. Over the period since 1850 there have been changes in the design and siting of thermometer enclosures; many early shelters can differ substantially from the modern Stevenson-type screen. It is sometimes possible to determine the time of change by the homogeneity assessments discussed in section 2.3.1, but this is only possible if changes at neighbouring stations are implemented at different times. The bias errors in this section, therefore, allow for the possible simultaneous replacement across entire countries with Stevenson-type shelters. The possible effect of such changes was investigated in [Parker, 1994], who concluded that there was a possible difference between 1900 and the present day of about 0.2°C because of such exposure changes.

    So they are ASSUMING the modern screen is superior to (a) old screens and (b) non-screens, and they are ASSUMING the exposure effect is dominated by screen design. But screen design, as A. Watts points out, is only one factor. Aside from all the other things influencing sensor microenvironemnt (e.g. whitewash vs. semigloss latex), the dominant factors may be meso-scale, in between that of the city and that of the sensor screen. The environment within 10-100m of the sensor, say. (asphalt, steel towers, bbqs, ac vents, auto rads, auto exhaust, etc.)

    Bottom line: they offer up 0.2°C as the net increase. Seems low to me. Anyone: do I hear 0.3°C? 0.4? Place your guesstimates now. I will collate them along with the A in AGW guesses.

  86. John A
    Posted May 28, 2007 at 7:55 AM | Permalink

    Bottom line: they offer up 0.2°C as the net increase. Seems low to me. Anyone: do I hear 0.3°C? 0.4? Place your guesstimates now. I will collate them along with the A in AGW guesses.

    Frankly I’ve seen less arm-waving at a traffic-cop convention.

  87. JerryB
    Posted May 28, 2007 at 8:08 AM | Permalink

    Let me further qualify my comment #84 as limited to years
    following 1903.

    BTW, it seems that for that station, and for those years,
    the filnet adjustments largely negated the shap adjustments.

  88. bender
    Posted May 28, 2007 at 8:37 AM | Permalink

    Re #86 Yes, understood. But I think maybe I am more shocked that you are by these revelations. For the first time I’m having a “stop the presses” moment … whereas you’ve been there from the start. It’s one thing for a denialist/skeptic to go around arm-waving about Jones underestimated UHI effects (along with a bunch of other effects), and quite another to have a disinterested party precisely identify and then quantify the nature of the bias. I’ve not been on the UHI bandwagon, but I’ve also not criticized anyone who suggested a better quantification of UHI. Now I include microsite in there as well. So, yes, arms are waving.

  89. JP
    Posted May 28, 2007 at 12:29 PM | Permalink

    I am normally just an interested and intermittent observer, but can’t resist asking if anyone has seen or reviewed the following article, which seems highly relevant to the thread:

    http://www.agu.org/pubs/crossref/2006/2006GL026358.shtml

    Apologies if this has been cited before – pl just ignore if that is the case.

    JP

  90. Posted May 28, 2007 at 2:03 PM | Permalink

    #85

    Bottom line: they offer up 0.2°C as the net increase. Seems low to me. Anyone: do I hear 0.3°C? 0.4? Place your guesstimates now. I will collate them along with the A in AGW guesses.

    How about zero? Folland 2001:

    Our estimated 2 \sigma extratropical hemispheric uncertainties are 0.2°C before 1900, declining to zero by 1930 because universal standard exposures are assumed and random errors in standard exposures are already included in the optimal averaging.

  91. Posted May 29, 2007 at 2:55 AM | Permalink

    Bit OT, but for those who are interested in instrumental uncertainties:

    Folland 2001 in above quote mentions optimal averaging, and refers to Shen et al 1998. Shen writes

    Using EOFs to take account of spatial inhomogeneity, Shen et al. (1994) developed such an optimal method for the average on the entire globe. They concluded that about 60 well-distributed stations can yield a global average annual mean surface air temperature with an error less than 10% compared with the natural variability.

    ( Shen et al 1994 ) Shen also assumes that

    The systematic errors are assumed to have been removed from the raw data, and hence the remaining random error is uncorrelated with the anomaly field and with errors at other locations:

    Note that gavin just mentioned 60 stations. IMO the problem here is that Shen 94 used U.K. dataset to get that number, does the same number apply to global dataset? I don’t know, I find Shen’s papers interesting. On the other hand, Folland 2001 and Brohan 2006 papers are clearly deeply flawed articles, and their publication has damaged the reputation of this promising new branch of science.

  92. MarkW
    Posted May 29, 2007 at 5:36 AM | Permalink

    How can anyone state, with any degree of confidence, that they know what the world’s temperature is?
    The majority of the planet is virtually unmeasured.
    Of the fraction that is measured, much is measured too sparsely to have any meaning.
    Of the tiny fraction that is measured with enough stations to have any meaning, the stations themselves are of poor quality.

    Anyone who declares that with this data, we can tell what the earth’s temperature is, within 2 degrees, much less a tenth of a degree, is delusioanl.

  93. Steve Sadlov
    Posted May 29, 2007 at 10:33 AM | Permalink

    Marysville is now a de facto suburb of Sacramento. The route 65 corridor is one of the fastest growing areas in the state if not the US. It’s right up there with the famous area between I-80 and US-50 to the NE of Sacramento, which was one of David Brooks’ archetypal new exurbs in his book a couple years ago “On Paradise Drive.” In the case of both the 65 corridor as well as Brooks’ archetypal (aptly named) Placer County, a widely sprawling suburbarn landscape with nodes of urban concentration has been superimposed on the rural infrastructure. It’s pretty amazing.

  94. Steve Sadlov
    Posted May 29, 2007 at 12:27 PM | Permalink

    Lincoln, CA – for decades since its Victorian Era founding date, a wide spot along the road from Sacramento to Yuba City – Marysville. Now, it’s the land of Del Webb and Starbucks – Montessori schools and endless Volvos can’t be too far behind:

    http://www.citytowninfo.com/places/california/lincoln

    A “rural” location …..

  95. John Nicklin
    Posted Jun 1, 2007 at 9:39 AM | Permalink

    92: Like you I can’t see how we can rely on these stations to provide any meaningful data. The scary thing is that the US stations are probably amoung the best in the world compared to many others. How this stuff can be used with any confidence is beyond me. It would be like finding out that Lonny T regularly urinates down his boreholes but still claims hundredth of a degree accuracy.

  96. Posted Jun 1, 2007 at 4:43 PM | Permalink

    Anthony Watts is doing more CSI, and asked me to post these pictures for him.

    Anthony writes:

    First site is the Lodi Municipal Service Center, 38.11619N 121.29003W.

    Its like “worlds collide” where solar power to offset GHG’s meets the reason for global warming in the first place.

    The second site is at the Lodi Fire Station #2, 38.10586N 121.28815W.

    I’m not sure which one is USHCN yet as will have to call NWS Sac, but they are both equally bad. To the right of the picture on north side of fire station is the fireman’s BBQ pit wit 3 grills…and like many others, this weather station comes with parking and trash service…

    I will have complete set of pictures later

    The two stations are about 1/2 mile apart, so lat/long is useless to tell which is USHCN.

    It gets curiouser and curiouser.

  97. Nicholas
    Posted Jun 3, 2007 at 8:20 PM | Permalink

    Based on a short discussion in another thread, here is what I think could be a very interesting study based on this new work.

    Once a large number of temperature stations are located and documented, I think a “test” should be drawn up. I saw someone posted guidelines for setting up a proper temperature station on one of these threads. I would create a score sheet. Assign a score to each requirement for a properly located and set-up temperature station. The scores add up to some number like 100 or 1000. For each documented station, determine whether it meets or fails the criteria, and sum the scores for those which it meets. This gives you an objective overall score for each station, which can be entered into a database. Hopefully these scores would not vary much depending on who is doing the scoring if the criteria are well enough defined, and thus can easily be reproduced and agreed upon.

    Similarly, come up with criteria for what defines a rural station – number and size of nearby structures, nearby population density, area of land use changes, etc. Score each station on this test and enter the score into the database also.

    Thirdly, I think there should be a homogeneity/data quality test. It could include scores for such things as the number of times a station has moved, how far it moved, how many years contain missing data, etc. Once temperature data and meta-data is available, score each station on data quality.

    Ideally this would all be done without actually looking at any of the temperature series. Since these would ideally be objective and deterministic measurements, once the tests are created, anybody can apply them to a station.

    Finally, download temperature series for all examined stations. Discard all of those with a station quality score below a certain threshold – ideally 100%. Discard any of these remaining series with a data quality score below a certain threshold. Sort them into a few bins based on their rural/urban score. Then average the series in each bin. Plot them, and compare the trends.

    I think the results could be fascinating, and importantly, totally replicable. I’d especially like to see the trend of the average of all rural stations with high scores on station quality and data quality, and compare that to the urban trend.

  98. MarkW
    Posted Jun 4, 2007 at 5:21 AM | Permalink

    There may need to be a third team. I don’t know how you could do metadata adjustments without looking at the data from both the station in question, and nearby stations.

    If you are going to exclude those stations that need metadata adjustments, I don’t think you will have many stations left.
    Even if the station hasn’t been moved, it’s been changed once or twice over the years.

  99. Steve Sadlov
    Posted Jun 4, 2007 at 11:00 AM | Permalink

    RE: #65 – Every one of those stations is in an area which has experienced massive growth since WW2. In some of them, it has literally gone from rural to heavy suburban during that time period. Even in the obviously urban ones like SF, Oakland and Berkeley, the density has increased and in SF, the City went from mostly low rise buildings everywhere, to a significant high rise financial district in the down town area with much mid rise infill in other areas, much of this since the 1960s. Another significant thing to note affecting the South and East Bay suburbs has been the advent of massive light industrial/R&D/business parks, with 2 – 3 story glass buildings containing data centers, PCs in every office and in some cases, semi conductor processing lines (you want to talk about electrical current!).

  100. John F. Pittman
    Posted Jun 30, 2007 at 7:39 PM | Permalink

    Thomas Jefferson and Special Awards
    ‘€¢ Holm Awards Year Length of Service Awards
    ‘€¢ 75-100 Year Institutional Length of Service Awards
    ‘€¢ 50 Year Length of Service Awards
    ‘€¢ 45 Year Length of Service Awards
    ‘€¢ 40 Year Length of Service Awards
    ‘€¢ 35 Year Length of Service Awards
    ‘€¢ 30 Year Length of Service Awards
    ‘€¢ 25 Year Length of Service Awards
    ‘€¢ 20 Year Length of Service Awards
    ‘€¢ 15 Year Length of Service Awards
    ‘€¢ 5-10 Year Length of Service Awards.

    The above awards are linked to the WWW and are part of the public domain.
    Senator Graham, and Jean Carter Johnson, FOIA officer of NOAA, please, note that NOAA has refused to answer FOIA requests to several FOIA submitters (legitimate requests as far as I can determine) based on the fact that NOAA sites and volunteer observers are claimed to be private even though their names and accomplishments have been made public (see above, details are available on climateaudit.org). Senator Graham, the emails to and from Jean.Carter.Johnson@noaa.gov, and other NOAA officials are easily obtainable by you or your staff. Up until approximately a week ago, these names, locations could be accessed by the internet. Positions and names were directly obtainable, accessible, and in the public domain, and available. Now there is the claim that they are private and this information has been removed from the web. There is no such thing as” a little bit pregnant”. Senator Graham as far as I can determine US law and NOAA FOIA guidelines have been violated. Could you and your office help me? Senator Graham, for your information, on a blog, I will ask that all that have been refused cooperation as required by law to contact you. Please, I hope you and your staff don’t mind a citizen of SC for all of his life of almost 54 years asks for this to be resolved according to US law and precedence.

  101. Steve McIntyre
    Posted Jul 31, 2007 at 10:11 AM | Permalink

    Eli Rabett objected to the following statement:

    But let’s say that they got all their adjustments exactly right. What does that say about the quantum of UHI? Here’s a town of 12,000 which qualifies as “rural” in all the UHI studies.

    Eli objected that Marysville was not “unlit” in the GISS data set and thus did not contribute to GISS trends.

    The full quote was:

    But let’s say that they got all their adjustments exactly right. What does that say about the quantum of UHI? Here’s a town of 12,000 which qualifies as “rural” in all the UHI studies. The non-climatic effect here is at least 3 deg C. In this case, we can estimate it because there’s a less bad station nearby. What does one do in China or Indonesia where nearly all the stations seem to be in large cities (even if they started out small, they’re large now)?

    The “UHI studies” that I had in mind (and had been discussing) were articles like Jones et al 1990, which defines Chinese “rural” stations as having populations under 100,000. Karl et al (J Clim 1988) said in regard to the US system:

    Due to the large number of stations located in sparsely populated areas [over 85% (70% of all stations had a 1980 population of less than 25 000 (10 000)], the impact of urbanization is not large in relation to decadal changes of temperature in the United States.

    My point here was that, if the GISS adjustment is right, then the required adjustment for Marysville in the GHCN data set used by NOAA, CRU and thus IPCC was up to a couple of deg C – despite claims that the UHI effect was no greater than 0.1 deg C per century. Or if Jones is right, then GISS is over-correcting. The GISS adjustments are not used in CRU or GHCN or USHCN.

  102. Steve McIntyre
    Posted Jul 31, 2007 at 10:58 AM | Permalink

    BTW the USHCN population for Marysville population in 1980 was 9,898 in 1980 (thus meeting the 10 000 rural benchmark of Karl et al 1988) referred to above. http://cdiac.ornl.gov/ftp/ushcn_monthly/metrof_orig

    So Marysville is a “rural” station in the rural-urban comparison of the “UHI study” Karl et al 1988, cited in many subsequent studies.

  103. SteveSadlov
    Posted Jul 31, 2007 at 12:39 PM | Permalink

    RE: #104 – And as Mr. Watt can rightly attest, since 1980, Marysville (and most other places along route 99) have exploded in population. The explosive growth of that exurb made famous in Brook’s “On Paradise Drive” – Placer County – is mimiced by the other communities just to the north, such as Marysville.

  104. steven mosher
    Posted Sep 5, 2007 at 4:04 PM | Permalink

    RE 91. UC

    Thanks for the pointer.

    Now, I will say That I enjoy whipping out the 60 site claim.

    I have transformed this into the 6 site claim.

    60 Sites for the NH.

    The US is 2% of the global land mass.

    The US is “approx” 4% of the NH land mass.

    Round up to 5% using Hansen Rounding.

    Round up to 10% because I hate to be wrong.

    So, if the NH temp trend can be captured by 60 stations, Then the US can be captured
    by 6.

    Six for Gavin.

  105. Sam Urbinto
    Posted Sep 5, 2007 at 4:32 PM | Permalink

    According to the site, there are already many more CRN stations than 6 in operation….

    That should be interesting once they gather the data and start using it.

  106. steven mosher
    Posted Sep 5, 2007 at 8:07 PM | Permalink

    re 108.

    You can get the data online going back to 2002.

    SteveMC has shown how to scrape it.

    BUT, todays focus is reverse engineering H87, H99, H2001.

    No cmplaints here

  107. Willis Eschenbach
    Posted Sep 5, 2007 at 8:09 PM | Permalink

    Bender, you say:

    Place your guesstimates now. I will collate them along with the A in AGW guesses.

    Not mine, but the guess of the CET (Table 6 in Parker et al 1992): zero up until ~1980, a sudden jump to 0.1°C in ~1980, no change until ~1995, a sudden jump to 0.2°C in ~1995, no change since then …

    This works out to about 1°C/century … but only since 1980. Go figure …

    w.

  108. Posted Sep 5, 2007 at 11:47 PM | Permalink

    #107

    Now, I will say That I enjoy whipping out the 60 site claim.

    And the original 60 site claim from Shen94 is (my emph)

    In spite of the limitations just alluded to we feel that the magnitude of the sampling error is probably small compared to the other errors in the budget once the number of well-distributed stations is above about 60 and the optimal weighting is applied.

    This was based on UK gridded data (Jones). Note that they had some problems with Hansen data,

    We note of course that the correlation to the U.K. data at fine scale is not necessarily the same as with perfect data. In fact, we correlated the coefficients of the zeroth harmonics in T11 truncation derived from the U.K. data with the Hansen-Lebedeff data and found the correlation coefficient to be only 0.93! This is presumably because the two groups used different means of removing biases.

  109. steven mosher
    Posted Sep 6, 2007 at 7:31 AM | Permalink

    Shen94

    Might be a paper SteveMc is interested in after the crosswrd puzzles are handled

  110. JerryB
    Posted Sep 10, 2007 at 3:28 PM | Permalink

    In the course of this thread the question arose “what USHCN adjustments are
    included in GHCN the adjusted mean file”? Previously, I had thought that the
    USHCN FILNET adjustments were included, but upon looking at Marysville station
    data, I concluded that SHAP, but not FILNET, adjustments were included.

    Today, pursuing a notion of a temporary substitute for the GISS special USHCN
    file with SHAP, but not FILNET, adjustments, I must conclude that the GHCN
    adjusted mean file sometimes include SHAP, some times FILNET, sometimes both,
    and sometimes something other than either.

    More anon.

  111. steven mosher
    Posted Sep 10, 2007 at 4:24 PM | Permalink

    JerryB.

    Since you are one of the resident gods of climate data… it might be cool to
    do a diagram of sorts… starting at the sources… elabrating on the process steps
    and then detailing the outputs.

  112. JerryB
    Posted Sep 10, 2007 at 4:35 PM | Permalink

    steve,

    Bite your tongue, or fingertips, or something. 🙂

    I expect to write to Peterson and Vose to inquire what USHCN adjustments are
    “supposed” to be included in the GHCN adjusted mean file. I expect to mention
    examples of GHCN adjusted numbers which do not match any USHCN numbers to within
    0.1 C.

    We shall see what we shall see when we see it.

  113. JerryB
    Posted Sep 11, 2007 at 7:32 AM | Permalink

    A sample of GHCN mean adjusted data sometimes matching USHCN SHAP, sometimes
    USHCN FILNET, sometimes both, and somtimes neither:

    pick your adjustment

5 Trackbacks

  1. […] http://www.climateaudit.org/?p=1603#more-1603   […]

  2. […] ISO-2000 Weather Stations from Jones and Hansen Quality Control – Jones and Hansen Style Some Northern California Station […]

  3. […] famous in Brook’s “On Paradise Drive” – Placer County – is … article continues at SteveSadlov brought to you by FASHION and […]

  4. […] of investigating alteration of Orland data by Steve McIntyre, with his customary thoroughness, here and here. Figure 2. Orland USHCN raw data/GISS homogenized data blink comparison. Animation […]

  5. […] station, discussed at the outset of surfacestations.org, discussed at WUWT here and CA here in early […]