Historical Station Distribution

In his comment to How much Estimation is too much Estimation?, Anthony Watts suggested I create a scatter plot showing station distribution with latitude/longitude. It turned out not to be the ordeal I thought it might be, so I have posted some of the results in this thread. I started with 1885 and created a plot every 20 years, ending in 2005. I deliberately ended with 2005 because this is the final year in the GHCN record prior to the US station die-off of 2006.

Every dot on a plot represents a station, not a scribal record. Stations may be comprised of multiple records. A blue dot represents a station with an annual average that was fully calculated from existing monthly averages. A red dot represents a station that had missing monthly averages for that year, so the annual average had to be estimated. Stations that had insufficient data to estimate an annual average are not shown.

In the case where multiple scribal records exist for a station in the given year, I assigned a blue dot if all records were fully calculated from existing averages, a red dot if at least one record was estimated, and no dot if none of the records could produce an estimate. I believe this errs in the direction of assigning more blue dots than is deserved. Hansen’s bias method mathematically forces estimation to occur during the period of scribal record overlap.

The first plot shows coverage in 1885, five years into the GHCN record.

sc1885.gif

1905 shows improved coverage across the continental US, Japan and parts of Australia. A few stations have appeared in Africa.

sc1905.gif

1925 shows increased density in the western US, southern Canada, and the coast of Australia.

sc1925.gif

At the end of WWII, not a lot of change is noticeable other than improved coverage in Africa and South America as well as central China and Siberia.

sc1945.gif

In 1965 we see considerable increases inChina, parts of Europe, Turkey, Africa and South America.

sc1965.gif

A decline in quality seems to be apparent in 1985, as many more stations show as red, indicating their averages are estimated due to missing monthly data.

sc1985.gif

A huge drop in stations is visible in the 2005 plot, notably Australia, China, and Canada. 2005 was the warmest year in over a century. Not surprising, as the Earth hadn’t seen station coverage like that in over a century.

sc2005.gif

The final plot illustrates the world-wide station coverage used to tell us “2006 Was Earth’s Fifth Warmest Year“.

sc2006.gif


Update (Steve Mc):
USHCN station information gets added into GHCN with a lag of almost a year (noted in comments below). Jerry Brennan, who’s followed this for some time, reports the following update schedule in the past:

USHCN station data for the year 2002 were published in the USHCN website by May 2003, and added to GHCN between November 8, and December 10, 2003.

USHCN station data for the year 2003 were added to GHCN between April 10, and May 6, 2004, and published in the USHCN website by January 2005.

USHCN station data for the years 2004, 2005, and the first three months of 2006, were added to GHCN between August 13, and September 11, 2006, and published in the USHCN website, with data through October 2006 by March 2007. The additional months of data were not added to GHCN.

USHCN station data through May 2007 were published in the USHCN website in October 2007, but the “new” data have not been added to GHCN as of this date.

By the end of February of each year, GHCN will usually have data for the full previous year from only 120 (non USHCN) stations in the 48 contiguous USA states.

I might add that there are two locations for USHCN data, one at NOAA and one at CDIAC. The NOAA version is more updated than the CDIAC version – perhaps there are other differences as well. I personally confirmed that the NOAA version (Oct 2007 edition) is updated to May 2007 for most USHCN stations. There was a USHCN update in May or June 2007 which updated to late 2006 for most USHCN stations.

I personally confirmed that the most recent GHCN version (ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2 Feb 2008 edition), as noted by Jerry, contains USHCN updates only to March 2006 or so. Thus, GHCN is two USHCN updates behind at present (May 2007, Oct 2007) and its USHCN version is at least 14 months stale relative to what it could be using.

114 Comments

  1. Gary
    Posted Feb 10, 2008 at 3:03 PM | Permalink

    Nice job, John. What are the details on how you made the maps? Not that I’m asking you to do it, but a time lapse movie showing the stations winking on and off would add the time dimension in a revealing way. Somebody who visits here could do it, I’m sure. The rapid decline in the 90s would really be apparent.

  2. Tony Edwards
    Posted Feb 10, 2008 at 3:06 PM | Permalink

    Wow! That is shocking! Worse than I had thought, even though there has been a lot to cause doubts to exist. How can the “experts” really expect anyone to trust the “global average temperature”?

  3. bmcburney
    Posted Feb 10, 2008 at 3:13 PM | Permalink

    What happened between 2005 and 2006 to account for the
    large drop in stations in the USA?

  4. steven mosher
    Posted Feb 10, 2008 at 3:22 PM | Permalink

    losing spatial coverage and temporal coverage..

  5. steven mosher
    Posted Feb 10, 2008 at 3:22 PM | Permalink

    losing spatial coverage and temporal coverage..

  6. Larry
    Posted Feb 10, 2008 at 3:30 PM | Permalink

    So by Rabbit’s logic, 20-century global warming is a local phenomenon limited to the continental US.

  7. Yancey Ward
    Posted Feb 10, 2008 at 3:30 PM | Permalink

    The drop from 2005 to 2006 has to be a lack data reporting. There is no other explanation that makes sense

  8. Bernie
    Posted Feb 10, 2008 at 3:37 PM | Permalink

    The video idea would be very helpful. I remembered this one from Statistics Canada. The 50 year census data provides one example.

  9. Francois Ouellette
    Posted Feb 10, 2008 at 3:42 PM | Permalink

    Technical question: how do they take into account the non-uniform distribution of the stations when calculating the “global” mean temperature? Regions like the Arctic, Siberia, large parts of South America and Africa have little or no coverage. Surely this introduces an error in the global mean. What is the estimate for this error?

  10. Francois Ouellette
    Posted Feb 10, 2008 at 3:48 PM | Permalink

    Follow up to my post #9

    If most of the warming has occured in the Arctic (including Greenland) and Siberia, as the data seem to show, surely the lack of stations there introduces a large uncertainty into the warming trend. In other words, if the regions where the trend is the strongest have the poorest coverage, then the error on the trend is larger than if the trend was uniformly distributed across all stations.

  11. Bernie
    Posted Feb 10, 2008 at 3:56 PM | Permalink

    Assuming that the satellite data now provides more complete coverage, is there a way of determining how well the satellite data may be calibrated against the ground station data? Is there enough overlap and stability in the satellite methods with sufficient numbers of ground stations?

  12. steven mosher
    Posted Feb 10, 2008 at 4:10 PM | Permalink

    John, Do you have the List of stations for the US in 2005…

    I’m wondering how many of those stations anthony has surveyed .

  13. Posted Feb 10, 2008 at 4:18 PM | Permalink

    Francois–
    The averages are supposed to be surface area weighted. That must mean that in areas with low thermometer density, each thermometer measurement contributes a lot to the average. To the extent that that thermometer is inaccurate or temperature varies spacially, this can add quite a bit of uncertainty to the GMST.

    In places like the US which, until recently was entirely covered with dots, each individual thermometer would contribute less to the average. In these areas, uncertainties in individual thermometer measurements are likely to cancel, and uncertainties due to real spacial variations will be smaller.

    Ideally, to minimize error, you have thermometers distributed more or less evenly by area. ( Though, if there are large regions that are climatically similar, you could deviate from this a bit.)

    It’s interesting to see Canada never quite jumped into engaged in the temperature measurement frenzy that appeared well under way in the US by 1885. I can’t help wonder what caused us to sprinkle thermometers everywhere?

    I’m squinting– it looks like Guatamala city had a station in 1905; that general area had many in 1965 (around when my family lived there.)

    Squinting again– are there no stations in Greenland?

  14. John Goetz
    Posted Feb 10, 2008 at 4:20 PM | Permalink

    #12 Steve

    Yes, I have lists now of the entire world, by year, and whether or not the station reported an average and if it was estimated. I also have summary lists by continent / by month, and can create them easily enough by country if I need to.

    #1 Gary

    I have considered creating an AVI or some other movie file. If I have time I will do it. The movie is easy enough. It is generating each graph that can be tedious.

  15. Bernie
    Posted Feb 10, 2008 at 4:32 PM | Permalink

    John:
    Can you change the color of the dots to make them stand out from the coastline. I would hate for Lucia to hurt her eyes! :)

    Lucia, there are stations in Greenland but they are on the coast and are masked in the coastline. You can see them if you look at 1965.

  16. Mike B
    Posted Feb 10, 2008 at 4:36 PM | Permalink

    It’s interesting to see Canada never quite jumped into engaged in the temperature measurement frenzy that appeared well under way in the US by 1885. I can’t help wonder what caused us to sprinkle thermometers everywhere?

    Squinting again– are there no stations in Greenland?

    My recollection that much of the network of weather stations was associated with agriculture. Hence the more temperate regions of Canada have coverage comparable to the US lower 48, but once you get outside of the areas where there is substantial farming, there isn’t much need for the coverage.

    Although John’s map doesn’t seem to have any Greenland Stations, GISS has several. Here is one .

  17. John Creighton
    Posted Feb 10, 2008 at 4:43 PM | Permalink

    It’s unfortunate that between 1985 and 2005 so many thermometers in Canada disappeared.

  18. Francois Ouellette
    Posted Feb 10, 2008 at 5:03 PM | Permalink

    #13 Thanks Lucia,

    I understand the idea of weigthing by surface area. The point I was trying to make is that if you’re looking for a trend (up or down), and that the trend is not the same everywhere, the extent of the coverage will affect the error on the trend differently for different locations.

    Say there’s no warming in the tropics, but you’ve got one station every kilometer, but there is a presumably strong trend in the Arctic, where you’ve only got a handful of stations. All those stations you have in the tropic are basically useless to determine what the trend is. You might as well not use them. So if you only use the Arctic stations, you will find that your uncertainty on the trend is mighty large. So having a large number of stations globally can give you a false sense of confidence in the actual uncertainty of the trend itself.

    For example, some data seem to show that Greenland was as warm in the 1930’s as it is today. But how are we to know: there were precious few stations, if any at all, during that period.

    I’m not too concerned about the data after, say, 1960. But I’d say the error on the first half of the 20th century may be larger than a simple estimate based on the sheer number of stations would give. I’m not quite sure how you would take that into account in making your error estimate. That’s basically the question I am asking to this board!

    This is important, not so much because the 1930’s, or the MWP or whatever other period may have been as warm as today. But if you’re going to calibrate your GCM based on the past, and our knowledge of the forcings (solar, volcanic, GHG), you need to take into account that uncertainty. Larger “natural” variability may mean that solar forcing, for example, has a larger effect than currently estimated.

  19. Raven
    Posted Feb 10, 2008 at 5:08 PM | Permalink

    lucia says:

    The averages are supposed to be surface area weighted. That must mean that in areas with low thermometer density, each thermometer measurement contributes a lot to the average. To the extent that that thermometer is inaccurate or temperature varies spacially, this can add quite a bit of uncertainty to the GMST.blockquote>The gridded data is fixed 5×5 lat/long blocks. This implies that a much smaller surface area near the poles would have a more significant impact on the global average. Do you know if they really use surface area or it the GMT a simple average of the gridded dataset?

  20. Raven
    Posted Feb 10, 2008 at 5:12 PM | Permalink

    lucia says:

    The averages are supposed to be surface area weighted. That must mean that in areas with low thermometer density, each thermometer measurement contributes a lot to the average. To the extent that that thermometer is inaccurate or temperature varies spacially, this can add quite a bit of uncertainty to the GMST.

    The gridded data is fixed 5×5 lat/long blocks. This implies that a much smaller surface area near the poles would have a more significant impact on the global average. Do you know if they really use surface area or it the GMT a simple average of the gridded dataset?

    Steve: They say that they area-weight and I have no reason to think otherwise.

  21. Posted Feb 10, 2008 at 5:35 PM | Permalink

    Raven–
    Well,I’m not certain. But it isn’t necessarily either/or. They could easily attribute an average temperature to each block, and then, later when finding the average for the planet, weight by the area for the block.

    I’d be a bit surprised if computations of GMST used temperatures weighting them like a distorted flat map. That makes no sense physically, and doesn’t strike me as the sort of thing either GISS or Hadley would be likely to do. (I guess we could break down and read the papers? There are links at each entities sites.)

  22. Larry
    Posted Feb 10, 2008 at 5:43 PM | Permalink

    13, until fairly recently, someone had to live there to have a weather station. That’s why most of Canada and all of Greenland are empty. Even now, if it’s unmanned, a source of power and a communication link of some sort is necessary. That can be done way out in the boonies with solar panels and a satellite uplink, but that’s a bunch of money.

  23. Steve McIntyre
    Posted Feb 10, 2008 at 5:49 PM | Permalink

    The GHCN station collation (used in turn by CRU and NASA-GISS) has a delay in incorporating USHCN network. It looks like the 2006 list lacks the USHCN stations, but they will get into the 2006 list at some point fairly soon.

    Also and I’ve mentioned this many times, GHCN collated a lot of historical data in the early 1990s that it promised to update “irregularly”. For the most part, the “irregular update” hasn’t occurred since then and subsequent data is mostly airport data. It’s my surmise that data is actually available in local services for many stations that “disappear” in the early 1990s and that the reduction shown here mainly reflects the fact that GHCN collation has been very lackadaisical. There may also be station closures, but I know for a fact that some Russian and Chinese stations unavailable after the GHCN last date in the early 1990s continued to functioun.

  24. Andrew
    Posted Feb 10, 2008 at 5:50 PM | Permalink

    This animation may be of interest (a bit dramatic):

    http://climate.geog.udel.edu/~climate/html_pages/Ghcn2_images/air_loc.mpg

  25. Posted Feb 10, 2008 at 5:54 PM | Permalink

    Francois– Yes. It’s the large areas with no stations that are a problem with respect to uncertainty. (That is, unless we can assume temperature vary much less spacially in those regions rather than other regions for some reason. Can we? I think there are spacial temperature variations in northern Canada. No reason to think otherwise. )

  26. Raven
    Posted Feb 10, 2008 at 6:07 PM | Permalink

    Lucia says:

    I guess we could break down and read the papers?

    From http://data.giss.nasa.gov/gistemp/sources/gistemp.html

    “A grid of 8000 grid boxes of equal area is used. Time series are changed to series of anomalies. For each grid box, the stations within that grid box and also any station within 1200km of the center of that box are combined using the reference station method.

    A similar method is also used to find a series of anomalies for 80 regions consisting of 100 boxes from the series for those boxes, and again to find the series for 6 latitudinal zones from those regional series, and finally to find the hemispheric and global series from the zonal series.”

    It appears they measure area with lat/long and attempt to compensate for the different physical areas by computing the zonal averages. However, they effectively negate this step by using the zonal means to calculate the global means. This implies the polar data makes up 1/3 of the GMST if they calculate a simple average of the zones.

  27. Posted Feb 10, 2008 at 6:42 PM | Permalink

    Re #26

    Here are the GISS anomaly values for 2005, as a test case:

    Equ to 24N: +0.53C
    24N to 44N: +0.50C
    44N to 64N: +1.19C
    64N to 90N: +2.17C

    This data, adjusted for the area of the NH which each band represents, should give the GISS value for the Northern Hemisphere (+0.82C).

  28. Raven
    Posted Feb 10, 2008 at 6:49 PM | Permalink

    I have been trying to determine that the GISS definition of grid boxes with “equal area” mean and grid boxes with equal number of degrees on each side rather than an equal number of meters.

    I found this that seems to say that they do mean equal, as in distance:

    http://badc.nerc.ac.uk/data/gedex/detail/giss_tmp.det

    That link has break down of station coverage. I did some calculations to estimate the effect that a station has on the global average using this data:

    64.2 – 90.0 N – 1.12
    44.4 – 64.2 N – 0.43
    23.6 – 44.4 N – 0.60
    0.0 – 23.6 N – 0.90
    0.0 – 23.6 S – 1.51
    23.6 – 44.4 S – 1.65
    44.4 – 64.2 S – 6.78
    64.2 – 90.0 S – 5.46

    I calculated this number by diving the % grid boxes per band by the % stations per band

  29. Posted Feb 10, 2008 at 7:06 PM | Permalink

    I can’t help wonder what caused us to sprinkle thermometers everywhere?

    Initially, war needs; the ‘wx services’ of the time were originally in the ‘Signal Service’ (think ‘telegraph services’ back then) branch of the US Army with subsequent reporting changes since then:

    … starting in 1873, reports on river conditions and flood warnings that same year, and frost warnings for Louisiana sugar growers in 1880, with subsequent reports targeted at growers of cotton, rice, tobacco, corn, and wheat. By 1883, the Signal Service budget was close to $1 million, considerably exceeding the sum that Joseph Henry had recommended.

    Over the years, the Signal Service proved to be something of a bureaucratic rolling stone: it was upgraded to the Signal Corps in 1880; in 1891, after a good deal of wrangling between civilian weather personnel and their military superiors, its ties with the army were severed and it was handed over to the Department of Agriculture as the U.S. Weather Bureau; then in 1940 it was passed on to the Department of Commerce, where it has remained ever since, under the jurisdiction of the Environmental Science Services Administration (ESSA) from 1965 to 1970 and since then under the National Oceanic and Atmospheric Administration (NOAA). (Ref. 13)

    David Laskin’s book, “Braving the elements: the stormy history of American weather” pg 143

  30. Mike B
    Posted Feb 10, 2008 at 7:06 PM | Permalink

    It appears they measure area with lat/long and attempt to compensate for the different physical areas by computing the zonal averages. However, they effectively negate this step by using the zonal means to calculate the global means. This implies the polar data makes up 1/3 of the GMST if they calculate a simple average of the zones.

    They don’t compute a simple average, and I haven’t been able to come up with a weighting scheme that gets me from the zonal averages to the global averages.

    Here’s the last seven years:

    24N 24S 90S 64N 44N 24N EQU 24S 44S 64S 90S
    Year Glob NHem SHem -90N -24N -24S -90N -64N -44N -24N -EQU -24S -44S -64S Year
    2001 48 59 36 75 37 36 101 79 64 36 38 43 21 43 2001
    2002 56 66 46 79 51 40 123 95 53 48 55 37 32 74 2002
    2003 55 69 41 81 54 31 144 101 46 51 56 31 24 48 2003
    2004 49 63 34 72 45 30 75 89 60 49 41 35 23 26 2004
    2005 62 82 43 100 55 33 217 119 50 53 57 37 19 53 2005
    2006 54 74 34 92 46 27 172 103 58 47 44 39 14 19 2006
    2007 57 80 34 109 35 34 221 129 60 36 34 36 6 96 2007

    Maybe somebody else will have better luck.

  31. Tim Ball
    Posted Feb 10, 2008 at 7:12 PM | Permalink

    Hansen says the adjustments he quietly made because of Steve’s discovery of an error in the post-2000 data was restricted to the US. These maps indicate the extent to which US stations dominate the global total. It is hard to imagine the correction didn’t have some effect on the global number. We also need to know how many stations Hansen uses for his global annual average are from outside the US.

    The other issue to consider is that in 1885 most stations were close to big cities. It is likely they would have experienced the most amount of Urban Heat Island Effect, which raises questions about which stations P.D Jones used to create his 130+ year claim of an unnatural rise in the annual average temperature.

  32. steven mosher
    Posted Feb 10, 2008 at 7:21 PM | Permalink

    RE 31. Dr. Ball, Hansens method wont Overweight the US. He divides the world into equal area chucks
    and then estimates the chuck based on the stations within the chuck. So, densely sampled areas get
    1 number with a small varience and less dense areas get a number with a bigger varience.

    Still, the lack of any semblance of uniformity either in the spatial dimension or the temporal dimension
    Should raise some legitamate doubt.

  33. Posted Feb 10, 2008 at 7:22 PM | Permalink

    Yes. It’s the large areas with no stations that are a problem with respect to uncertainty. (That is, unless we can assume temperature vary much less spacially in those regions rather than other regions for some reason. Can we? …

    Grant me some literary licence in the following: With ‘cold air’ being produced in the polar regions and warm, humid air in the tropics (we will ignore oceans for the moemnt) under the influence of the global ‘three-cell GCM’ (GCM as in General Ciculation Model as the term is used introduced to those studying meteorology) with a sufficiently long average (multi-year averaging) I as an engineer would be somewhat content that we would be measuring climate scale temperatures (and any changes) of that region and not simply weather variability.

  34. steven mosher
    Posted Feb 10, 2008 at 7:26 PM | Permalink

    re 28. Raven go read Hansen 87 ( i think) where he describes the equal area tiling of the sphere.

    Within each equal area tile he makes further tilings. I think its hansen 87. if you cant find it
    or I’m wrong, throw a rock at me, and I’ll search my stack of junk

  35. Posted Feb 10, 2008 at 7:28 PM | Permalink

    RE #30 For 2005 Northern Hemisphere I get –

    Eq to 24N: +0.53C for 40.7% of NH area
    24N to 44N; +0.50C for 28.8%
    44N to 64N: +1.19C for 20.4%
    64N to 90N: +2.17C for 10.1%

    which comes out to the +0.82C reported by GISS for the Northern Hemisphere

  36. John Creighton
    Posted Feb 10, 2008 at 7:29 PM | Permalink

    #33 Not sure if it is on topic but from what I gather you are suggesting the use of time averages to make for the lack of ensemble run. I think this is not a hopeless suggestion but it clearly has its problems.

  37. steven mosher
    Posted Feb 10, 2008 at 7:34 PM | Permalink

    RE 25. Lucia. There is something VERY FISHY about the 1200KM figure that hansen uses
    to “estimate” or adjust stations.

    In Hansen 87 ( I’ll check if I’m right) he looked at Correlations between stations.

    At 1200KM he found a correlation of .6 in the NH .5 in the SH.

    He tested a relatively small number of stations.

    I find this utterly insane. Especially given the teleconnection argument where A grid
    square on one side of the planet is correlated with a grid square on the other side.

    more later

  38. Posted Feb 10, 2008 at 7:46 PM | Permalink

    steven– he found a correlation of 0.6 between what and what?

  39. Steve McIntyre
    Posted Feb 10, 2008 at 7:50 PM | Permalink

    Not all the methods in Hansen are the same as those in CRU. Be careful to specify.

  40. John Creighton
    Posted Feb 10, 2008 at 7:51 PM | Permalink

    What is CRU?

  41. Dasher
    Posted Feb 10, 2008 at 7:56 PM | Permalink

    #24

    13, until fairly recently, someone had to live there to have a weather station. That’s why most of Canada and all of Greenland are empty. Even now, if it’s unmanned, a source of power and a communication link of some sort is necessary. That can be done way out in the boonies with solar panels and a satellite uplink, but that’s a bunch of money.

    Of course in the arctic and antarctic much of the winter months are without sunlight to power a solar powered device.

  42. John Goetz
    Posted Feb 10, 2008 at 7:56 PM | Permalink

    From HL87:

    3. SPATIAL AVERAGING: BIAS METHOD
    Our principal objective is to estimate the temperature change of large regions. We would like to incorporate the information from all of the relevant available station records. The essence of the method which we use is shown schematically in Figure 5 for two nearby stations, for which we want the best estimate of the temperature change in their mutual locale. We calculate the mean of both records for the period in common, and adjust the entire second record {T}_{2} by the difference (bias) \delta T . The mean of the resulting temperature records is the estimated temperature change as a function of time. The zero point of the temperature scale is arbitrary.

    A principal advantage of this method is that it uses the full period of common record in calculating the bias \delta T between the two stations. Determination of \delta T is the essence of the problem of estimating the area-average temperature change from data at local stations. A second advantage of this method is that it allows the information from all nearby stations to be used provided only that each station have a period of record in common with another of the stations. An alternative method commonly used to combine station records is to define \delta T by specifying the mean temperature of each station as zero for a specific period which had a large number of stations, for example, 1950-1980; this alternative method compares unfavorably to ours with regard to both making use of the maximum number of stations and defining the bias \delta T between stations as accurately as possible.

    A complete description of our procedure for defining large-area temperature change is as follows. We divide each of the 80 equal-area boxes of Figure 2 into a 10 by 10 array of 100 equal-area subboxes. (The east-west and north-south dimensions of a subbox are about 200 km; in the polar boxes the equal area requirement for the subboxes causes the polemost subboxes to be noticeably elongated in the north-south direction.) For each subbox we use all stations located within 1200 km of the subbox center to define the temperature change of that subbox. The N stations within 1200 km are ordered from the one with the greatest number of years of temperature record to the one with the least number. The temperature changes for the first two stations are combined, then the third with the first two, and so on, using

    \delta {T}_{n} = \bar{T}_{n} - \bar{T}_{1,n-1}
    {W}_{n} = (D - {d}_{n})/D
    {W}_{1,n}(t) = {W}_{1,n-1}(t) + {W}_{n}
    {T}_{1,n}(t) = [{W}_{1,n-1}{T}_{1,n-1} + {W}_{n}({T}_{n} - \delta {T}_{n})] / {W}_{1,n}

    for t without available {T}_{n} . T represents temperature change, t is time, and n identifies the station. {T}_{1,n}(t) is an intermediate estimate of the temperature change based on stations 1 through n; these equations are applied repeatedly until {T}_{1,N}(t) is obtained, where N is the total number of stations within 1200 km of the subbox center. Here {d}_{n} is the distance of the nth station from the subbox center, and {d}_{n} is used to calculate the weight {W}_{n} by which the nth station temperature change is weighted. {W}_{n} decreases linearly from 1 at the subbox center to 0 at a distance D, where we have taken D = 1200 km as a representative direction-independent distance over which the temperature changes exhibit strong correlation.

  43. Anthony Watts
    Posted Feb 10, 2008 at 8:12 PM | Permalink

    Thanks John for doing this, this is splendid!

    OT I’m a bit late in joining the conversation again, as I just spent most of the weekend developing confirmation that a volcano near the Ross ice shelf once thought to be extinct, is now becoming active again. I welcome any thoughts anyone might have.

    http://wattsupwiththat.wordpress.com/2008/02/10/photo-evidence-for-eruption-of-an-extinct-volcano-near-the-ross-ice-shelf/

  44. John Goetz
    Posted Feb 10, 2008 at 8:12 PM | Permalink

    #13 Lucia

    The Greenland stations are there. Any station on a coast will be hard to pick out, because the color will be hidden by the grayscale of the coastal outline. I toyed with larger station marks, but they gave the false impression of really great global coverage. I tried with different color schemes, but I am sensitive to those that are color-blind. Rest assured, though, the stations are there.

  45. John Creighton
    Posted Feb 10, 2008 at 8:22 PM | Permalink

    #45 How about some kind of contour plot to indicate station density. Given stations occupy a discrete point some kind of low pas filter would need to be used. Perhaps a Gaussian shaped low pas filter. I’m not sure how you would decide on what the best spatial width of the filter should be.

  46. Posted Feb 10, 2008 at 8:46 PM | Permalink

    Anthony: If there was an eruption there would likely be earthquake activity. Have you checked this yet?

  47. Anthony Watts
    Posted Feb 10, 2008 at 9:00 PM | Permalink

    RE47, yes lots of it, but please carry on the conversation over there rather than OT here. Just wanted to advise people here who are good at pointing out flaws so I could strengthen the presentation or go down in flames gracefully.

  48. Posted Feb 10, 2008 at 9:36 PM | Permalink

    I’ve combined John’s graphics into an animated GIF here:

    Station_coverage

  49. aurbo
    Posted Feb 10, 2008 at 9:46 PM | Permalink

    Re #41;

    CRU is the Hadley Cenre Climate Resarch Unit in East Anglia, UK.

    Between 1890 and 1940 the Weather Bureau was a part of the Dept. of Agriculture. Temperatures were important because the daily max and min temps affected crops, especially around 32°F and in the 100+°F ranges. Because of the need for detailed and frequent observations for aircraft operations, the Weather Bureau moved into the Dept of Commerce (FAA) and the interest in parameters such as temperature and quantitative precipitatiion were of low priority as compared to wind direction and speed, ceiling heights and horizontal visibilities. None of the parameters most important to agriculture and today’s climate studies was of much concern to the FAA observers. The switch from manned observations by specially trained NWS employees to automated observations, notably ASOS and the replacement of CRS platforms with ASOS and MMTS equipment occurred in the late 70s and early 80s. The reliability and precision of thermometry in the automated systems was sharply lower than the certified error range of mercury-in-class thermometers used in the CRS screens. Certified mercury-in-glass thermometers were calibated to about 0.1°C. The acceptable range of calibration for ASOS instrumentation was plus or minus 1.2°C.

    The development of the North American Radar Defense network, usually referred to as the DEW (Distant Early Warning) Line, brought manned weather observations along the Arctic Circle in Canada. Some 58 Dew Line stations were constructed during the 1954-1957 period. These were usually the only regular observations available from these remote locations. The end of the cold war saw a reduction and essentially an elimation of most of these observations. I suspect that similar intstallations in the FSU fell by the wayside as well. A great history and a map of the Dew Line stations with other informative goodies can be found here.

    With today’s observations mostly taken from airports in or close to urban areas, there are very few continuous observations taken from rural locations left to provide reliable climate data. Urban areas amount to well under 2% (some estimations put them, at 0.2%) of the Earth’s surface. It would be surprising if, as these urbam areas continue to expand, there were not upward trends in the average of currently observed land temperatures. To refer to these averages as representing global average temperatures is patently absurd.

  50. aurbo
    Posted Feb 10, 2008 at 10:01 PM | Permalink

    Re#43 & 47:

    Tony, you might want to check out this NSF article released in May 2004 re Ross Shelf volcanic acivity.

  51. Louis Hissink
    Posted Feb 11, 2008 at 4:34 AM | Permalink

    Just a quick note: Stations seem to proliferate where humans are. More humans means more heat from metabolism, (irregardless of urbanisation).

    Hence increase in global surface temperature simply means that life proliferated.

    Over to you

  52. JerryB
    Posted Feb 11, 2008 at 7:24 AM | Permalink

    Some timeline information regarding USHCN data and GHCN.

    About 1200 US stations in GHCN are USHCN stations, and their data
    are handled differently than those of other stations.

    USHCN station data for the year 2002 were published in the USHCN website
    by May 2003, and added to GHCN between November 8, and December 10, 2003.

    USHCN station data for the year 2003 were added to GHCN between April 10,
    and May 6, 2004, and published in the USHCN website by January 2005.

    USHCN station data for the years 2004, 2005, and the first three months
    of 2006, were added to GHCN between August 13, and September 11, 2006,
    and published in the USHCN website, with data through October 2006 by
    March 2007. The additional months of data were not added to GHCN.

    USHCN station data through May 2007 were published in the USHCN website
    in October 2007, but the “new” data have not been added to GHCN as of
    this date.

    By the end of February of each year, GHCN will usually have data for the
    full previous year from only 120 (non USHCN) stations in the 48
    contiguous USA states.

  53. John Goetz
    Posted Feb 11, 2008 at 7:47 AM | Permalink

    #52 JerryB
    Thanks. This confirms what Steve has been saying about the missing US data (what I like to refer to as “die off”).
    When the data for 2007 and 2006 appears, it will be fun to see if / how it changes the results for those two years here. Right now those two year’s anomalies are calculated without the USHCN stations.

  54. JerryB
    Posted Feb 11, 2008 at 8:29 AM | Permalink

    Re: 53,

    John,

    NCDC has a USHCN version 2 in the works, and the next addtion
    of USHCN data to GHCN may not occur until USHCN v 2 gets
    published. The to be published date has already passed (July 2007)
    so it’s anyone’s guess as to when it will be.

  55. PhilH
    Posted Feb 11, 2008 at 8:34 AM | Permalink

    C’mon Steve, tell us about your trip. Are you or are you not now a Ramblin’ Wreck fromm Georgia Tech?

  56. Posted Feb 11, 2008 at 8:51 AM | Permalink

    Hello all:

    Steve, congratulations on the invitation. I am looking forward to hearing how it went.

    As for the coverage issue, it is something I brought up some time ago on CA before. For example, see http://www.climateaudit.org/?p=1985#comment-131636 (I remember making other posts on the topic as well but I can’t find them).

    There is also some numbers of contributing stations and countries at http://www.unur.com/sinan/outbox/070816-us-is-only-two-percent.html

    As someone else suggested, it might be worthwhile to put together an animation of station locations from scratch. I’ll take a stab at it but I probably won’t have any time before March.

    — Sinan

  57. Mike B
    Posted Feb 11, 2008 at 9:02 AM | Permalink

    RE #30 For 2005 Northern Hemisphere I get –

    Eq to 24N: +0.53C for 40.7% of NH area
    24N to 44N; +0.50C for 28.8%
    44N to 64N: +1.19C for 20.4%
    64N to 90N: +2.17C for 10.1%

    which comes out to the +0.82C reported by GISS for the Northern Hemisphere

    Thanks David. I was missing the sine function factor there and was too distracted to track it down.

  58. Steve McIntyre
    Posted Feb 11, 2008 at 9:11 AM | Permalink

    #532. Jerry B, thanks for this. I’ve added this info to the post.

    I might add that there are two locations for USHCN data, one at NOAA and one at CDIAC. The NOAA version is more updated than the CDIAC version – perhaps there are other differences as well. I personally confirmed Jerry’s observation that the current NOAA version of USHCN (Oct 2007 edition) is updated to May 2007 for most USHCN stations. In addition, there was a USHCN update in May or June 2007 which updated to late 2006 for most USHCN stations.

    I also confirmed Jerry’s observation that the most recent GHCN version (ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2 Feb 2008 edition) contains USHCN updates only to March 2006 or so.

    Thus, GHCN is two USHCN updates behind at present (May 2007, Oct 2007) and its USHCN version is at least 14 months stale relative to what it could be using.

    I wonder what accounts for the delay which seems completely unnecessary. It takes only a few minutes to do the programming to perform the update.

  59. steven mosher
    Posted Feb 11, 2008 at 9:15 AM | Permalink

    RE 54. Jerry. V2 has employed some change point analysis, but the cited papers are behind the green
    wall. Do you have the relevant papers?

  60. JerryB
    Posted Feb 11, 2008 at 10:40 AM | Permalink

    Re: 58,

    Steve,

    I believe the delay is due to the expectation that v 2 will
    be published real soon. In the late summer of 2002 I spoke
    with someone working on USHCN v 2 at NCDC, and he then
    expressed the opinion that it would be published by early 2003.

    BTW, the two developers of GHCN v 2, Peterson and Vose, do
    not participate in the month to month updating process.

    Re: 59,

    steven,

    I do not have any of those papers.

  61. EW
    Posted Feb 11, 2008 at 11:27 AM | Permalink

    It’s my surmise that data is actually available in local services for many stations that “disappear” in the early 1990s and that the reduction shown here mainly reflects the fact that GHCN collation has been very lackadaisical.

    Indeed. When I checked Czech stations, some of these also disappeared, although they are run by Hydrometeorological Institute, properly stationed and well and alive.

  62. Posted Feb 11, 2008 at 11:32 AM | Permalink

    Great graphs, John!

    If the updating lag is part of the appearance of recent station loss, it might be useful to go back a couple of years before 2005, say 2000, when reporting is presumably as complete as it is ever going to be.

    John’s graphs (and many other such climate graphs) are equirectangular, meaning that the vertical axis is proportional to latitude. This eliminates the distance distortion of the Mercator projection in the vertical direction, but leaves a horizontal distortion proportional to secant(longitude). As a consequence, polar areas like Greenland and Antarctica still have too much area, though not as badly as Mercator.

    This area distortion can be eliminated by an appropriate compression of the vertical axes as the poles are approached. I believe making the vertical axis proportional to sin(latudide) would have this effect, since the derivative of this function is cos(latitude), the reciprocal of the secant(latitude) horizontal distortion. This projection would doubly mash shapes near the poles, but would show relative areas at a glance, without the N-S directional distortion of ovoid projections.

    Has anyone in Climatology ever used such an equiareal rectangular projection? Is it available on canned mapping routines? The standard equirectangular projection maps make a little bit of local Arctic warming look like the whole earth is on fire!

  63. aurbo
    Posted Feb 11, 2008 at 12:12 PM | Permalink

    The real question of interest in USHCNv2 is in how they’re going to handle UHI effects. Their UHI corrections as applied in part-4 of version one are totally ridiculous. See the NYC Central Park version of this which is completely untenable. Despite this, except for a brief presentation of the NYC data on this site and a few other blog sites, the silence from the AGW community has not only been been deafening, but also revealing. The numbers are so outrageous that any attempt to support these numbers would justifiably provoke ridicule. On the other hand, the raw data has been generally faithful to the actual observations. Only in the world of climate science could this be considered a novel approach.

    I understand that they have made some sort of “adjustment” to the raw data in v2 to provide a uniform UHI adjustment, but they may be spending some time on figuring out how to justify these changes when the data is released to the masses. Whatever; when the data finally is released, look for the adjusted version to become the focus of reinvigorated controversey.

  64. Gary
    Posted Feb 11, 2008 at 12:28 PM | Permalink

    #62 The Lambert Equal Area Projection avoids some spacial distortion, although it’s not rectangular.
    http://en.wikipedia.org/wiki/Lambert_azimuthal_equal-area_projection

  65. SteveSadlov
    Posted Feb 11, 2008 at 12:28 PM | Permalink

    Warm 1930s may well be global. And any appearance that is is not, may be a result of poor non US coverage.

  66. Steve McIntyre
    Posted Feb 11, 2008 at 12:30 PM | Permalink

    #62. Equi-area projections are used in many contexts. In terms of graphic presentations, you’re 100% right and anything other than equi-areal should be banned from scientific journals.

    In practical terms at my level, the only issue is the availability of routines in R (and for someone else Matlab). Doug Nychka of NCAR maintains the R-package fields. He’s very responsive to inquiries. I’ll check the manual and otherwise check with him.

  67. Posted Feb 11, 2008 at 12:42 PM | Permalink

    Steve – A very important add-on to this analysis would be to quantify the actual overlap in the GISS, NCDC and CRU data sets in terms of the raw station data that they draw from. Phil Jones told us several years ago that about 90-95% of the sites are from the same raw data, but this has not been confirmed by quantitative analysis. If the data sets are so interdependent, this means that it is misleading to present the trends from the separate analyses as independent assessments. Roger

  68. Bob McDonald
    Posted Feb 11, 2008 at 12:50 PM | Permalink

    Steve Mc,

    Long time reader, first time commenter. Kudos for all your, et al, work.

    I’ve not seen any discussion in the great debate that deals with the accuracy of the sensor that
    records the temperature. I’m not talking about an external interference, such as rooftop or parking lot locations,
    but the inherent accuracy of the gauge. Suffice it to say that thermometers have gotten better since biblical times.

    Is there any way to account for this in the modeling?

  69. steven mosher
    Posted Feb 11, 2008 at 12:50 PM | Permalink

    re 63… Ushcnv2 will do NO UHI adjustments, If I read the posting correctly.
    1. peterson ( 2003) beleives them unecessary ( cool parks thesis)
    2. Change point analysis solves everything..

    Just when you guys thought you had it all figured out, the method changes.

  70. steven mosher
    Posted Feb 11, 2008 at 12:50 PM | Permalink

    re 63… Ushcnv2 will do NO UHI adjustments, If I read the posting correctly.
    1. peterson ( 2003) beleives them unecessary ( cool parks thesis)
    2. Change point analysis solves everything..

    Just when you guys thought you had it all figured out, the method changes.

  71. Steve McIntyre
    Posted Feb 11, 2008 at 1:02 PM | Permalink

    http://cran.r-project.org/doc/packages/mapproj.pdf has a package with equi-area maps.

    Somebody should be able to locate parameters that yield a “good” map for representing areas in climate contexts.

  72. Steve McIntyre
    Posted Feb 11, 2008 at 1:07 PM | Permalink

    #67. I agree entirely that they are not “independent”. My guess right now is that GISS has more smoothing at a gridcell level than CRU3. For example, the CRU3 gridcell containing Barabinsk can be tracked to Barabinsk, but the GISS gridcell will be a complicated blend.

    The allocation of critical enterprise between land SST is totally skewed. Hardly any critical effort has been expended on SST and IMO it’s a greater priority.

  73. Posted Feb 11, 2008 at 1:11 PM | Permalink

    Re # 62, 64, 66, after a little Wikisurfing, it appears that what I am after is known as The Lambert Cylindrical Equal Area projection. The following graph is lifted from http://www.progonos.com/furuti/MapProj/Normal/ProjCyl/ProjCEA/projCEA.html:

    The Lambert Equal Area that Gary refers to in #64 is his Azimuthal, rather than Cylindrical projection.

    Another good option (but one that distorts N-S directions) is the Mollweide projection:

    A good example of the distortion of impression that is caused by the equirectangular projection are Hansen’s graphs in the 2/3 thread “Hansen in Antarctica” (#2658). His “Fig. 3.19″ copied there is somewhat like Mollweide, though I don’t think it’s quite equal-area.

  74. Tim Ball
    Posted Feb 11, 2008 at 1:33 PM | Permalink

    #32
    I appreciate the averaging approach, but the problem is what do you do when you have no station in an area? In fact, there are many areas where you have many rectangles over a large area with no stations. As I recall they used to take averages for the four corners and then produce an average for the entire area. Again as I recall there were several problems with this because many rectangles had no stations but were various proportions of land and ocean or freshwater. I also recall questions about differences in elevation. One debate I had with a modeler involved a rectangle that was half flat prairie and half high mountains and there were no stations in the mountainous half. As I have noted many times the surface data is inadequate as the basis for determining global annual average temperature let alone as the basis for models, it is made much less than adequate with the lack of above surface data.

    Discussion about the arctic, lack of data, and the equi-rectangular problem is exacerbated by the lack of data. Take a look at the map in the Arctic Climate Impact Assessment Report (ACIA)(the main source of information for the IPCC) here:

    http://www.acia.uaf.edu/PDFs/ACIA_Science_Chapters_Final/ACIA_Ch01_Final.pdf

    and on page 3 of Chapter one you will see a map depicting the lack of data. Elsewhere in the report you will see maps showing the paucity of data throughout the arctic region, however you define it.

  75. steven mosher
    Posted Feb 11, 2008 at 2:16 PM | Permalink

    RE 74. Hi Dr. Ball. I’m not sure about hadcru, but Hansens method is to average over a 1200km radius.
    ( after carving the globe into equal area tiles)

    I think Hansen 87 has the relevant graphs and analysis. In essence Hansen studied the correlation
    in temp between some 50 or so sites and found that at 1200KM the correlation was .6

    .6 being a nice number and 1200km being a nice round number ( in metric, of course in the
    english system it’s downright ugly) the science was settled. So, thou shalt average stations within
    1200km of each other, even when said 1200km crosses oceans, mountains,deserts, and polar ice caps.

    sarcasm off.

  76. MrPete
    Posted Feb 11, 2008 at 2:19 PM | Permalink

    If you modify latitude by sin(lat in radians) you get the simplest equal-area projection. No need to use fancier things for basic display.

  77. BarryW
    Posted Feb 11, 2008 at 2:20 PM | Permalink

    One possible reason for some of the reduction in stations is that the FAA had an automation program in the 1980’s that involved closing Flight Service Stations. They were mostly consolidated into 64 AFSS facilities. One of the FSS Specialist jobs was taking local weather observations, so that may have caused some of the US reduction.

  78. Larry
    Posted Feb 11, 2008 at 2:27 PM | Permalink

    Something intuitively bothers me about lumping such diverse climates into these tiles. I can’t quite put it into words what seems so wrong about, for example, lumping all of California and it’s surrounding land and ocean into one tile, but it seems awfully coarse. Can anyone either show why I’m wrong, or why I’m right?

  79. Mike B
    Posted Feb 11, 2008 at 2:28 PM | Permalink

    .6 being a nice number and 1200km being a nice round number ( in metric, of course in the
    english system it’s downright ugly) the science was settled. So, thou shalt average stations within
    1200km of each other, even when said 1200km crosses oceans, mountains,deserts, and polar ice caps.

    Next time you need to leave your sarcasm on long enough to note that a correlation coefficient of 0.6 is equivalent to an R-squared of 0.36. IOW, the 1200km radus explains 36% of the temperature variation. Science Settled. Moving on.

  80. John Creighton
    Posted Feb 11, 2008 at 3:26 PM | Permalink

    #74 Tim Ball, They should really use satellite data to help to interpolate the missing pieces.

  81. Jim Arndt
    Posted Feb 11, 2008 at 4:14 PM | Permalink

    Hi,

    John #80,
    I think they don’t use satellite data is that the satellite and GISS have diverged lately. Current January 2008 RSS -.629, UAH -.588, GISS +.12. That is the problem. As for coverage I see the tropic and desert seem to have few stations. That is where the AGW group says we should have changes along with the well known Arctic and Antarctic coverage problems. GISS has yet to explain the divergence from the satellite data. Some say its the polar regions but if you do or do not include them it still doesn’t seem to match anything.

  82. steven mosher
    Posted Feb 11, 2008 at 4:21 PM | Permalink

    re 79. I think we’re in violent agreement, unless you want to fight about it

  83. Posted Feb 11, 2008 at 5:04 PM | Permalink

    Here is a graph with much higher station-to-station correlation in high lattitudes, the correlation is worst near the equator, which is logical as he value of annual average temperature is dominated by winter temperature.

    which is actually not bad for interpolation to 1200 km at high lattitudes.
    A similar interpolation is done with global heat flow measurements.

    http://en.wikipedia.org/wiki/Spherical_harmonics

  84. Bob North
    Posted Feb 11, 2008 at 5:05 PM | Permalink

    RE: Greenland data – From my review, there are two stations with long-term data (at least 1930 – present), Gadthab Nuuk and Angmagssalik, in the GISS station data. For both of these sites, the temp anomaly in the 1930s/40s was about the same as the past ten years or so. The two long-term sites in Iceland (Reykjevik and Akureyri) as well as one (Jan Mayen) on the big island north of Norway and Sweden show the same pattern. So for this part of the Artic, the current temperatures are not much different than historic temps.

    Re: Central Park, NYC – If you look at the pre-homgeneization data from this site (i.e., not corrected for UHI), there is very little, if any increase in temperatures since the 1940s. For this specific site, UHI may not be that big a deal in the latter half of the 20th century since the entire area around Central Park was already built up by that time. Since we are looking at anomalies instead of actual temperature, I don’t know that it would play that big of a deal in locations that have been heavily urbanized throughout the temperature record.

    Re: lumping different climatic zones into one grid – Again – since what they are looking at are the temperature anomalies for each station, I don’t think that lumping the sites together makes that big a difference.

    The easiest way to evaluate the question of whether there is long-term global warming would actually to calculate the long-term temp anomaly trend for an spatially representative set of stations (probably 1000 would be enough) with the most reliable data set and determine if there is a statistically significant increasing trend at each location. If 90% of these stations showed a statistically significant increase, some showed no change, and less than 5% showed cooling, we could be pretty damn certain that there has been world-wide warming. I am sure it would be possible to develop specific cutoff percentages (e.g., 70% warming, 20% no trend, 10% cooling) to determine the answer. Of course, you don’t get a global average temp. anomaly to worry about with this method.

    Bob North

  85. Larry
    Posted Feb 11, 2008 at 5:14 PM | Permalink

    Re: lumping different climatic zones into one grid – Again – since what they are looking at are the temperature anomalies for each station, I don’t think that lumping the sites together makes that big a difference.

    Then why grid the earth at all? Why not one big lump? I can’t see any more rationale for lumping the Mojave desert in with the Pacific ocean than lumping the Andes in with the Indian ocean.

  86. Posted Feb 11, 2008 at 5:19 PM | Permalink

    RE Hans Erren #83,

    Very interesting! However, these correlations must be sensitive to time interval. At a daily frequency, 1200 KM is much too large, though it evidently works well at annual or even monthly frequencies.

    Incidentally, Hans’s maps appear to be based on the 1805 Mollweide projection, which I’m coming to appreciate more and more as I think about this. The 1772 Lambert projection is a big improvement over the 200x Hansen projection, but it makes NAm and Eurasia look like they have been worked over with a meat tenderizer, while SAm is horribly crippled. Mollweide is just as equi-areal, yet much more elegant.

  87. steven mosher
    Posted Feb 11, 2008 at 5:23 PM | Permalink

    RE 83.. thanks Hans, I wonder why hansen did not adjust the range ( 1200km) by latitude.

    Its just code.

  88. steven mosher
    Posted Feb 11, 2008 at 5:26 PM | Permalink

    re 85. If all you care about is global “trend”, use one thermometer.

    central england has a friggin long record. the biases will all work out in the long term.

  89. Mike B
    Posted Feb 11, 2008 at 5:30 PM | Permalink

    re 79. I think we’re in violent agreement, unless you want to fight about it

    I agree. Put up your dukes. ;-)

  90. Sam Urbinto
    Posted Feb 11, 2008 at 6:01 PM | Permalink

    The grids that are a mix of land and water are factored to remove one or the other, supposedly. My beef is that if I’m tracking a grid and the variation between coldest and warmest in the year is 100F, why do I care if the global anomaly is off by 1.2 or so, when you can probably pick any day in a year and have much much more of a spread between the high and low of the day? What’s 1.2 if the temperature changes 40 in a day? Plus, do they track and adjust for heat flow from one 2×2 or 5×5 grid and another? I just don’t see it.

    Anyway, if you want the data, it’s on the GISS site. GLB.Ts+dSST.txt

    Global-mean monthly, annual and seasonal land-ocean temperature index, 1880-present, updated through most recent month on http://data.giss.nasa.gov/gistemp/

    Different than what’s here (although it seems it’s not supposed to be) but it’s the NOAA site NESDIS:

    http://www.ncdc.noaa.gov/gcag/gcagmerged.html

  91. Steve Hempell
    Posted Feb 11, 2008 at 10:15 PM | Permalink

    My question is why isn’t the GISS, and other like temperature data compared to the RSS and UAH data over the time period Jan 1979 to Jan 2008. RSS and UAH are probably the best and most consistant (within themselves and reasonably between them) temperature data that we have. The problems with these satellite data sets are probably well known.

    I realize that there are different baselines etc, etc, but surely the statistical geniuses here can compensate for that (no slight intended). It seems to me that too much time is spent trying to fix the GISS type of data.Lets just compare the two and see what comes out of it I might be interesting.

    For example, I came across this site and what struck me about the bar graphs was what a different impression they gave about the period 1979 to 1997 and how dissimiliar the different regions were. I seemed to me, that to a visual approximation, the years between 1979 and 1998 showed no temperature increase. So I asked William Briggs what he thought. His rather completereply was very interesting:

    Most graphs of temperature show an simple regression like this:

    Dr. Briggs look into the trends more deeply and one of the results was this

    Note: No warming from 1979 to 1996 (or 98 depending on the transects). Warming to 2004 and then cooling again.

    He also, indicates some cycles, and other patterns of unknown origonand in need of more study to see if they mean anything. Dr. Briggs wisely makes no conclusions (except one) for this analysis, but a layperson like myself sure finds them interesting. I would like to see more comment on them and the same type of analysis done on GISS type data to see if it is at all the same, somewhat the same or completely different.

    My question is why isn’t the GISS, and other similar temperature data compared to the RSS and UAH data over the time period Jan 1979 to Jan 2008. RSS and UAH are probably the best and most consistant (within themselves and reasonably between them) temperature data that we have. The problems with these satellite data sets are probably well known.

    I realize that there are different baselines etc, etc, but surely the statistical geniuses here can compensate for that (no slight intended). It seems to me that too much time is spent trying to fix the GISS type of data. Lets just compare the two and see what comes out of it. It might be interesting.

    For example, I came across this site http://mclean.ch/climate/Tropos_temps.htm and what struck me about the bar graphs was what a different impression they gave about the period 1979 to 1997 and how dissimilar the different transects were. I seemed to me, that to a visual approximation, the years between 1979 and 1998 showed very little no temperature increase. So I asked William Briggs http://wmbriggs.com/blog/ what he thought. His rather complete reply was very interesting:

    Most graphs of temperature show an simple regression like this:

    Dr. Briggs look into the trends more deeply and one of the results was this:

    Note: No warming from 1979 to 1996 (or 98 depending on the transects). Warming to 2004 and then cooling again.

    He also, indicates some cycles, and other patterns of unknown origon in need of more study to see if they mean anything. Dr. Briggs wisely makes no conclusions (except one) for this analysis, but a layperson like myself sure finds them interesting. I would like to see more comment on them and the same type of analysis done on GISS type data to see if it is at all the same, somewhat the same or completely different. After all, I would think that at least the trens between the two types of data should be at least similiar.

  92. aurbo
    Posted Feb 11, 2008 at 10:19 PM | Permalink

    Re #73:

    Thanks Steve for the clarification. The “cool parks” theory doesn’t seem relevant unless you only analyze stations where data is collected in cool parks.

    The change-point analysis is designed to find inflection points in a temporal data stream rather than the trend itself. It can only be analyzed well after the fact and perhaps that’s its attraction; it can’t be used for real-time tracking. It’s also a little cumbersome to use, even with large data streams, unless one presumes that the large data stream is in some way systematically correlated.

    Re #84 and NYC data:

    That’s my point. The raw data, or even the TOB corrected or Filnet data shows little warming over the past several decades. However, the UHI corrected data in that same data file differs from the raw data by as much as 7°F (cooler) for some of the months between 1960-1990, but this “correction” is gradually reduced to less than 2°F by the early 2000s. The effect of this “correction” is to create a hockey-stick trace in the past 40+ years where no such curve is detectable in the raw data. In other words, it’s an absurd manipulation of the data base under the guise of being an urban heat island correction in a city that has been urbanized for over 100 years. Still, I hear nothing but the sound of crickets from the AGW crowd on this particular USHCNv1 data series.

  93. Steve Hempell
    Posted Feb 11, 2008 at 10:19 PM | Permalink

    Aghh

    Sorry everyone but kept on getting errors on the web server so two separate tries at sending got combined into one.

  94. Larry
    Posted Feb 11, 2008 at 10:27 PM | Permalink

    88, you just made the ergodic assumption. In the corner. Wih the dunce hat.

  95. David Clark
    Posted Feb 11, 2008 at 11:56 PM | Permalink

    For a quick station coverage animation you can use Microsoft PowerPoint and
    paste each one of John’s Station coverage GIF images onto a seperate slide.
    You can then step forward or back through the years using the up and down
    arrow keys. Down and dirty but works well.

  96. Posted Feb 12, 2008 at 1:38 AM | Permalink

    Still, my UHI corrected average annual temperature in Central Europe does show significant warming over the 20th century.

    http://home.casema.nl/errenwijlens/co2/europe.htm

    Waldo is here.

  97. Gaudenz Mischol
    Posted Feb 12, 2008 at 2:09 AM | Permalink

    Hans

    but when you compare it to temperatures in the late 18th century there’s only minimal warming. When you start in 1880 as Giss and HadCruT do, there is significant warming

  98. rafa
    Posted Feb 12, 2008 at 2:48 AM | Permalink

    Re:grids, equirectangular projections, etc. Some years ago two solar physicists had an angry debate (thru regular mail and scientific magazines, no internet at that time). One of them had published a paper showing a 14 day period solar oscillation. The other one complained the oscillation detected was half the Sun rotation period so what the first one was actually measuring was an harmonic not filtered by the instruments or the data analysis. They finally agreed on building a computer model (CM) simulating the rotation of the sun to avoid instrumental bias. They built inside the CM a grid for the star feeding each cell of the grid with the actual luminosity (or whatever they were measuring) and then made the Sun rotate using the CM to see the average result for the whole Sun. They discovered everything was useless. The method used to map the grid within the CM had too much influence on the final results and afaik they finally switched – side by side-to a different strategy. Sorry if too much off topic. Best

  99. Posted Feb 12, 2008 at 3:18 AM | Permalink

    re 98:
    Indeed, The station data for the graph comes from http://data.giss.nasa.gov/gistemp/station_data/ before the truncation to post-1880 values in january 2005.
    In january 2005 GISS decided to truncate temperature records at 1880…

  100. Bob North
    Posted Feb 12, 2008 at 9:28 AM | Permalink

    This has been a very excellent thread. Despite the widespread issues with the land-based data, I do think we can say that there has been a warming trend over many parts of the world in the latter half of the 20th century. However, as shown in #96 above, the current temperatures do not appear to be wildly different than in many periods in the past, at least based on available instrument records (which probably have issues of their own). What gets me is why don’t the GISS folks just come clean with the many issues surrounding the temperature record and put some much bigger error bars on the trend estimates? Also, why do so many insist on keeping hundreds of records of qeustionable quality, yet when it comes to the pre-instrumental records, they are willing to rely on very sparsely distributed proxies?

    Bob North

  101. Bruce
    Posted Feb 12, 2008 at 11:30 AM | Permalink

    If you look at the CRU Central England Temperatures for the month, do you see anything worth spending 20 trillion dollars to “fix”?

    http://www.cru.uea.ac.uk/cru/climon/data/cet/

  102. Sam Urbinto
    Posted Feb 12, 2008 at 3:20 PM | Permalink

    The trend is telling us anything other than some averaged average of averages of sampled tMeans at random locations. Really, there’s is no evidence it has any meaning at all one way or the other.

  103. Harry Eagar
    Posted Feb 12, 2008 at 3:43 PM | Permalink

    Well, I asked the question (in the Estimation thread) about whether we were to believe there has been 80+% coverage of the Southern Hemisphere, but I already knew the answer.

    I cannot post it, because it’s on paper, but somewhere I have a page from Science around 1999 of a study claiming to tease out a human component of GW. It is accompanied by a map, which shows white for areas with not enough coverage to assess.

    As I recall, in addition to the Roaring ’40s, areas not assessed including all of Africa except the coasts, all of souhwest Asia, almost all of the Amazon basin, all of the Tibetan plateau and Sinkiang, and all of Antarctica.

    In other words, this ‘global’ average disregarded the three hottest and the three coldest parts of the globe. I call shenanigans.

    The graphs are cute but don’t tell me much I didn’t already know.

  104. M.Villeger
    Posted Feb 12, 2008 at 6:45 PM | Permalink

    Sam post #102, that was exactly the gist of my post that was removed. When I posted “corect me if I am wrong” I was expecting to be corrected and learn but not shut.
    In the end, this thread and the previous one is displaying the complexity of the disparate temperature data and of the statistical treatment the data is submitted to. Simply put, my question is: is the signal really better than the noise? Is it why this blog is so critical to make scientists who are not statisticians understand the temperature data quality problem in relation to the climate change issue? Isn’t that the reason why as suggested by Leroux, adding or substracting temperatures is meaningless at explaining climate change?Thank you for an answer.

  105. An Inquirer
    Posted Feb 13, 2008 at 3:28 PM | Permalink

    Bob North, (84) I am not sure you are aware of all the issues for the Central Park, NYC site. Yes, very little, if any increase in unadjusted temperatures since the 1940s. However, in the seventies and eighties, the HCN adjusted temperature was about 7 degrees less than actual reading. Now, the HCN adjusted data is 2 degrees less. The decreased differential adds about 5 degrees (F) of reported warming. Meanwhile, the GHCN adjusted temperature(apparently used for global temperature) is almost seven degrees warmer than the HCN adjusted data — and both come from the same thermometer. Moreover, it is not clear that the UHI effect is no stronger today than it was 30 years ago. While inhabitants may have not increased in numbers, it seems to me in my trips to the area that number of workers in the area has increased, visitors to the park have increased, there are more offices in the area, and and traffic has increased. (Again, personal observation — no empirical study.) It would not surprise me if there had been some increased UHI effect even if the population has not increased.

    Neverheless, even if NYC has not had a warming effect, I am comfortable with the conclusion that on average temperature are higher than they were two or three hundred years ago. Higher temperatures would explain glacial melt and extended growing seasons. Yet, because of proposed regulations, it is important to get an handle on whether current trends match projections of Global Climate Models if these Models are the basis of adopting regulations.

  106. aurbo
    Posted Feb 13, 2008 at 8:03 PM | Permalink

    Re #105;

    I agree. See my point in the last paragraph of #92.

  107. JerryB
    Posted Feb 15, 2008 at 9:27 AM | Permalink

    The USHCN v2 announcement at

  108. JerryB
    Posted Feb 15, 2008 at 9:31 AM | Permalink

    The USHCN v2 announcement
    appears to have been updated, deleting mention of when
    the data are to be made available.


    Steve:
    Interesting. It used to say that the data would be available in July 2007. It’s hard not to think that the delays are related to their awareness that scrutiny of this release is likely to be more substantial than the prior release. Maybe they’ll even have proper documentation and technical reports accompanying the release.

  109. thomas
    Posted Feb 17, 2008 at 12:04 AM | Permalink

    global warming skips Vermont

    I did a quick comparison of the sites with historic data prior to 2006 at co2 science http://www.co2science.org/scripts/CO2ScienceB2C/data/ushcn/ushcn.jsp

    and compared them to the current Giss set.

    at c02 science you can look by state and record the latitudes. If I remember correctly all the sites in vermont save one or two which show some kind of warming actually show cooling . AT co2 science, the state of vermont has escaped all the controversy!
    Then I looked over at the Giss data. It is impossible to find most of these sites: they have been eliminated. But the one or two which remain are the warming ones. One station, enosberg looks very different at Giss than at co2 science. most troubling.

    perhaps stations which were not behaving have gotten the boot

  110. JerryB
    Posted Mar 7, 2008 at 9:04 AM | Permalink

    Belated, but perhaps useful:

    Various organizations have collected temperature data from meteorological
    organizations around the world. Some continue to do so. Some pass on
    much of that data in more or less convenient manners. One such
    organization is the US NCDC, a part of NOAA.

    Among their collections of data is GHCN V2, a combination of previously
    collected (i.e. mostly old) data from numerous sources, and relatively
    recent data from some US locations, and some non-US locations that are
    MCDW locations, i.e. many fewer locations than were among the collections
    of old data. The apparent “drop” in the numbers of GHCN stations was
    largely due to the large number of non-MCDW stations that were included
    among the collections of old data.

    See the paper:

    Peterson, T.C., and R.S. Vose, 1997: An overview of the Global
    Historical Climatology Network temperature database. Bulletin of the
    American Meteorological Society, 78 (12), 2837-2849. (PDF Version)

    linked at:

    http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php?name=temperature

    for more information on GHCN V2 sources.

    Non-US locations which are not MCDW locations are not included in the
    GHCN process for gathering relatively recent data.

  111. steven mosher
    Posted Mar 7, 2008 at 9:44 AM | Permalink

    RE 108. YUP! and the link to the “visual” ftp site, the one with the folders has vanished.

    Hmmm. Changes are a foot laddies. Also, GISS changed their monthly .txt output creating some minor annoyances for excell users of data. Adding text headers, adding additional headers
    every 20 years… Silly boys. what a piece of code can add another peice of code can subtract

    Off topic but of interest to the excell Wennies: Switch to Openoffice. you can extend it
    with R!

  112. steven mosher
    Posted Mar 7, 2008 at 9:54 AM | Permalink

    re 108. Well, Menne’s paper is available but it was kinda milk toast

  113. frank
    Posted Jan 4, 2009 at 1:07 PM | Permalink

    Sooooo, where is the science behind “man-made” global warming?

  114. Posted May 5, 2009 at 1:36 AM | Permalink

    I am going to use this sequence of graphs by John Goetz for a presentation in a couple of days but don’t really understand the following ”

    “In the case where multiple … records exist for a station in the given year, I assigned a blue dot if all records were fully calculated from existing averages, a red dot if at least one record was estimated, and no dot if none of the records could produce an estimate.“

    Even if one of the multiple records at a location exists continuously without breaks, surely the station deserves a blue dot?

    Tom Harris
    tom.harris@sympatico.ca

3 Trackbacks

  1. [...] questo link su ClimateAudit, il blog di Steve McIntyre, ci sono delle mappe molto interessanti sulla distribuzione delle stazioni di osservazione sul [...]

  2. [...] are both from comments in Climate Audit, where there is this telling map about the distribution of actual measurement stations across the [...]

  3. By Spot-The-Warming | Omnologos on Mar 30, 2012 at 7:30 PM

    [...] are both from comments in Climate Audit, where there is this telling map about the distribution of actual measurement stations across the [...]

Follow

Get every new post delivered to your Inbox.

Join 3,263 other followers

%d bloggers like this: