CRU – Then and Now

In respect to the alleged CRU confidentiality agreements (which look increasingly fictitious), Jean S made the interesting observation that CRU archived station in the 1980s and early 1990s at CDIAC (ndp020) and that the alleged CRU confidentiality agreements, for some reason, did not interfere with that data being archived.

Interestingly, I happen to have a station list and station data of 1994 vintage – a little later than NDP020 – and compared the present station list to the 1994 list, showing both deletions and additions. The number of stations increased from 3524 in the 1994 vintage to 4138 in the current vintage, which gives the impression of more activity and expanded coverage. But see the remarkable maps below. Most of the additions are in the US and a couple of other countries, while deletions are widespread.

I’ve spent a fair bit of time comparing the lists. There’s a lot of sloppiness in the present CRU list – country identifications have not been updated for example. The USSR lives on at CRU. Not infrequently the same station is picked up from two different sources, each duplicate having a separate CRU identification.

But I had another interest in the comparison. In refusing a prior FOI request, Jones said that the data was already publicly available at GHCN. This is not true for the entire data set, but many CRU stations can be conclusively identified with GHCN stations. If Jones is deriving his CRU version for (say) Bahrain from GHCN, he cannot also rely on a confidentiality agreement with Bahrain as an excuse for not providing data.

It should be possible to go through the countries with data additions since 1994 one-by-one, mark off the ones whose data additions come from GHCN and see what’s left. This sort of thing takes a bit of time, but the results are always interesting. Here’s a preliminary report on the provenance of the additions – the ones to which the alleged confidentiality might apply.

In the list below, there are 59 countries in which a station has been added since the 1994 “public” version.

The vast majority of the increase came from the US through the addition of much of the USHCN network (frequently duplicating prior sources). Some US stations could not be tied so far to USHCN or GHCN versions.

New non-GHCN versions were added for Canada and Australia, frequently as duplicates.

A number of European countries added non-GHCN versions: Austria, Denmark, Sweden, Norway, Finland, Netherlands, one UK station (Wick), one Ireland station (Phoenix Park), one Italy station (Milano/Brera).

In the rest of the world, the additions were nearly all GHCN stations. Exceptions were Syria and Taiwan. Perhaps China, though I suspect that more identifications with GHCN versions are possible than I’ve accomplished so far. Given that the additional data for Spain comes from GHCN stations, the question of a confidentiality agreement with Spain becomes moot. Similarly for say South Africa or PNG. There are a few odds and ends that I couldn’t match right away – a few stations in Iran, one in Brazil, a couple in Bolivia. Argentina has a number of additions (nearly all airports).

It’s hard for me to imagine that CRU has entered into confidentiality agreements during the IPCC period with the countries adding non-GHCN versions to the version of the data to which no restrictions applied. Countries like the US, Canada, Australia, Denmark, Netherlands, Sweden, Finland, Norway – a list that includes some self-righteous countries.

SUMMARY:
country count Comment
ALGERIA-2: Both GHCN stations
ANTARCTICA-1. GHCN stations (Bernardo O’Higgins renumbered)
ARGENTINA 21. New stations are mostly airports
AUSTRALIA 147 Mostly GHCN. Need to crosscheck others.
AUSTRIA 11 Mostly new.
BAHRAIN 1 GHCN station.
BANGLADESH 2 GHCN stations
BOLIVIA 2 Two new: Potosi, Central La Paz
BRAZIL 2 One GHCN (Brasilia airport); one new- Caceres
BURKINA FASO 1 GHCN station.
CANADA 95 Many new; many duplicates
CHILE 4 Two GHCN; two “Unknown” 858341, 858921
CHINA 34 Many GHCN; many unmatched, but perhaps can be collated.
COTE D’IVOIRE 1 GHCN station
CROATIA 4 Renumbered GHCN
DENMARK 3 Three new.
DOMINICAN REPU 1 GHCN station
ECUADOR 1 GHCN station
EGYPT 1 GHCN station
FINLAND 3 Three new
FRANCE 1 GHCN station
GERMANY 13 GHCN stations
GREECE 3 Two GHCN; check ATHENS/HELLENIKON
GREENLAND 2 GHCN – check renumbering of Kap Tobin
HONDURAS 1 GHCN station
HUNGARY 2 GHCN stations
ICELAND 1 GHCN station renunbered
INDIA 2 GHCN stations
IRAN 35: 32 GHCN, Oroomieh, Bandar Azali, Chahbahar
IRELAND 3 Two GHCN- check Phoenix Park
ISRAEL 5 All new
ITALY 1 Check Milano/Brera
JAPAN 4 GHCN stations
JORDAN 1 GHCN stations
KOREA 9 GHCN stations
KUWAIT 1 GHCN stations
MADEIRA 1 GHCN stations
MALAWI 1 GHCN stations
MALAYSIA 1 GHCN stations
MEXICO 14 GHCN stations
MONGOLIA 2 GHCN stations
NETHERLANDS 4. One GHCN – check De Kooy, Eelde, Vlissingen
NEW ZEALAND 7 GHCN stations
NORWAY 12 Six GHCN, some new
PAPUA NEW GUIN 5 GHCN stations
PHILIPPINES 1 GHCN stations
PORTUGAL 3 GHCN
PUERTO RICO 1 GHCN stations
SLOVENIA 1 Renumbered GHCN
SOUTH AFRICA 34 GHCN
SPAIN 4 GHCN stations
SWEDEN 20 Mostly new
SYRIA 6 Mostly new
TAIWAN 14 Mostly new
TURKEY 1 GHCN station
UK 2 One GHCN – check Wick
URUGUAY 1 GHCN stations
USA 1051 additions – mostly USHCN
RUSSIA (“USSR”) 1 GHCN station

25 Comments

  1. Antonio San
    Posted Jul 25, 2009 at 1:09 PM | Permalink

    One cannot look at the new map and its US stations concentration without keeping in mind Anthony Watts work in the US and draw their own conclusions.

  2. Craig Loehle
    Posted Jul 25, 2009 at 1:17 PM | Permalink

    Once again, an unfunded person (Steve) can readily identify duplicates in just a few hours that CRU has not bothered to check. wow. The different coverage of the two maps is stunning. Siberia is missing from both maps, but there are stations there.

    Steve: Craig – this is a plot of additions and deletions – not the continuing stations.

  3. R
    Posted Jul 25, 2009 at 1:30 PM | Permalink

    Have you tried an FOI for the confidentiality agreements?

  4. Jan F
    Posted Jul 25, 2009 at 2:47 PM | Permalink

    Netherlands can’t be a problem with confidentiality, all the data is available at the KNMI website.

  5. Thor
    Posted Jul 25, 2009 at 4:06 PM | Permalink

    Data covering Norway shouldn’t be a problem either, it can be freely downloaded from the eklima.met.no website.

  6. Steve McIntyre
    Posted Jul 25, 2009 at 4:14 PM | Permalink

    Then please tell me which “European meteorological agencies” are preventing CRU disclosure?

    And please remember that I want to see the “value added” CRU versions, which I presume differ in some respects from the original.

    • Soronel Haetir
      Posted Jul 25, 2009 at 6:35 PM | Permalink

      Re: Steve McIntyre (#6),

      Steve,

      The confidentiality agreement is between Jones and himself. His brain must be kept in the dark about what his heart and hands are doing.

  7. Posted Jul 25, 2009 at 4:43 PM | Permalink

    Historic data for The Netherlands is completely in the public domain, free of charge.
    For monthly values for De Kooy and Eelde (Airports) and Vlissingen (Harbour) see
    http://www.knmi.nl/klimatologie/maandgegevens/index.html

    even down to hourly values
    http://www.knmi.nl/klimatologie/uurgegevens/

  8. Jon
    Posted Jul 25, 2009 at 5:09 PM | Permalink

    Then please tell me which “European meteorological agencies” are preventing CRU disclosure?

    Isn’t that a reasonable next line of FOI inquiry? Their last reply would be sufficient if they forgot some of which institutions requested secrecy, but have they lost track of all of them?

    • Steve McIntyre
      Posted Jul 25, 2009 at 8:12 PM | Permalink

      Re: Jon (#9), Huh?? People from all over the world have sent FOI inquiries asking exactly that.

  9. Posted Jul 25, 2009 at 8:43 PM | Permalink

    Is it the “value-added” part that you cannot get, rather than the raw data? Whoever “added value” to the data through adjustments doesn’t want you to look for errors in that “value,” right?

    • Steve McIntyre
      Posted Jul 25, 2009 at 8:49 PM | Permalink

      Re: Micajah (#12),

      I want to see the before and after. They are refusing both.

  10. Barry R
    Posted Jul 25, 2009 at 9:20 PM | Permalink

    It sounds as though you could probably piece together most of the before, though it is ridiculous that you would have to do that.

  11. steven mosher
    Posted Jul 26, 2009 at 12:55 AM | Permalink

    It’s hard for me to imagine that CRU has entered into confidentiality agreements during the IPCC period with the countries adding non-GHCN versions to the version of the data to which no restrictions applied. Countries like the US, Canada, Australia, Denmark, Netherlands, Sweden, Finland, Norway – a list that includes some self-righteous countries.

    harder yet to believe that they all had agreements that prevented release to “non academics”

  12. Geo
    Posted Jul 26, 2009 at 8:22 AM | Permalink

    Duplicate stations in the station list? Is the detail data duplicated for those stations as well?

  13. Geoff Sherrington
    Posted Jul 26, 2009 at 8:44 AM | Permalink

    The “Old” Australian stations seem to match loosely what became the Reference Climate Station Network from about a 1990 start. Of the order of 100 stations, but hard to see from the map scale.

    The “New” version removed Australian Antarctic and added many stations that were previously rejected here on quality grounds. So this seems a backwards step if that is so.

    Maybe they were merely kept on record and only some were used in computations of matters like global averages. The news today will clarify that speculation in due course.

  14. BrianMcL
    Posted Jul 26, 2009 at 4:24 PM | Permalink

    With all of this talk of “secret” weather data I’m reminded of a story told by my high school Geography teacher roughly 20 years ago about Cambodia during the early days of the reign of the Khmer Rouge.
    His story was that weather data was the only data they exchanged with the outside world, on the basis that it was so important that even they knew that it crossed national and ideological boundaries.
    I’ve often wondered if this were true, but given the goings on at the CRU I can only conclude that it can’t have been.

  15. Rui Sousa
    Posted Jul 27, 2009 at 4:21 AM | Permalink

    Just a quick note:

    MADEIRA is a Portuguese region, not a country by it self.

    You can easily get duplicate data when this kind of errors occur, if you have different sources for you data.

  16. Rui Sousa
    Posted Jul 27, 2009 at 5:38 AM | Permalink

    Portuguese data can be requested here:

  17. Rui Sousa
    Posted Jul 27, 2009 at 5:40 AM | Permalink

    here

  18. Patrick M.
    Posted Jul 27, 2009 at 5:41 AM | Permalink

    Dear Mr. McIntyre,

    Please send me a copy of the CRU data.

    Yours Sincerely,
    Patrick M.

    p.s. Don’t give me any lame excuses about not being able to release the data due to the confidential manner in which you obtained it. No really, I’m serious.

    Steve: I suggest that you direct your question to Phil Jones at CRU. Let us know how you do.

    • Alan Wilkinson
      Posted Jul 27, 2009 at 3:04 PM | Permalink

      Re: Patrick M. (#23),

      Getting a bit ahead of yourself, I think. When Steve publishes some conclusions drawn from the data he will doubtless also publish the data – not just for you but for everyone to review.

      In the meantime there are other things to take care of, particularly that nothing in the data can aid the identification and persecution of its source.

      Stunts like yours are a pointless waste of time.

  19. rtgr
    Posted Jul 28, 2009 at 3:53 AM | Permalink

    NETHERLANDS 4. One GHCN – check De Kooy, Eelde, Vlissingen

    Metainfo on all the dutch stations
    http://www.knmi.nl/klimatologie/metadata/stationslijst.html

    hourly data (not homogenised)
    http://www.knmi.nl/klimatologie/uurgegevens/#no
    daily data all stations (Not homogenised)
    http://www.knmi.nl/klimatologie/daggegevens/download.html

    Central Netherlands temperature (homogenised 4 to 6 stations)
    http://www.knmi.nl/klimatologie/onderzoeksgegevens/CNT/tg_CNT.txt

    method used;
    “construciton of a central dutch temperature”

    Click to access CNT.pdf

  20. Richard
    Posted Jul 30, 2009 at 8:39 PM | Permalink

    Steve,

    This is a edited version of what I posted at WUWT (since I got snipped last time):

    I have plotted the slopes of the temperature data of UAH, RSS, Hadley CRU and GISS from 1979 to 2008.

    (In other words the slopes of the temperature data of all 4 data sets from 1979 to 2008, 1980-2008, 1981-2008… upto 2007-2008, and then plotted the graph of these values for all 4 data sets)

    Between the satellite data’s UAH and RSS there is a pretty good match all the way through.

    GISS starts diverging from 1992. (Before this date all 4 lines are practically identical). Then the other 3 continue practically identically till 2002, when Hadley starts diverging.

    After the above dates not only are the temperatures elevated in the case of GISS, but the downward slope of the graph decreases from 1992 and in the case of Hadley from 2002. (All 4 graphs of the slopes start themselves sloping downwards from 1992)

    In the case of GISS the downward trend of the slope levels off from 2002 to 2005. Then it appears to precipitously play “catch-up”.

    Is there anything significant about these graphs? Do they tell us anything?

3 Trackbacks

  1. […] CRU – Then and Now […]

  2. […] […]

  3. […] couple of other posts at the time of the Mole incident were here and Dr Phil, Confidential Agent , in which I observed that Jones, as a temperature accountant, was […]