Comments on: CRUTEM and HadCRU October 2008

By: Inquiry Disinformation about CRUTEM « Climate Audit

Inquiry Disinformation about CRUTEM « Climate Audit — Wed, 21 Jul 2010 18:16:56 +0000

[...] long before my own FOI requests for CRU station data, I discussed CRU calculations in more detail here, concluding with the observation that “if, like GISS, they are doing nothing other than [...]

By: Sven

Sven — Thu, 20 Nov 2008 09:38:40 +0000

I checked the trend lines. With official HadCrut3 annual data the trend lines starting with 97, 98, 99 and 00 are all going up and only with a starting point being 2001 and onwards, they are going down. But if I use simple averages of monthly data for all the years between 1997 and 2008, then the trend line with a starting point of 1997 is almost flat, a 1998 starting point shows downward trend and only using 1999 and 2000 as starting point do I get a clear upward trend…

By: Sven

Sven — Wed, 19 Nov 2008 21:18:39 +0000

Can someone enlighten me on how the HADCRUT yearly anomaly figures are calculated. The most logical thing would seem to be to just take a simple average of monthly figures by adding the monthly anomalies together and dividing the result by 12. I started looking at the years that are similar to 2008 and found out that (apart from 1995) there are significant differences between the HADCRUT yearly figure and the average of monthly anomalies. For example (the first figure being the HADCRUT yearly and the second the average of months):
1990 – 0.248, 0.254
1995 – 0.276, 0.276
1997 – 0.355, 0.350
1999 – 0.262, 0.296
2000 – 0.238, 0.270
2008 – 0.304, 0.315
The largest difference is somehow in the years 1999 and 2000. I wonder how would the higher anomalies for these years influence the trend line? Would there be a decline since 1997 already? Strange anyhow…

By: steven mosher

steven mosher — Tue, 18 Nov 2008 10:17:35 +0000

RE55.

Most of the files were relatively easy to check, kinda like a bad dream. I’ll double check
it all again, but I couldnt find anything like outlier analysis for example. It’s clear
there are checks for “bad” data, but its data that has been labelled somewhere else as
“bad” No in situ QA. The approach is a file is prepped. Fed to GISSTemp. Gisstemp
checks for obvious blunders in file length, missing data, etc. So there Appear to be no checks within the GISStemp for bad data being ingested.

Most of the QA performed in GISS per their documentation happens offline. Like dropping stations,
dropping periods for stations. Steps that have never been fully documented.

A few more notes:
Step 1 has changed from my last download. new python code written May31 2008. There are a few
adjustments to records ( as mentioned in Hansen 2001, and calculation of stats functions.)
and undocumented routines like “dropstrange.py . I’m currently seeing nothing I would call QA, except rudimentary checks.. ( missing data, etc). I found this though..

C?*** Special title for Jim’s plots..

I’ll go over it again, but its appears that that SOME OTHER program or process was used to identify problem data.( like St Helena) and then input files were altered ( like start date end date) so that GISSTemp assume that the data AS INGESTED is good. Remember there are steps described in Hansen 2001 that are not captured by the source code published.

More later I want to go over the Python again since its new code since I looked at it last. I might not get to that until Thursday.

It sure would be nice if they used version control.

By: John Finn

John Finn — Tue, 18 Nov 2008 09:55:24 +0000

Re: #57

Chris M

Several UK stations had the same Oct=Sep error as those in Siberia – though with less dramatic effect obviously.

By: steven mosher

steven mosher — Tue, 18 Nov 2008 03:01:48 +0000

RE 55. I’ll redownload and have a look. Some of the QA/QC is done offline, some is manual by their own admissio. I’ll run through the code later tonight after dinner

By: Chris M

Chris M — Mon, 17 Nov 2008 22:42:41 +0000

As a UK resident I know we just had a cold October – the official UK Met Office site has data for the month that shows the month was -0.7C below the 1961-90 average! Yet the HadCRU map (and this is produced by a research ‘arm’ of the UK Met Office apparently shows a positive temperature anomaly of about +0.5C. Seems very odd.

By: AlanB

AlanB — Mon, 17 Nov 2008 19:08:47 +0000

Re: steven mosher (#54), Thanks Mosh. looks like it is fixed now. The comment about the "grahics bug" has been removed as well I think. Just a hiccup!

By: Steve McIntyre

Steve McIntyre — Mon, 17 Nov 2008 18:57:53 +0000

Steve Mosher, if you feel like sticking needles in your eyes again — can you see anywhere in the Hansen programs where any QC is done? Hansen et al stated:

Our analysis programs that ingest GHCN data include data quality checks that were developed for our earlier analysis of MCDW data. Retention of our own quality control checks is useful to guard against inadvertent errors in data transfer and processing, verification of any added near-real-time data, and testing of that portion of the GHCN data (specifically the United States Historical Climatology Network data) that was not screened by Peterson et al. [1998c].

I took a quick look at Step 0 and Step 1 and didn’t see anything that could be construed as QC.

By: steven mosher

steven mosher — Mon, 17 Nov 2008 18:51:12 +0000

re 52 worked fine yesterday. I did about 20 of them