http://chiefio.wordpress.com/gistemp/

where I’m documenting my “discoveries”…

I’ve run it up through step3. At step4_5 I hit the ‘endian’ problem with the read of the SBBX.HadR2 dataset in Step5 SBBXotoBX.f file (read(11) at line 112 or so) and convert.HadR2_mod4.upto15full_yrs.f with the same file. I’ve not yet reached the point of finding out if the oisstv2_mod4… et al files have similar problems.

So I’m now looking at making an “endian conversion tool”. We’ll see. It might be easier just to get an old Macintosh PowerPC box (also big endian) and put linux on it with the G95 compiler…

The major issue I found with the G95 compiler was the use, by GIStemp, of a non-standard extension to the language for doing data declarations in a variable declaration. Where they have:

INT foo(3)/1,2,3/

You need instead:

INT foo(3)
DATA foo /1,2,3/

FWIW, I’ve moved the source files into a ‘src’ directory, the executables into a ‘bin’ directory and added a Makefile to do all the compiles (pulling that out of the other scripts). At this point I have a version that works pretty well (up to the bigendian point…) compiles well, and generally seems usable for testing.

I would like to help get the code up and running. I have all the FORTRAN code compiled, but I did not yet look into the details of that coding. I’ve noticed that all the independent efforts started back in 2007/2008 seem to have come to a dead stop. I remain amazed that whatever it is that prevents independent porting of the code so problematic. Can code actually be so tightly tied specific machines and OS configurations?

Drop me an e-mail dhughes4 at mac dot com.

I just ran across this comment on the GISTemp Updates page:

Feb. 11,2009: Two bugs in STEP0 programs were corrected before they had a chance to affect any results. Thanks to Dr. Peter O’Neill for discovering and reporting them to us.

I don’t understand how bugs discovered in 2009 cannot have affected results of calculations made in, say, the 1990s.

Surprising as that may seem, that is the case here.

The first bug I reported was located in unreachable code in antarc_comb.f, and would have had serious consequences but for the fact that it could never be executed. The danger of course would be that at some future date the code might be taken over by someone with experience only of structured programming, who might make the mistake of believing that FORTRAN code of a certain era was always first designed and then coded – and who might then “fix” the unreachable code.

The second bug was in USHCN2v2.f, where the format statement used to read ushcn.tbl selected the wrong (and blank) column for a single digit numeric value, but as the FORTRAN code reads a blank as zero, and as all the values in the correct column are in fact zero for ushcn.tbl, again the current calculations were not affected. The other .tbl files, where the corresponding values would not have been zero, are read instead (correctly) in Python.

The coders of GISTemp seemed to have liked to live dangerously. I’m curious whether this lucky streak will still hold should I find any further bugs as I work my way through the later steps.

On reflection I’m beginning to think releasing the code could have been a clever strategy to get it debugged at no cost before old-timers like myself, who started out with FORTRAN II and may still be able to put ourselves in the mindset needed to follow some of the more esoteric code, die off.

Perhaps this could be our final business opportunity – “we can review your ‘legacy’ FORTRAN II/IV, ALGOL, PL/1, 360 ASSEMBLY, COBOL, APL, RPG II … code; GOTOs welcome; all common, even if now deprecated, stupid programming tricks accepted”?

Feb. 11,2009: Two bugs in STEP0 programs were corrected before they had a chance to affect any results. Thanks to Dr. Peter O’Neill for discovering and reporting them to us.

I don’t understand how bugs discovered in 2009 cannot have affected results of calculations made in, say, the 1990s.

declare @Month as int
Set @Month = 1
Declare @SQLString as varchar(500)

while (@Month <= 12)
BEGIN

SET @SQLString = ‘insert into dbo.StationMonthValue
SELECT CAST( v2mean.stationID as bigint), Tie, cast(v2mean.Ano as int) as [year],
cast(? as varchar) as [Month], cast( v2mean.[?] as int) as value
FROM GHCNStationIventory INNER JOIN
v2mean ON GHCNStationIventory.StationID = v2mean.stationID’

SET @SQLString = REPLACE(@SQLString,’?',cast(@Month as varchar))

EXECUTE (@SQLString)

SET @Month = @Month + 1
END

Then I queried this table to obtain the global average of the values, with this querie:

SELECT AVG(Value)
FROM dbo.StationMonthValue

The result was: 120

Does this mean that the whole dataset presents a raw average deviation from the reference period of 1.2 ºC ?

CREATE TABLE [dbo].[GHCNStationIventory](
[StationID] [varchar](11) NULL,
[Column 1] [varchar](1) NULL,
[Station Name] [varchar](31) NULL,
[Latitude] [varchar](6) NULL,
[Longitude] [varchar](8) NULL,
[Elevation] [varchar](5) NULL,
[col 6] [varchar](5) NULL,
[Urbanization] [varchar](1) NULL,
[Column 8] [varchar](5) NULL,
[Column 9] [varchar](2) NULL,
[Column 10] [varchar](2) NULL,
[Column 11] [varchar](7) NULL,
[LandType] [varchar](16) NULL,
[C13] [varchar](1) NULL,
[Filler] [varchar](1) NULL
) ON [PRIMARY]

My next step was to create a new table that in my opinion addresses better the Normals Forms of relational database (http://en.wikipedia.org/wiki/Database_normalization), here is the script to create this new table:

On this new table I will have one record per value, which makes future queries much easier.

I have taken the v2.mean file from GHCN and I have imported it to an SQL Server 2008 database, to explore the data with the tools I am used to.

I imported the raw data to a new table and then runned a SQL script to transform -9999 into Null values.

The create table script is:

CREATE TABLE [dbo].[v2mean](
[stationID] [varchar](11) NULL,
[Tie] [varchar](1) NULL,
[Ano] [varchar](4) NULL,
[1] [varchar](5) NULL,
[2] [varchar](5) NULL,
[3] [varchar](5) NULL,
[4] [varchar](5) NULL,
[5] [varchar](5) NULL,
[6] [varchar](5) NULL,
[7] [varchar](5) NULL,
[8] [varchar](5) NULL,
[9] [varchar](5) NULL,
[10] [varchar](5) NULL,
[11] [varchar](5) NULL,
[12] [varchar](5) NULL,
[filler] [varchar](1) NULL
) ON [PRIMARY]

The script to handle the null values is here:

declare @Month as int
Set @Month = 1
Declare @SQLString as varchar(500)

while (@Month <= 12)
BEGIN

SET @SQLString = ‘update dbo.[v2mean]
set [{0}] = null
where [{0}] = {1}’

set @SQLString = REPLACE(@SQLString,’{0}’,cast(@Month as varchar))
set @SQLString = REPLACE(@SQLString,’{1}’,'-9999′)

execute (@sqlstring)

set @Month = @Month + 1
END