Last week, I wanted to determine what GISS’ tropical land-and-ocean time series was. This did not prove as easy as it sounds. Nothing in GISS is intrinsically complicated – it;s all just averaging and smoothing and adjusting. But the code is written as though the whole thing were being done on a Commodore 64 with negligible memory – which it probably was at one time. The code works to a degree, but what’s actually being done is made hard to see merely from all the operational load of inputting and outputting as though it were a Commodore 64.
To make matters more complicated, their online data is mostly in binary. This is a rather ingenious compromise between being their post-Y2K requirement to put their data online and at the same time keep the hoi polloi out of the data. If you’re operating Fortran on an Unix machine (as medievalists and climate scientists do), then it’s not a problem; but if you’re working in modern languages, it is a problem.
Anyway, I think that I’ve come out alive and can now retrieve and handle GISS gridded data in a modern language and might even figure out the GISS tropical land-and-ocean time series.🙂
Gridded Data SBBX
There are three gridded data sets online at GISS in ftp://data.giss.nasa.gov/pub/gistemp with older versions in subdirectories of ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/ARCHIVE/ such as ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/ARCHIVE/2009_04/binary_files.
Two of the data files are based on GHCN land data and one is based on SST. Surprisingly, there doesn’t seem to be a gridded version for the combined data.
SBBX1880.Ts.GHCN.CL.PA.250 – land with 250 miles influence. About 19 MB.
SBBX1880.Ts.GHCN.CL.PA.1200 – land with 1200 miles influence. About 42 MB
SBBX.HadR2 – SST. I think that this uses a NOAA version after 1980 and HadSST before 1980. About 33 MB.
The program read_sbbx will download and place this into an R-list say) sbbx with two items
sbbx$info – the information list with 8000 rows and 10 columns of information
sbbx$sbbx – the data as a time series starting in 1880 with 8000 columns. Missing data has already been converted to NA. The R- objects take are 30-50% less space than the binary data/
dim(sbbx$sbbx) #1560 8000
There are a variety of series in the directory ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/lplots.
tsp(zon) # 1880.00 2009.25 12.00
I think that the zonal data is sitting in the monthly archives (say)
ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/ARCHIVE/2009_04/ZON/. The dataset LOTI.zon.web appears to be the land-and-ocean time series for 46 latitudes from 90S to 90N in 4 degree increments – other zonal versions SST.zon, Ts.zon and Ts.zon250 appear to be zonal versions of the three gridded data sets.
It looks like I can extract the desired tropical data from this. After a LOT of experimenting – much of which was occasioned by my prior lack of understanding of how binary files worked – I developed a program to extract this binary data – following methods in Nicholas’ scripts to some extent, but modifying them to work better in R now that I sort of understand how binary files work.
dim(loti) # 1552 46
tsp(loti) # 1880.00 2009.25 12.00
From this we can get the tropical average as:
loti.trp=ts(apply ( loti[, abs(lat)< =20],1,mean,na.rm=TRUE),start=1880,freq=12)
There, wasn’t that easy. You’d think that GISS would welcome this sort of software, but they seem uninterested in anything other than Fortran.