GISS Gridded and Zonal Data

Last week, I wanted to determine what GISS’ tropical land-and-ocean time series was. This did not prove as easy as it sounds. Nothing in GISS is intrinsically complicated – it;s all just averaging and smoothing and adjusting. But the code is written as though the whole thing were being done on a Commodore 64 with negligible memory – which it probably was at one time. The code works to a degree, but what’s actually being done is made hard to see merely from all the operational load of inputting and outputting as though it were a Commodore 64.

To make matters more complicated, their online data is mostly in binary. This is a rather ingenious compromise between being their post-Y2K requirement to put their data online and at the same time keep the hoi polloi out of the data. If you’re operating Fortran on an Unix machine (as medievalists and climate scientists do), then it’s not a problem; but if you’re working in modern languages, it is a problem.

Anyway, I think that I’ve come out alive and can now retrieve and handle GISS gridded data in a modern language and might even figure out the GISS tropical land-and-ocean time series. :)

Gridded Data SBBX
There are three gridded data sets online at GISS in ftp://data.giss.nasa.gov/pub/gistemp with older versions in subdirectories of ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/ARCHIVE/ such as ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/ARCHIVE/2009_04/binary_files.

Two of the data files are based on GHCN land data and one is based on SST. Surprisingly, there doesn’t seem to be a gridded version for the combined data.

SBBX1880.Ts.GHCN.CL.PA.250 – land with 250 miles influence. About 19 MB.
SBBX1880.Ts.GHCN.CL.PA.1200 – land with 1200 miles influence. About 42 MB
SBBX.HadR2 – SST. I think that this uses a NOAA version after 1980 and HadSST before 1980. About 33 MB.

The program read_sbbx will download and place this into an R-list say) sbbx with two items
sbbx$info – the information list with 8000 rows and 10 columns of information
sbbx$sbbx – the data as a time series starting in 1880 with 8000 columns. Missing data has already been converted to NA. The R- objects take are 30-50% less space than the binary data/

source(“http://data.climateaudit.org/scripts/gridcell/read.binary.txt”)
url=”ftp://data.giss.nasa.gov/pub/gistemp/SBBX1880.Ts.GHCN.CL.PA.250″ #18577
sbbx=read_sbbx(url)
dim(sbbx$sbbx) #1560 8000

Lplot Data
There are a variety of series in the directory ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/lplots.

url=”ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/lplots/LOW.Ts.GHCN.CL.PA.lpl”
zon=read.lpl(url)
tsp(zon) # 1880.00 2009.25 12.00

Zonal Data
I think that the zonal data is sitting in the monthly archives (say)
ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/ARCHIVE/2009_04/ZON/. The dataset LOTI.zon.web appears to be the land-and-ocean time series for 46 latitudes from 90S to 90N in 4 degree increments – other zonal versions SST.zon, Ts.zon and Ts.zon250 appear to be zonal versions of the three gridded data sets.

It looks like I can extract the desired tropical data from this. After a LOT of experimenting – much of which was occasioned by my prior lack of understanding of how binary files worked – I developed a program to extract this binary data – following methods in Nicholas’ scripts to some extent, but modifying them to work better in R now that I sort of understand how binary files work.

url=”ftp://data.giss.nasa.gov/pub/gistemp/GISS_Obs_analysis/ARCHIVE/2009_04/ZON/LOTI.zon.web”
loti=read.zon.web(url)
dim(loti) #[1] 1552 46
tsp(loti) #[1] 1880.00 2009.25 12.00

From this we can get the tropical average as:

lat=as.numeric(dimnames(loti)[[2]]);lat
loti.trp=ts(apply ( loti[, abs(lat)< =20],1,mean,na.rm=TRUE),start=1880,freq=12)
ts.plot(loti.trp)

There, wasn’t that easy. You’d think that GISS would welcome this sort of software, but they seem uninterested in anything other than Fortran.

142 Comments

  1. Gary Strand
    Posted Jun 5, 2009 at 4:36 PM | Permalink

    If you’re operating Fortran on an Unix machine (as medievalists and climate scientists do), then it’s not a problem; but if you’re working in modern languages, it is a problem.

    Neither Unix nor Fortran are medieval.

    • Steve McIntyre
      Posted Jun 7, 2009 at 9:38 PM | Permalink

      Re: Gary Strand (#1),

      Calling this code “medieval” is an insult to the 12th century cathedral designers. Perhaps “Neolithic” would be a mot more juste.

      • Gary Strand
        Posted Jun 7, 2009 at 9:49 PM | Permalink

        Re: Steve McIntyre (#50), getting snotty about Fortran will win you many supporters. It’s not that difficult a language to learn.

        • Steve McIntyre
          Posted Jun 7, 2009 at 9:58 PM | Permalink

          Re: Gary Strand (#51), Gary, my remark in 50 was really intended at the GISTEMP code. It more or less turns modern machines into emulators of machines with very limited memory (hence the Commodore 64 remark) and richly deserves to be called Neolithic. Its authors would undoubtedly agree that the code is obsolete.

          It’s not that difficult a language to learn.

          I don’t know how old you are, but my guess is that I learned Fortran before you were born. I identified Mann’s PC error by parsing Fortran code that he had inadvertently left on the website that was made public after our 2003 article (concurrent with him deleting the “wrong” data.)

          There is an enormous amount of time and effort in index management in Fortran code – be it Mann’s or Hansen’s. Things that take hundreds of line of code can be written in a few lines in a modern language R, Matlab. The heavy code requires more documentation, which is typically neglected e.g. the code in question.

          In my opinion, for statistical calculations and the sort of thing discussed here, I simply do not believe that continued use of Fortran can be justified – it requires too many pointless lines of code and, as a result, ends up wasting money through low productivity. If I were a manager, I would require analysts to convert to a more modern language.

          Let me add a rider – I’ll exempt climate models and computation-intensive operations as I can picture situations where extra lines of code may pay off in machine speed.

        • Gary Strand
          Posted Jun 7, 2009 at 10:04 PM | Permalink

          Re: Steve McIntyre (#52), old code isn’t that difficult to understand. Someone as versed in old Fortran as you claim to be (and I seriously doubt you learned it before I was born) shouldn’t have any problems. Besides, Fortran isn’t a non-modern language. It’s no more dead than English is.

        • Steve McIntyre
          Posted Jun 7, 2009 at 10:19 PM | Permalink

          Re: Gary Strand (#53), I learned Fortran in 1966. As I said, I don’t know how old you are, but your cv didn’t list any publications prior to 1997, from which I guessed that you would not be older than 43. But I don’t rely on this guess. Perhaps you’re as old as me.

          Steve Mosher described Hansen’s code as like sticking needles in your eyes. It’s execrable code.

    • Scott Lurndal
      Posted Jun 8, 2009 at 2:39 PM | Permalink

      Re: Gary Strand (#1),

      Selecting example C code from the obfuscated C contest is hardly
      a fair comparison. C code can be very readable and maintainable.

      It is hardly a modern language, being less than 20 years younger than fortran
      which should hit 50 this year.

  2. Posted Jun 5, 2009 at 4:40 PM | Permalink

    “…but they seem uninterested in anything other than Fortran.” All official documents in the Roman Catholic Church are written in Latin, possibly for similar reasons. Will climate “scientists” soon don the Roman collar? (Phil Jones, on the other hand, I see in something more flamboyant, along the lines of the Swiss Guard.)

    • Andrew
      Posted Jun 5, 2009 at 5:30 PM | Permalink

      Re: jorgekafkazar (#2), I find the implication towards the Church insulting sir. Latin is traditional. Fortran is just an obstacle.

  3. tesla
    Posted Jun 5, 2009 at 5:21 PM | Permalink

    Steve I know you’re an R guy but MATLAB has a number of nice built-in functions for dealing with binary data.

    • AnonyMoose
      Posted Jun 5, 2009 at 6:28 PM | Permalink

      Re: tesla (#3), Shh, before he goes medieval on more.

    • Chad
      Posted Jun 5, 2009 at 9:26 PM | Permalink

      Re: tesla (#3), They don’t call it the language of technical computing for nothing!

  4. Posted Jun 5, 2009 at 9:32 PM | Permalink

    Nice work. I learned to program on a Commodore Pet, – pre Vic 20 but just after punch cards – thank god.

    • tty
      Posted Jun 6, 2009 at 1:44 AM | Permalink

      Re: Jeff Id (#7)

      I thought I was about last person alive who still remembers the Commodore Pet, though I did start with punch-cards.

      Actually there is a saying around that fortran is like latin, there is a lot of it still around, but it is a dead language.

  5. DG
    Posted Jun 6, 2009 at 12:03 AM | Permalink

    My first foray into programming was punch cards, then our school purchased the first generation TRS80 (1977?), which was plagued with RF interference. The Sinclair 1000 was my first home “computer”; the only affordable solution at the time.

    I’m inclined to learn R just to stop feeling left out of the pack.

  6. pjm
    Posted Jun 6, 2009 at 3:42 AM | Permalink

    Why binary? Because it takes less space and software runs more quickly in less memory (important back then). Why now? I imagine it’s because the existing I/O code works, so it is easier to keep using it. It also avoids any inconsistencies produced by changing binary to text and back again. (And do you think somebody might use EBCDIC instead of ASCII?)

    BTW don’t sneer at the C64. I have fond memories – my first ever machine code was written for it. The 64K of memory was quite respectable at the time; the university mainframe I did Fortran on only 6 years earlier had only 32K of core memory.

  7. jeez
    Posted Jun 6, 2009 at 4:09 AM | Permalink

    Steve, have you looked over any of E. M. Smith’s work in regards to GISS?

    http://chiefio.wordpress.com/category/gisstemp-technical-and-source-code/

    It may be helpful.

  8. J.Hansford
    Posted Jun 6, 2009 at 4:30 AM | Permalink

    Ah, the arcane language of the AGW Climate alchemist….. Picking around in the entrails of dead computer program, so as to divine the future of the climate :-)

  9. per
    Posted Jun 6, 2009 at 5:12 AM | Permalink

    just for the record, i was trying to play with R. When i tried to source “read.binary.txt”, R threw errors unexpected } on lines 167 and 172.

    Is that a glitch in my version of R ?
    ta
    per

    • Steve McIntyre
      Posted Jun 7, 2009 at 9:33 PM | Permalink

      Re: per (#13),

      per, sometimes my saved script differs a little from what I exeuted e.g if I had a hanging chad and fixed it on the console without concurrently changing my script. I’ll check. I’ve been away for a couple of days.

    • Steve McIntyre
      Posted Jun 8, 2009 at 2:35 PM | Permalink

      Re: per (#13),

      I had some stray lines in the script that I’ve fixed. Sorry bout that. Hope it works now.

  10. Kusigrosz
    Posted Jun 6, 2009 at 6:46 AM | Permalink

    Modern languages? From the R FAQ question 2.5.1 How can R be installed (Unix):

    Note that you need a FORTRAN compiler or perhaps f2c in addition to a C compiler to build R.

    find R-2.9.0/src -name ‘*.f’
    reveals 45 medieval artifacts.

  11. Craig Loehle
    Posted Jun 6, 2009 at 6:51 AM | Permalink

    Nothing like casting aspersions on a programming language to start a food fight…

  12. dearieme
    Posted Jun 6, 2009 at 9:12 AM | Permalink

    Punched cards were an innovation for me – I started on punched tape.

  13. Gary
    Posted Jun 6, 2009 at 9:32 AM | Permalink

    It’s easy to laugh at arcane and outmoded languages and formats, but the truth is that porting to “modern” versions is expensive if you are going to do it thoroughly and nobody is willing to pay for it. Maybe you can bury it in a budget for a project that has another objective. Librarians are confronted with this issue in the creation of digital repositories. The modern is always in the future and nobody really can predict what is cost-effective for today and tomorrow. That is why this site and others are so valuable. It’s volunteers willing to pursue the task for personal reasons rather than making a living at it that keeps the data alive.

  14. John S.
    Posted Jun 6, 2009 at 9:53 AM | Permalink

    If computers understood source languages, I would understand this endless debate.

  15. Posted Jun 6, 2009 at 10:59 AM | Permalink

    Before we get too bogged down in theological problems such as calculating how many angels can dance on the head of a pin (or climate modellers on an ice floe) perhaps we should look at what the data actually show. Land temperature was below sea temperature from 1880 to 1980. Since then land temperatures have risen much faster than sea temperatures, from 0.2 o C below to 0.3 o C above. Is CO2 forcing only effective over land or is the heat island effect the real reason?

    See http://www.climatedata.info/Temperature/assets/09-Difference%20land-ocean%20temperature.gif .

    • RomanM
      Posted Jun 6, 2009 at 12:54 PM | Permalink

      Re: Anne T Cyclone (#19),

      Land temperature was below sea temperature from 1880 to 1980.

      I don’t think that this is an accurate statement. It appears that your graph was calculated from anomalies and the only information in the graph itself is that the difference has in fact increased over time.

      In particular, there is nothing special about 1980, since a zero value merely indicates that the difference between the temperatures at that time is equal to the difference between the average temperatures over the anomaly calculation period. It does not tell whether land temperature is specifically higher or lower than the sea temperature.

      • Posted Jun 6, 2009 at 11:47 PM | Permalink

        Re: RomanM (#23),

        I accept that as the temperature values in the graph from http://www.climatedata.info were anomalies the use of the word “below” was not correct. The fact remains that the relationship between land and sea temperatures changed quite dramatically for last 2 or 3 decades of the 20th century. This change, with an increase in land temperature relative to sea, is consistent with poor compensation of the urban heat island effect. It does not prove that that is the explanation but it is so marked that I feel it does need an explanation.

        • Geoff Sherrington
          Posted Jun 7, 2009 at 5:55 AM | Permalink

          Re: Anne T Cyclone (#42),

          In a study I am doing of 17 truly rural stations from around Australia, about half are by the seaside or on islands. They showed almost nil warming over the past 40 years. OTOH, the half that were well inland (but not greatly elevated above sea level) showed quite strong temperature rises over the last 40 years. So it’s more complicated than just sea versus land and UHI. No UHI in sight. There seems to be an effect that operates inland, but not at or near the sea (or vice versa). Plausibly, this is a transient effect, because if you project straight lines back from that 40 year term for inland sites, you reach situations that disagree with older data. Each day I tend more towards instrumental problems. Most of my studied inland sites have had thermocouple devices since the early 1990s. Some makers say they should be renewed each 2-3 years. I do not know if they have been. These inland sites are much less fun to travel to and have no electronics shops for hundreds of km.

          Please see some work in progress at http://landshape.org/enm/40-years-of-some-bom-australian-rural-temperature-data/, the Niche Modeling blog, for a preview.

          This is in the category of “Is global warming global?” like the Antarctic. It also causes a complication because some terms drived to adjust data, from papers a decade or more ago, now need some different terms because their basis of calibration has changed since they were written. Like temperature correlations with distance for near-neighbour weighted corrections, when the neighbours have moved.

        • RomanM
          Posted Jun 7, 2009 at 6:38 AM | Permalink

          Re: Anne T Cyclone (#42),

          I don’t think that you will get an argument here there there have been changes in the relationship between the land temperatures and the SSTs. However, UHI is just one possible factor which can contribute to these changes.

          Besides the ones mentioned by pjm in comment 41, one needs to look at the possible effects of clouds. From the Cloud Amount graph on the web page, on can see that there has been a variation of the order of 10% (not 10 percentage points) of the mean global cloud amount since 1984. Since the clouds can have different effects on the temperature depending on the latitude and the season, their relationship is not obvious.

          There have also a number of changes in local climate regimes in the recent past as well. From the Alaska Climate Research center site, (which someone linked to on another CA thread yesterday) one can see the effects of quantum changes in weather patterns in Alaska since the mid 1970s. As well, large increases seem to have occurred in Siberia since the middle of the 20th century. The latter (due to the enormity of the geographic area) could well represent a very large large part of the recent “global warming”.

          Without doing the math, many of us would not speculate on how much of the changes in land and sea temperatures are due to just UHI.

  16. Larry Huldén
    Posted Jun 6, 2009 at 11:09 AM | Permalink

    RE: tty (June 6th, 2009 at 1:44 am)

    “Actually there is a saying around that fortran is like latin, there is a lot of it still around, but it is a dead language.”

    Latin is NOT A DEAD LANGUAGE !!!

    It is the only formally acknowledged language in several sciences (Zoology, Botany, Microbiology, Bacteriology, Virology etc. etc. etc.)

    Larry Huldén
    Finnish Museum of Natural History

    • D. Patterson
      Posted Jun 6, 2009 at 11:42 AM | Permalink

      Re: Larry Huldén (#20),

      Latin is NOT A DEAD LANGUAGE !!!

      The only reason it’s still sittin’ on its perch, is that you NAILED it there….

    • tty
      Posted Jun 6, 2009 at 1:13 PM | Permalink

      Re: Larry Huldén (#20),

      It was not my intention to disparage latin, which in my experience is a quite useful language to know. However a language is considered dead when it no longer has any native speakers, which in the case of latin happened about 1500 years ago.

    • Posted Jun 6, 2009 at 4:41 PM | Permalink

      Re: Larry Huldén (#20),

      Latin is NOT A DEAD LANGUAGE !!!

      It is the only formally acknowledged language in several sciences (Zoology, Botany, Microbiology, Bacteriology, Virology etc. etc. etc.)

      And they end up mis-pronouncing most of it.

  17. AJ
    Posted Jun 6, 2009 at 12:29 PM | Permalink

    FORTRAN is for girlly men… Real programmers use COBOL:

    000100 IDENTIFICATION DIVISION.
    000200 PROGRAM-ID. HELLOWORLD.
    000300
    000400*
    000500 ENVIRONMENT DIVISION.
    000600 CONFIGURATION SECTION.
    000700 SOURCE-COMPUTER. RM-COBOL.
    000800 OBJECT-COMPUTER. RM-COBOL.
    000900
    001000 DATA DIVISION.
    001100 FILE SECTION.
    001200
    100000 PROCEDURE DIVISION.
    100100
    100200 MAIN-LOGIC SECTION.
    100300 BEGIN.
    100400 DISPLAY ” ” LINE 1 POSITION 1 ERASE EOS.
    100500 DISPLAY “Hello world!” LINE 15 POSITION 10.
    100600 STOP RUN.
    100700 MAIN-LOGIC-EXIT.
    100800 EXIT.

  18. Ryan O
    Posted Jun 6, 2009 at 2:04 PM | Permalink

    I used to know assembly language. TAKE THAT, COBOL-LOVERS! :)

    • TAG
      Posted Jun 6, 2009 at 3:57 PM | Permalink

      Re: Ryan O (#25),

      I used to know assembly language. TAKE THAT, COBOL-LOVERS

      In the original Terminator movie, Arnold saw computer code in his fileld of vision. It was 6809 assembler. If it is good enough for Arnold then it is good enough for me.

    • John
      Posted Jun 8, 2009 at 4:49 PM | Permalink

      Re: Ryan O (#25), Ah, but WHICH assembly language? Each processor family had their own flavor. I remember some debates of nearly religious fervor over Motorola vs. Intel.

      • Ryan O
        Posted Jun 8, 2009 at 5:22 PM | Permalink

        Re: John (#89), Intel, 8086 processor. But the whole 8088 thing and its stupid 8-bit bus soured me on it, because the 8086 stuff didn’t directly translate. After that, I played with FORTRAN and BASIC. I became quite fluent in BASIC, okay in FORTRAN. I didn’t bother programming anything else until I had to learn PHP for some web stuff, and then I decided to learn R because of the Steig paper.

  19. jeez
    Posted Jun 6, 2009 at 2:14 PM | Permalink

    Assembly language?

    LUXURY!

    Ya pampered toffs. Next you’ll all be braggin bout your electricity and all that.

    I remember as a lad envious of my cousin Phil, him with that string and two cans. I mean..

    LUXURY!

    All we had growin up was rocks. And not the little ones which go down easy mind you. I mean ROCKS. Big as Aunt Sally they was.

    When we needed to compile it was more damn work for a single bit on the register, than you puffs put in a year of your “Assembly Language”.

  20. MrPete
    Posted Jun 6, 2009 at 2:46 PM | Permalink

    F0RTRAN can be coded with punch cards, audio cassettes, DIP switches, toggle switches, even paper tape. The hard part is converting to something useful…
    :-D

  21. D. Patterson
    Posted Jun 6, 2009 at 3:40 PM | Permalink

    A nephew wanted a digital computer like the Apple II for Christmas of 1980. Gave him an abacus. It did have a nice marble base. Now they want clustered Playstations, X-Box, and the like to run numerical GCM.

  22. Posted Jun 6, 2009 at 4:42 PM | Permalink

    I learned on punch cards, but my first home computer was a TI-99/4a. 3k of memory, tape storage.

  23. David_a
    Posted Jun 6, 2009 at 7:06 PM | Permalink

    tag:
    The first 6800 on a board came with no assembler just a monitor that allowed u to key in opcodes then run. I still remember the opcode for load accumulator A.
    Seriously, it’s a very good way to learn how to program because higher level languages make sense when u understand what they are being translated into and why. It’s why it helps to have a handle on the linear algebra behind matrix decompositions before declaring yourself a climate scientist :)

  24. John F. Pittman
    Posted Jun 6, 2009 at 7:17 PM | Permalink

    Mine was a TI-99/4A with extended basic. Why? Because it emulated fortran to the point that I could do engineering problems at home, translate to fortran and save computer itme. At this time, part of your grade was determined by how much computer time you used. I got in an argument with a professor who said they real programming and programs occurred on mainframes and Vaxes; the PC would never be able to run Chemtran or other computer intensive programs. I stated that with memory and CPU power getting cheaper every year, PC’s would be the way to go. Graduated and did lots of engineering on PC’s and occassionaly a VAX, but no mainframes.

  25. Geoff Sherrington
    Posted Jun 6, 2009 at 8:07 PM | Permalink

    Anyone remember the Data General Nova (1969 was my first take-home one) and the Digital Equipment Corporation PDP8 series? Once flew Sydney to Los Angeles to a huge factory where many girls with good eyes were threading fine wires through donut ferrite cores. They fixed a couple of memories on the spot. A typical memory card was 4K (no misprint, not 4M). An early Basic program took up over 3 K. Machine language was hard to master, but very efficient.

    • compguy77
      Posted Jun 6, 2009 at 10:25 PM | Permalink

      Re: Geoff Sherrington (#35),
      Geoff,

      Loved the Nova. You had a whole 16 bits, not the puny 12 from Digital. Only problem was, you had to toggle in the bootstrap loader program after turning on the power. Human Bios was what it was!

      We repurchased a Nova that was stored in a warehouse in ’76. After cleaning off the mice droppings and nests we fired it up. The programs were still loaded in the donut memory from 4 years ago. Thing worked like a charm.

  26. BarryW
    Posted Jun 6, 2009 at 8:24 PM | Permalink

    The first computer I actually “saw” was a IBM1409, back in college. We never did get to run anything on it, the professor dropped the box of cards that had the Fortran compiler in it (it was actually a physics course). Ran my first jobs on Univac 1108’s and IBM 360’s. I remeber the memory cards with ferrite cores, worked on an airborne computer that was the size of a bread box with all of 16k of memory. Assembled the program on a 360 and loaded the actual computer with paper tape.

  27. David_a
    Posted Jun 6, 2009 at 8:33 PM | Permalink

    George
    I think in the geek olympics one of the events was fastest to toggle in the bootstrap loader on the front panel of the pdp10. If I recall it had an 18 bit bus and you needed both hands to make good time….

    • Geoff Sherrington
      Posted Jun 7, 2009 at 5:31 AM | Permalink

      Re: David_a (#37),

      You had a PDP10? You wealthy robber baron.

      And what happened when you toggled in just one wrong setting when booting? Remember? Go back to startRe: Anne T Cyclone (#42), .

  28. Anthony Watts
    Posted Jun 6, 2009 at 8:49 PM | Permalink

    I coded Fortran on punched cards, and also on paper tape of an ASR-33 teletype. Then we got decwriters and thought “we are really coding now”.

    The cruelest prank I ever saw in college was guy that got even with his roommate for some issue by holding the uncoiled paper tape of a big program outside the third floor dorm window and having his buddy set fire to the bottom of the tape.

    The flame was brief, but large.

    The poor guy had to recode from scratch.

    I guess today the same prank is to flush the USB flash drive down the toilet.

    • pjm
      Posted Jun 6, 2009 at 10:33 PM | Permalink

      Re: Anthony Watts (#38), then as now, Backup, backup, backup.

      Re: Anne T Cyclone (#19), the graph indicates that land temperature measurements have increased more than ocean temperature measurements, not from below to above. As the article states, this may be due to CO2 etc (AGW) or it may be other warming around the land-based thermometers. There has been much discussion on CA about the quality of some land temperature stations. As the article does not say, it may be the general pattern with a natural temperature rise, or it may be that UHI has not been fully allowed for. Another complicating factor is the change in methods of measuring ocean temperature, which may have affected the SST values.

  29. Alan Wilkinson
    Posted Jun 6, 2009 at 10:11 PM | Permalink

    The saddest face I ever saw outside our computer centre had just dropped a box of 2000 cards in a howling norwest gale.

  30. david_a
    Posted Jun 7, 2009 at 8:07 AM | Permalink

    Hi Geoff (sorry i called you george before, i have trouble with names)
    I didn’t know that the 10’s were only available to the upper classes. It wasn’t mine but belonged to a startup company that I worked for when I dropped out of school. They had a product which was a box made from 8000 TTL chips that was used for laboratory automation. It sat between machines that output either analog signals or RS232 and the PDP11. There was also a card which plugged into the bus of the pdp. We eventually replaced the whole box with a 6800 and a couple of more cards as a to d converters started showing up on single chips along with rs232 ports. Life was opcodes and oscilloscopes. I started off in the hardware end but migrated towards software as the hardware was overtaken by off the shelf stuff. I spent about 10 years in the program for hire field mostly with real-time operating systems and application development tools. Somewhere along the line I did a project for a small hedge fund that was in the commodity space and got lured into the world of finance and modeling. Its been twenty years since. Now I spend my days building prediction machines.
    Its a much different world than the academic one of which climate modeling is certainly a strange part. Unlike academia, no one really cares about your pedigree or lack thereof (good for me) and there is little genuflecting or appeal to authority. Its kind of like a continuous world series of poker though the players hardly interact at all except through their machine proxies and the occasional ‘conference’ which is really a marketing get together with sometimes edible food and drink. One of the things that is clear is that winning in the finance game is much more about looking at processes and data from a different perspective than being mathematically adept. Certainly you need to be able to draw straight lines and have a basic feel for the tradeoff between backcasting with more degrees of freedom and forecasting with less. My experience has been that things that ‘work’ seem to do so because the data that you model with is hard come by or hard to handle because of its sheer volume. I would suspect that building an outperforming equity model using daily closing prices is nigh on impossible given the amount of brain and compute power that has already been applied to the problem.
    I am a huge fan of this site and that of JeffId’s. In addition to it being a continuing education its great fun to watch the interactions between the ‘amateurs’ and the ‘professionals’ though unfortunately the professionals don’t show up nearly often enough. I guess there is really not a lot of upside in it for them and a whole lot of downside. Kind of sucks when you are a proclaimed expert and an ‘amateur’ looks at your work product and shreds it. Especially when they are right.
    I have always thought that a lot of the ‘science’ would get cleaned up in a hurry if there was a good (non-politcal) way to bet on the outcome of the models. For example, if you could get a futures contract up and running which paid off against say an index of ocean heat content which is probably the best measured and by far the most important global metric available, you would find out in short order what the market really thought about AGW. I know without writing a single line of code I would take the under against the mean IPCC forecast and do it in size.

  31. BarryW
    Posted Jun 7, 2009 at 9:17 AM | Permalink

    Forgot to mention the Univac 1219’s which had a speed control so you could slow down program execution and watch the registers change from lights on the front panel. You could also change the values in the registers from the panel.

  32. John S.
    Posted Jun 7, 2009 at 6:44 PM | Permalink

    The side-benefit of punch-card program coding in any language was that one became an expert in the tensile strength of rubber just to survive. Prior to that, matrix inversion was done mechanically on state-of-the-art Olivetti Tetractys (ka chunk, ka chunk, ka chung for 10 seconds for a single division). Its 20kg weight would be lugged from office to some remote missile testing site. Built up the abs, you know. Modern computing offers no such benefits to sissies with laptops.

  33. Gary Strand
    Posted Jun 7, 2009 at 10:26 PM | Permalink

    I’m not interested in ageist arguments – they’re an irrelevant tangent. Suffice to say I’m older than 43, so your snarky comment was unnecessary and silly.
    I’ve never seen Fortran as icky as much too much C that I’ve seen.

    • Steve McIntyre
      Posted Jun 7, 2009 at 10:43 PM | Permalink

      Re: Gary Strand (#55),

      Have you looked at HAnsen’s code?

      • Gary Strand
        Posted Jun 7, 2009 at 10:49 PM | Permalink

        Re: Steve McIntyre (#56), I can’t say that I have. Try this fun little bit of “C” (you know, another medieval language) and see what it does:

        #include
        main(t,_,a)
        char *a;
        {
        return!0<t?t<3?main(-79,-13,a+main(-87,1-_,main(-86,0,a+1)+a)):
        1,t<_?main(t+1,_,a):3,main(-94,-27+t,a)&&t==2?_<13?
        main(2,_+1,”%s %d %d\n”):9:16:t<0?t<-72?main(_,t,
        “@n’+,#’/*{}w+/w#cdnr/+,{}r/*de}+,/*{*+,/w{%+,/w#q#n+,/#{l+,/n{n+,/+#n+,/#\
        ;#q#n+,/+k#;*+,/’r :’d*’3,}{w+K w’K:’+}e#';dq#’l \
        q#’+d’K#!/+k#;q#’r}eKK#}w’r}eKK{nl]’/#;#q#n’){)#}w’){){nl]’/+#n';d}rw’ i;# \
        ){nl]!/n{n#'; r{#w’r nc{nl]’/#{l,+’K {rw’ iK{;[{nl]’/w#q#n’wk nw’ \
        iwk{KK{nl]!/w{%’l##w#’ i; :{nl]’/*{q#’ld;r’}{nlwb!/*de}’c \
        ;;{nl’-{}rw]’/+,}##’*}#nc,’,#nw]’/+kd’+e}+;#’rdq#w! nr’/ ‘) }+}{rl#'{n’ ‘)# \
        }’+}##(!!/”)
        :t<-50?_==*a?putchar(31[a]):main(-65,_,a+1):main((*a==’/’)+t,_,a+1)
        :0<t?main(2,2,”%s”):*a==’/’||main(0,main(-61,*a,
        “!ek;dc i@bK'(q)-[w]*%n+r3#l,{}:\nuwloca-O;m .vpbks,fxntdCeghiry”),a+1);
        }

        • Kusigrosz
          Posted Jun 8, 2009 at 9:28 AM | Permalink

          Re: Gary Strand (#57), This looks like an example of deliberately obfuscated C code (from IOCC 1988, quoted in the Wikipedia entry for “obfuscated code”). Not a good comparison standard for programs intended to do something useful.

        • Thor
          Posted Jun 9, 2009 at 9:51 AM | Permalink

          Re: Gary Strand (#57),

          Finding the poem in the obfuscated C code you provide is certainly possible. However, the poem would probably reach a wider audience if written in plain English.

          Now, why the GISS code had to be written in obfuscated Fortran is beyond me. Using non-obfuscated Fortran or a different, higher level language would both have been much better choices.

    • TAG
      Posted Jun 8, 2009 at 4:12 AM | Permalink

      Re: Gary Strand (#55),

      I’ve never seen Fortran as icky as much too much C that I’ve seen

      C is not a modern language

      We had some consultants once who were convinced that technology reached its ultimate expression around 1980 when Smalltalk was invented. There was no higher concept than that of the Object. Fortran and Unix apprarently have the same sort of adherents. Technology moves on and the programming model of Fortran has been left behind in the 1950s when Fotran was invented. I learned Fortran in teh 60s. I wouldn’t use it now

  34. Geoff Sherrington
    Posted Jun 8, 2009 at 1:10 AM | Permalink

    Are you sure this should not read ((*a=’=’/’)+ rather than ((*a==’/’)+.
    (Don’t answer this, I’m fooling around. You made your pont brilliantly as usual.)

  35. Mike Lorrey
    Posted Jun 8, 2009 at 3:29 AM | Permalink

    Argh, I chipped ones and zeros out of flint with an antelope knee bone.

    • John S.
      Posted Jun 8, 2009 at 6:41 AM | Permalink

      Re: Mike Lorrey (#59),

      Respect! You must be a contemporary of the mentor who taught me so much on my first job.

      Re: TAG (#60),

      Fortran is undoubtedly burdened by many coding limitations and inefficencies. Nevertheless, for certain scientific computations it remains the language of choice, if for no other reason than the numerical strength of its library routines. Try computing Bessel functions accurate to 15 digits and see how far modern script languages will take you. (On the other hand, Microsoft Fortran can’t even get atan2(x,y) right to 3 digits without going to double precision.)

  36. jeez's dad
    Posted Jun 8, 2009 at 4:25 AM | Permalink

    Flint…

    FLINT???? Antelope knee bones????

    Well your pampered, smooth skinned, rollin in riches, royal highness, if we’d have had flint to work with when I was compootin’ we’d of been to Jupiter and back by now-twice.
    All me family had for computation was a two inch mud puddle. I had to start entering data two hours before I woke up, compile 27 hours a day and finish 3 hours after I went to bed then drink the mud for dinner while I slept. Then me Dad would beat me while with a broken bottle if I was lucky (like I didn’t wish I could use that for me calculations)

  37. pjm
    Posted Jun 8, 2009 at 6:08 AM | Permalink

    As demonstrated above (and in other places) you can write bad code in any language. For that matter you can write good code in any language too, but some make things easier than others. Of course languages change. Compare a line number BASIC with the latest VB. (Visual Basic, not Victoria Bitter). The problem is still that when you have old code you are likely to keep re-using it until you are dragged kicking & screaming to a new language. And you keep whingeing for a while after you have decided the new stuff is great, just on principle.

    BTW There was a story in Analog Science Fiction magazine many years ago connecting the Tower of Babel with computers using hard disks, being cuneiform tablets. Floppy disks (papyrus) just weren’t as durable. BASIC stood for Babylonian-Assyrian something or other. Now that was a hard disk crash!

  38. david_a
    Posted Jun 8, 2009 at 6:52 AM | Permalink

    Mostly agree with Steve’s take on languages though I would take issue with his characterization of UNIX. Since the only widespread alternative in use is Windows unless you want to support your own operating system there really is no choice.
    I think he made the correct observation that at least for data analysis type applications it makes sense to use something designed to manipulate matrixes and vectors natively. It removes the indexing from view which is very important because the reader does not know what the indexing is for until they have read the whole routine.

    If you are taking say a sum of each column in a matrix which you would then store in a vector you could write in c:

    for (i = 0; i < numcols; i++)
    {
    sum = 0;
    for (j = 0; j < numrows; j++
    {
    sum += matrix[j * numcols + i];
    }
    vector[i] = sum;
    }

    you could turn this into a subroutine so that anytime you wanted to do the operation it became

    col_sum(matrix, vector, numrows, numcols)

    so if all of your ‘operators’ were coded this way the code would be quite readable, but the allocation of objects would have to be done explicitly as would the choice between call by value and call by reference.

    Another simplification you could make is to restrict the choice to call by value and not allow call by reference. If results are only returned syntactically by assignment then the language is easier to parse with the human eye. In addition there are no side effects because things change out from under you.

    so the col_sum subroutine would return a vector rather than to pass it as an argument which is then filled.

    vector = col_sum(matrix, numrows, numcols);

    if a matrix and vector are truly native data types which keep track of their own sizes then things could get simpler yet and you would have

    vector = col_sum(matrix);

    this would now allow for the simple nesting of function calls so say you had a function which displayed the contents of a vector you could then write

    display(col_sum(matrix));

    if you allowed for the overloading of operators so the language understood the objects and applied the correct subroutine depending on the objects with which it is was called, and you allowed the language to dynamically create objects by type as they were required things could get pretty intuitive.

    matrix = matrix_create(3,3); // create a matrix with 3 rows and 3 columns.
    matrix = 5; // set all values equal to five
    display(matrix);
    matrix[0] += 2; // add 2 to all the values in colum zero
    matrix[1,1] = 4; // set row 1, col1 to 4
    display(matrix);
    display(col_sum(matrix)); // print the sum of each column

    5 5 5
    5 5 5
    5 5 5

    7 5 5
    7 4 5
    7 5 5

    21 14 15

    finally if you made the whole thing interpreted rather than compiled the development time would speed up considerably and there would be effectively no penalty in execution because the actually operator calculating code would all be written in some language that was closer to the machine like c. Each time you wished to add another native operation or native datatype to the language you would write the appropriate c code and recompile your language.

    • Steve McIntyre
      Posted Jun 8, 2009 at 7:29 AM | Permalink

      Re: david_a (#64),

      People are losing sight of the underlying point and perhaps I didn’t make it clearly enough.

      In this post, I presented tools for reading Hansen’s online data in a modern language. It required a not insignificant amount of time and ingenuity to develop these tools. I do not believe that the average person who might otherwise be interested in Hansen’s results would necessarily have the time or energy or background knowledge to figure out how to decode the data.

      Gary Strand’s answer – learn Fortran.

      Look, I know enough Fortran to know that I don’t want to use Fortran to access online data sets.

      Nor in 2009 is it reasonable for a data provider to assume that all interested parties are going to be Fortran users on Unix. They could easily compress the data available in *.gz.

      I provided some utilities here for people to access Hansen’s data in R (and which could be adapted to Matlab, I presume.) It was open to Gary to observe that this was a small but useful contribution to making data accessible, but he chose not to, while not criticizing Hansen’s decision to make data available only in binary.

      • John S.
        Posted Jun 8, 2009 at 7:46 AM | Permalink

        Re: Steve McIntyre (#67),

        There can be no question that data distributed to the public should be coded in standard ASCII. Even Fortran readily recognizes such coding.

        • Gary Strand
          Posted Jun 8, 2009 at 7:57 AM | Permalink

          Re: John S. (#68),

          There can be no question that data distributed to the public should be coded in standard ASCII. Even Fortran readily recognizes such coding.

          ASCII? You’ve got to be joking.

        • John S.
          Posted Jun 8, 2009 at 8:21 AM | Permalink

          Re: Gary Strand (#69),

          You deserve an eight-pack of beer!

  39. david_a
    Posted Jun 8, 2009 at 6:59 AM | Permalink

    JohnS:
    There is no reason that the fortran libraries which you speak of can not be called thru some other method other than using a fortran compiler and linking directly. Just because there may be some great numerical routines written in some language does not suggest that one must be tied to that language to use them. In the end they are just object files which have some stack frame conventions which can be wrapped. Now this changes radically if the language is more of a run time environment like c# or some such nonsense. Then u are generally up the creek unless you want to spend your life wading thru manuals

    • John S.
      Posted Jun 8, 2009 at 7:26 AM | Permalink

      Re: david_a (#65),

      The issue goes beyond mere calling or wrapping of library object files. It’s Fortran’s ability to create arbitrarily higher orders of precision within the program that allows the computation of pi to a ridiculous number of digits. I’m not aware of this ability in other languages, but programming is just a tool for me, rather than my professional field.

  40. Gary Strand
    Posted Jun 8, 2009 at 8:01 AM | Permalink

    The data provided by NASA could be in netCDF, but that’s not an obligation.

    Providing an interface via “R” is nice only for folks that use “R” – would you, Steve, recommend that everyone learn “R”?

    • Steve McIntyre
      Posted Jun 8, 2009 at 2:42 PM | Permalink

      Re: Gary Strand (#70),

      The data provided by NASA could be in netCDF, but that’s not an obligation. Providing an interface via “R” is nice only for folks that use “R” – would you, Steve, recommend that everyone learn “R”?

      I would certainly recommend learning R to anyone interested in statistical analysis. If you were employed as a retail clerk or pipefitter and did not have a specific interest in statistical analysis, I would not recommend going to the trouble of learning R.

      If I were managing a group carrying out statistical analyses today, I would require the people in the group to use R simply because Fortran is not sufficiently productive.

      If I were publishing data for users across many platforms, I would recognize that some users use language that I would not recommend and make the data available in an ASCII form, rather than as compressed R objects, and have done so in connection with my own publications.

      • Gary Strand
        Posted Jun 8, 2009 at 3:19 PM | Permalink

        Re: Steve McIntyre (#78),

        If I were managing a group carrying out statistical analyses today, I would require the people in the group to use R simply because Fortran is not sufficiently productive.

        Based on what metric?

        • Steve McIntyre
          Posted Jun 8, 2009 at 3:33 PM | Permalink

          Re: Gary Strand (#79),

          Gary, please withdraw your untrue allegation made on the other thread that I had altered comments to make you look foolish.

        • Ryan O
          Posted Jun 8, 2009 at 3:34 PM | Permalink

          Re: Gary Strand (#79), Based on the fact that R rules.
          .
          On a serious note, R is powerful, easy to understand, has a professional look to the output of the graphics package, has a wide following, has custom packages for all kinds of field-specific uses, doesn’t require the use of proprietary formats, can read most formats (and can be configured to read the rest if you want) and it’s free. Accessible to everyone. No worries with multiple, expensive licenses.

        • Gary Strand
          Posted Jun 8, 2009 at 3:50 PM | Permalink

          Re: Ryan O (#81),

          On a serious note, R is powerful, easy to understand, has a professional look to the output of the graphics package, has a wide following, has custom packages for all kinds of field-specific uses, doesn’t require the use of proprietary formats, can read most formats (and can be configured to read the rest if you want) and it’s free. Accessible to everyone. No worries with multiple, expensive licenses.

          Fortran gives you all that – excepting the plotting, but then you’re not tied into “R”‘s style. The other benefit is that you can trace every bit – no black box routines. That “R” is built on Fortran is to “R”‘s favor.

        • Ryan O
          Posted Jun 8, 2009 at 4:04 PM | Permalink

          Re: Gary Strand (#82), You’re seriously arguing that it is better to write your own routines in FORTRAN than use a higher-level interface? What purpose does that serve, except to potentially invalidate a whole bunch of results because people made simple coding errors?
          .
          And you are seriously contending that something written in FORTRAN is easier to audit and follow than something written in R? I would recommend you spend a few minutes at Dan Hughes’ blog as he has attempted to do just that with GISS ModelE – without much success. I would argue that the lower-level languages are significantly less conducive to auditing than higher-level languages like R.

        • Gary Strand
          Posted Jun 8, 2009 at 4:11 PM | Permalink

          Re: Ryan O (#84),

          You’re seriously arguing that it is better to write your own routines in FORTRAN than use a higher-level interface?

          Relying on a package like “R” does require a bit of faith. How much of “R”‘s source is auditable?

          I would argue that the lower-level languages are significantly less conducive to auditing than higher-level languages like R.

          I disagree.

        • Greg F
          Posted Jun 8, 2009 at 4:12 PM | Permalink

          Re: Gary Strand (#82),

          That “R” is built on Fortran is to “R”‘s favor.

          R is not built on Fortran, it’s core was written in C.

        • Gary Strand
          Posted Jun 8, 2009 at 4:20 PM | Permalink

          Re: Greg F (#87),

          R is not built on Fortran, it’s core was written in C.

          To build “R” from scratch, one needs a Fortran compiler – according to the “R” FAQ.

        • Earle Williams
          Posted Jun 8, 2009 at 5:07 PM | Permalink

          Re: Gary Strand (#88),

          Your sin of omission was to neglect to quote the entire paragraph:

          Note that you need a FORTRAN compiler or perhaps f2c in addition to a C compiler to build R. Also, you need Perl version 5 to build the R object documentations. (If this is not available on your system, you can obtain a PDF version of the object reference manual via CRAN.)

          as stated at http://cran.r-project.org/doc/FAQ/R-FAQ.html

          The R package does use some FORTRAN but is indeed primarily C. The source code available at http://cran.r-project.org/src/base/R-2/R-2.9.0.tar.gz (should answer your question above about auditing the code) contains all of the FORTRAN and C code. A quick check on the file extensions of the source files yields:

          earle@feynman:~/R$ ls -lR | grep –count ‘\.c’
          562
          earle@feynman:~/R$ ls -lR | grep –count ‘\.f’
          88

          FORTRAN still has a role in the software world. FORTRAN as practiced by GISS does not, and it should die a quick death.

          As to your assertion that you have a sense of humor I am reminded of the Dunning-Kruger effect. But then again, perhaps I misread your statement. Judging from other posters here there is great humor associated with your comments.

        • Kusigrosz
          Posted Jun 8, 2009 at 7:21 PM | Permalink

          Re: Earle Williams (#90), Doesn’t

          ls -lR | grep –count ‘\.f’

          also count files with names like ‘choose.files.R’ and similar? Executing

          find . -name ‘*.f’ | wc

          yields 45 F files; for C the number is 511. However,

          wc `find . -name ‘*.f’`

          counts 175395 F lines (5.6 MB); for C it is 319193 (9.3MB), so the dominance of C is not that overwhelming.

          This of course only shows that FORTRAN can be used for things like writing R modules, not that it can compete with R in R’s main field.

        • Gary Strand
          Posted Jun 8, 2009 at 7:29 PM | Permalink

          Re: Kusigrosz (#94),

          This of course only shows that FORTRAN can be used for things like writing R modules, not that it can compete with R in R’s main field.

          “R” is basically a wrapper around C and Fortran routines.

          One can replicate all of “R” with Fortran; one cannot replicate Fortran with “R”.

        • Ryan O
          Posted Jun 8, 2009 at 7:40 PM | Permalink

          Re: Gary Strand (#96), So what? Again, why would do you feel that having a bunch of individual people write their own FORTRAN code when they can all use – in your words – the wrapper for FORTRAN? What sense does that make?

        • Gary Strand
          Posted Jun 8, 2009 at 8:21 PM | Permalink

          Re: Ryan O (#98),

          Again, why would do you feel that having a bunch of individual people write their own FORTRAN code when they can all use – in your words – the wrapper for FORTRAN?

          No need for all UPPERCASE in describing Fortran any more.

          “R” is a fine package. It’s not a replacement for Fortran.

        • Willis Eschenbach
          Posted Jun 8, 2009 at 7:49 PM | Permalink

          Re: Gary Strand (#96), I’m not sure what your point in all of this is. Is it that Fortran is better than R? Is it that we should not diss Fortran?
          .
          As someone who learned to program in 1963, I have a few years and far too many computer languages under my belt. I learned R at the urging of Steve Mc. and I have never looked back. Yes, I can program the exact same thing in Fortran … but why on earth should I? Yes, any function in any language can be written in most other languages … but again, why? Things that I can do in one line in R take a half page in Fortran. Computer languages are generally optimized to do one thing well. Fortran, as the name suggests, was built as a scientific language for FORmula TRANslation. COBOL (COmmon Business Oriented Language) was built for business applications.
          .
          R, on the other hand, is optimized to do statistical analyses and mathematical transformations on large datasets. I don’t write business programs in R, but then I don’t write business programs in COBOL any more either. I use whatever language is best for the particular application … which for climate science is definitely not FORTRAN.
          .
          So … just what is it that you are advocating here?
          .
          w.

        • Gary Strand
          Posted Jun 8, 2009 at 8:23 PM | Permalink

          Re: Willis Eschenbach (#99),

          I use whatever language is best for the particular application … which for climate science is definitely not FORTRAN.

          How do you figure?

          “R” couldn’t exist without the Fortran routines it uses.

        • Greg F
          Posted Jun 8, 2009 at 8:44 PM | Permalink

          Re: Gary Strand (#101),

          “R” couldn’t exist without the Fortran routines it uses.

          Oh my. It most certainly could. The reason it has Fortran is because R was written in the early 90’s. The Fortran routines work just fine. So, If it ain’t broke don’t fix it.

          If you go look at the documentation you will find a number of reference to the Fortran 77 compiler and early problems with a substitute compiler f2c. f2c is just short for FORTRAN-to-C converter.

          As further evidence:

          r40362 | ripley | 2007-01-04 09:11:53 -0500 (Thu, 04 Jan 2007) | 2 lines
          Changed paths:
          M /trunk/NEWS
          M /trunk/src/appl/fmin.c
          M /trunk/tests/reg-tests-1.R

          Fix PR#9438, make fmin.c a closer translation of the original Fortran

          The Fortran is getting gradually replaced with C code.

        • Gary Strand
          Posted Jun 8, 2009 at 11:07 PM | Permalink

          Re: Greg F (#104),

          Oh my. It most certainly could. The reason it has Fortran is because R was written in the early 90’s. The Fortran routines work just fine. So, If it ain’t broke don’t fix it.

          I did a quick perusal of the R source and found literally hundreds of good ol’ F77 routines. I doubt R would work at all without them.

          Steve: One more time, so what? That doesn’t mean that people should do statistical analysis in Fortran or that Hansen should archive in binary.

        • pjm
          Posted Jun 9, 2009 at 12:17 AM | Permalink

          Re: Gary Strand (#109),
          Gary, Fortran is good. R is good. R without Fortran behind it might not have been as good, and would have needed a lot more work.

          People here complain about non-archivers of code and data. If you do the right thing you are not being complained about. If the cap don’t fit, don’t wear it.

          Steve,

          The Matlab code in Mann 2008 is heavily influenced by Fortran and, at times is pretty much equivalent to Fortran in layout

          It is very hard to break habits, in programming as much as anything else. When the C64s and Apples had BASIC built-in, programming lecturers at some schools preferred their students to have had no experience, rather than to have used BASIC.

          Real Programmers can write Fortran in any language! Real Programmers Don’t Use Pascal

        • Steve McIntyre
          Posted Jun 9, 2009 at 6:44 AM | Permalink

          Re: pjm (#112),

          Real Programmers can write Fortran in any language!

          Perfect. So can realclimatologists.

        • Greg F
          Posted Jun 8, 2009 at 5:20 PM | Permalink

          Re: Gary Strand (#88),

          To build “R” from scratch, one needs a Fortran compiler

          Which means there is Fortran code. Go download the source. From the Windows source go to:

          R-devel_2009-06-07\R-devel\src\main

          “Main” is the core of the application. It’s all C code.

  41. david_a
    Posted Jun 8, 2009 at 9:00 AM | Permalink

    John:
    Arbitrary precision arithmetic on most machines nowadays is a software implementation. It is not at all unique to fortran. On all the desktop PC’s the floating point processor is a 64 bit machine so if you want to get beyond that for precision you have to implement it in software. I believe that some of the early IBM machines had hardware capability to do very large (512 bit?) calculations. There are many open source libraries of arbitrary precision arithmetic with the GNU one having C bindings. I am by no means even mildly expert in the field of numerical computation but it has been my experience that if you need more than 64 bits of precisions to hold onto numerical stability there is probably a better way to formulate your problem. The only field that I know of that routinely needs this stuff is cryptography…..

    Steve:
    I am in complete agreement with you as to your thoughts about data archiving. It should be done as a matter of course and with the greatest transparency and ease of use for would be users of the data. I would think that a professional society would be a good place to start so that the membership would be encouraged to follow along some set of dictum.

    Were I inclined to divert time away from my current pursuits I would probably start with R just because you guys have though I have a home grown modeling language that is probably pretty similar. I have installed a copy of R on my linux machine but haven’t yet had the time to take the plunge other than to try out a few test scripts to see that it worked.

    As far as Hansen’s making the data available in other forms other than binary I don’t take too much issue with that though I have no idea as to what the documentation looks like. If it was provided in NETCDF that is well and good if you are using a package that supports it otherwise you have to write some chunk of code to unpack the stuff. I looked at it because I had downloaded some ocean temp data from NOAA and it was available in NETCDF format. It was also available in binary. I spent a few minutes with the NETCDF documentation and didn’t feel like writing a parser to interpret it for my environment so I used the binary files instead. I have not looked but there is probably some translator available which moves the data from one form to another. If not someone who has a grad student should give him/her a project :)

    Switching topics, I know that there is a kriging package available for R, and was wondering if you had ever had the chance to do an antarctic reconstruction using it on ground station data only? If you are wandering around looking at mineral deposits I’m guessing that you are more than familiar with it. Not that you need more to do but it would make for an interesting comparison to JeffId’s area weighted method.

    • John S.
      Posted Jun 8, 2009 at 9:49 AM | Permalink

      Re: david_a (#72),

      After submitting my comment, I immediately thought of C as another word-manipulating language that makes implementation of arbitrary orders of precision quite convenient. While inept numerical programming no doubt is endemic and the capabilities of 64-bit words are not fully exploited, there are numerous scientific problems besides those of cryptography that require more than double precision for accurate results. Fortran is the language in which most of the successful solutions to numerically challenging problems have been coded. To my limited knowledge, modern script languages, whatever their unmistakable advantages in ordinary circumstances, are not as well-adapted to such challenging tasks.

      All of this, of course, has little to do with any processing or dissemination of data by GISS or Hadley Centre.

  42. Posted Jun 8, 2009 at 12:11 PM | Permalink

    Only in climatology can you generate a 74 response forum post by simply…

    providing raw data in an alternative format.

  43. Kenneth Fritsch
    Posted Jun 8, 2009 at 3:56 PM | Permalink

    Gary Strand, I could care less about the software issues here other than to address the issue of accessibility to online data for both professional and amateur studies and analyses and whether one agrees that those studies/analyses can be a good thing in furthering the science.

    What is it that you are defending or advocating? It would appear to get lost in the software debate. Do you agree that easy accessibility for all potential users is the important issue or do you feel that the too easily accessible data might be abused?

    • Gary Strand
      Posted Jun 8, 2009 at 4:09 PM | Permalink

      Re: Kenneth Fritsch (#83),

      Do you agree that easy accessibility for all potential users is the important issue or do you feel that the too easily accessible data might be abused?

      I’ve spent the last 7+ years helping to make data readily available to users – at last count, over 2,000.

      • Kenneth Fritsch
        Posted Jun 8, 2009 at 6:34 PM | Permalink

        Re: Gary Strand (#85),

        I’ve spent the last 7+ years helping to make data readily available to users – at last count, over 2,000.

        Please expand on this, as I, and I suspect others here, are not familiar with this work.

        Do you judge that more efforts should be made to increase the accessibility of data sources to the general public and if so what would you prescribe?

        • Gary Strand
          Posted Jun 8, 2009 at 7:33 PM | Permalink

          Re: Kenneth Fritsch (#93),

          Please expand on this, as I, and I suspect others here, are not familiar with this work.
          Do you judge that more efforts should be made to increase the accessibility of data sources to the general public and if so what would you prescribe?

          I’ve been involved with ESG since mid-2001. It has since expanded beyond just PCM data to include CCSM and served as the template for the CMIP3 archive. It will be expanded again for CMIP5. I’ve also been involved with the distribution of CCSM data via the GIS portals at NCAR and the UNH.

          We’ve worked very hard to make CCSM and PCM climate model data available to anyone who’s interested. That’s why I get chapped, a bit, when I read CA commentators claim that “data” is impossible or difficult to access.

        • Steve McIntyre
          Posted Jun 8, 2009 at 8:29 PM | Permalink

          Re: Gary Strand (#97),

          That’s why I get chapped, a bit, when I read CA commentators claim that “data” is impossible or difficult to access.

          The fact that you’ve made data available is excellent but is irrelevant to the question of whether Phil Jones, Lonnie Thompson, Michael Mann, Keith Briffa, Ben Santer, etc etc have refused to provide data. Their refusals have been well documented.

          Well, Gary, if you get “chapped a bit” at their refusals to provide data even under FOI requests, then your complaint lies with Phil Jones, Lonnie Thompson, Michael Mann, Keith Briffa, Ben Santer, etc and not with us.

          To the extent that you have implied that our criticisms ( of Jones ec) are unfounded is invalid and once again, I invite you to clarify or withdraw your untrue allegation.

        • Gary Strand
          Posted Jun 8, 2009 at 10:42 PM | Permalink

          Re: Steve McIntyre (#102),

          The fact that you’ve made data available is excellent but is irrelevant to the question of whether Phil Jones, Lonnie Thompson, Michael Mann, Keith Briffa, Ben Santer, etc etc have refused to provide data.

          Seeing as how I’m none of those individuals, complaining to me about them doesn’t make sense.

          The efforts of myself and others have made far more climate model data available to far more people than ever before; to dismiss our efforts as “irrelevant” because of your issues with specific individuals is unfair. Additionally, the tarring of all climate scientists with a broad brush, as practiced by some here at CA, is also unfair.

        • Steve McIntyre
          Posted Jun 8, 2009 at 11:29 PM | Permalink

          Re: Gary Strand (#108),

          Gary, you seem to be hard-wired into non-sequiturs. Re: Gary Strand (#108),

          The efforts of myself and others have made far more climate model data available to far more people than ever before; to dismiss our efforts as “irrelevant” because of your issues with specific individuals is unfair.

          I didn’t dismiss your efforts as “irrelevant”; I said that such efforts were “excellent”. However, they were irrelevant to whether Jones, Mann, etc had archived their data. And they do not justify the actions of the scientists in question. You seem to have trouble with reading comprehension.

          I said previously:

          The fact that you’ve made data available is excellent but is irrelevant to the question of whether Phil Jones, Lonnie Thompson, Michael Mann, Keith Briffa, Ben Santer, etc etc have refused to provide data.

          You said:

          Seeing as how I’m none of those individuals, complaining to me about them doesn’t make sense.

          Again, you seem to have trouble with reading comprehension and non-sequiturs. My statement above wasn’t “complaining” to you about those individuals – I was merely observing that your good behavior didn’t mean that the other boys hadn’t behaved badly and could not be put forward as a defence of the bad behavior of the other boys.

          I seriously think that you would benefit from some exercises in reading comprehension as your comments here are not making a lot of sense, I’m sorry to say. This is nothing to do with being critical – I don’t mind you criticizing me. I’m sorry to speak in these terms. I’ve got a long history of being patient with critics, but I’m really don;’t have time to deal with such dyslexic comments.

        • Gerald Machnee
          Posted Jun 9, 2009 at 7:05 AM | Permalink

          Re: Gary Strand (#108),

          Additionally, the tarring of all climate scientists with a broad brush, as practiced by some here at CA, is also unfair.

          Again, reading comprehension. ALL climatologists are not tarred here, only the one that deserve to be.

        • Gary Strand
          Posted Jun 9, 2009 at 8:07 AM | Permalink

          Re: Gerald Machnee (#118),

          ALL climatologists are not tarred here, only the one that deserve to be.

          I do see a lot of non-specific “climate scientists are (insert random bad trait)” commentary. It’s unfortunate.

        • Gerald Machnee
          Posted Jun 9, 2009 at 8:44 AM | Permalink

          Re: Gary Strand (#120),

          I do see a lot of non-specific “climate scientists are (insert random bad trait)” commentary. It’s unfortunate.

          Yes, it is unfortunate that too many of the top dogs (start with Phil Jones and Lonnie Thompson) do not archive and give the rest a bad name.

        • Gary Strand
          Posted Jun 9, 2009 at 9:48 AM | Permalink

          Re: Gerald Machnee (#122),

          Yes, it is unfortunate that too many of the top dogs (start with Phil Jones and Lonnie Thompson) do not archive and give the rest a bad name.

          Who’s giving all climate scientists a bad name? The folks with issues with Jones, et.al.

        • Gerald Machnee
          Posted Jun 9, 2009 at 11:57 AM | Permalink

          Re: Gary Strand (#124),

          Who’s giving all climate scientists a bad name? The folks with issues with Jones, et.al.

          You and the “Team” just do not get it. Over and Out.

  44. Gary Strand
    Posted Jun 8, 2009 at 7:28 PM | Permalink

    In looking over the “R” code, it uses quite a few well-tested and correct Fortran routines – LAPACK among them. Clearly, without Fortran, “R” wouldn’t work nearly as well as it does.

  45. David_a
    Posted Jun 8, 2009 at 8:38 PM | Permalink

    #96
    And fortran is a wrapper for assembler which is in turn a wrapper for the instruction set which in some cases is a wrapper for microcode. Continuing on maybe we should all burn programmable logic arrays so we can get right down to the nitty gritty and we can audit the code with microscopes so we can see if the transistors are all hooked up well

  46. theduke
    Posted Jun 8, 2009 at 8:47 PM | Permalink

    So … just what is it that you are advocating here?

    I think it’s obvious he’s not advocating anything. He’s exercising his right to quibble over any little ol thing. Which is unfortunate since he probably has much to offer if he chose to contribute in a thoughtful way.

  47. pjm
    Posted Jun 8, 2009 at 9:11 PM | Permalink

    Is everybody talking about the same thing? For statistical analysis, use R (or perhaps a better stats package if there is one). For climate modeling use Fortran, or C, or whatever you know and love. If you need better numerical libraries, include them. There is a good chance that your supercharged numerical library was written using Fortran anyway, as it’s easier and more portable than assembler.

    You’d think that GISS would welcome this sort of software, but they seem uninterested in anything other than Fortran.

    I suspect many climate scientists would get more done with less effort by using R instead of writing Fortran code for this type of work. That is, of course, after climbing the learning curve. But it doesn’t mean they could forget all their Fortran – there are plenty of other jobs where it is appropriate.

    Of course it may be they do try things out in R, then do it *properly* in Fortran.

    • Steve McIntyre
      Posted Jun 8, 2009 at 10:34 PM | Permalink

      Re: pjm (#106),

      Is everybody talking about the same thing?

      Not really,- mostly because of Gary Strand’s non-sequiturs.

      As a supposed answer to criticizing Hansen for archiving only in binary leading to an assertion that R is superior to Fortran for statistical analysis, he asserts that Fortran is better at something else or is used in a compiler.

      Which doesn’t deal with any issue that anyone had actually raised. He might as well asserted that elephants are grey.

      I suspect many climate scientists would get more done with less effort by using R instead of writing Fortran code for this type of work.

      Yep. Some are trying to use higher level languages, but dissipate the effort by continuing to write code with a heavy Fortran dialect. The Matlab code in Mann 2008 is heavily influenced by Fortran and, at times is pretty much equivalent to Fortran in layout – over-subscripted and over do-looped relative to the style that a “native” speaker would use.

      • Ryan O
        Posted Jun 9, 2009 at 5:32 AM | Permalink

        Re: Steve McIntyre (#107),

        The Matlab code in Mann 2008 is heavily influenced by Fortran and, at times is pretty much equivalent to Fortran in layout – over-subscripted and over do-looped relative to the style that a “native” speaker would use.

        Hey! That sounds like me . . . :)

        • pjm
          Posted Jun 9, 2009 at 6:14 AM | Permalink

          Re: Ryan O (#114),

          Given that R depends on Fortran, you’re doing analysis with Fortran, whether you consider it medieval or not.

          I don’t think anybody disagrees, but most are happy not to look at that level.

          And, it’s nice of Hansen to provide data at all

          I think that most people here agree that making the data available is not a matter of being nice, but of fulfilling responsibilities.

          he could have encoded it in EBCDIC.

          Nothing wrong with that. The conversion to more conventional text is quite well understood, and less ambiguous than if binary data was given with no supporting instructions (and I am NOT suggesting whether that did or did not happen). For that matter, any numerical data would be just binary-coded decimal – not hard at all.

    • Kenneth Fritsch
      Posted Jun 9, 2009 at 8:27 AM | Permalink

      Re: pjm (#106),

      Is everybody talking about the same thing?

      I have not received a satisfactory answer from Gary Strand on making climate science data more readily accessible to the general public. I think Gary is stuck somewhere thinking in terms of exchanging data and code between climate modelers and/or climate scientists.

      Gary, this thread was introduced pointing to the efforts that an unpaid interested party made to make the GISS data more accessible to the general public. What are your views on that work?

      I think also that Steve M was strongly indicating that (although in my layperson’s view Fortran and other programs at that level are probably fine for climate modeling but not practical for getting that data to the general public) what is of more interest to him and many of us here is more easily accessing that data at another level.

      I think, Gary, you would be better served by keeping the discussion at the level that Steve M initially targeted.

      • Gary Strand
        Posted Jun 9, 2009 at 9:47 AM | Permalink

        Re: Kenneth Fritsch (#121),

        I have not received a satisfactory answer from Gary Strand on making climate science data more readily accessible to the general public.

        I’ve already pointed you to the Earth System Grid> page; it and the CMIP3 portal at PCMDI have made more climate model data available to the general public than ever before.

        What would be a “satisfactory answer” to you?

        • Kenneth Fritsch
          Posted Jun 9, 2009 at 12:44 PM | Permalink

          Re: Gary Strand (#123),

          What would be a “satisfactory answer” to you?

          Instead of pointing to a link, how about with a few words of your own telling us how these data are accessed, by what formats, who generally uses the data and how would you improve access to it by more of the general public.

        • Gary Strand
          Posted Jun 9, 2009 at 1:35 PM | Permalink

          Re: Kenneth Fritsch (#130),

          Instead of pointing to a link, how about with a few words of your own telling us how these data are accessed, by what formats, who generally uses the data and how would you improve access to it by more of the general public.

          How are these data accessed? Via the ESG web interface, by and large. Also ‘wget’ for the data available on disk. All the data are in netCDF format, as that best suits our needs. Generally, the userbase from ESG is made up of university and governmental researchers, but also a fair number of commercial interests. We have users from all over the world.

          We’ve talked about converting the data to different formats – even Excel, but that’s not really practical for model output. I don’t believe the “general public” is really that interested in model output per se.

        • Posted Jun 9, 2009 at 1:51 PM | Permalink

          Re: Gary Strand (#132),

          Is the code available to the public?

        • Gary Strand
          Posted Jun 9, 2009 at 2:22 PM | Permalink

          Re: jeff Id (#133),

          Is the code available to the public?

          What code?

  48. jeez
    Posted Jun 9, 2009 at 12:06 AM | Permalink

    I must admit, before I saw Steve’s post above the issues with reading comprehension seemed to jump out rather dramatically.

    Understanding if P then Q generally does not imply that also if P then cows don’t eat rocks.

  49. Gary Strand
    Posted Jun 9, 2009 at 3:54 AM | Permalink

    One more time, so what? That doesn’t mean that people should do statistical analysis in Fortran or that Hansen should archive in binary.

    Given that R depends on Fortran, you’re doing analysis with Fortran, whether you consider it medieval or not. And, it’s nice of Hansen to provide data at all – he could have encoded it in EBCDIC.

    • David Cauthen
      Posted Jun 9, 2009 at 6:25 AM | Permalink

      Re: Gary Strand (#113),
      Assuming you are the Gary Strand who is the software engineer at NCAR, your problem with R is disconcerting to say the least.

      • Gary Strand
        Posted Jun 9, 2009 at 8:05 AM | Permalink

        Re: David Cauthen (#116),

        Assuming you are the Gary Strand who is the software engineer at NCAR, your problem with R is disconcerting to say the least.

        I don’t have a problem with R. I do have a problem with people dismissing Fortran.

        • RomanM
          Posted Jun 9, 2009 at 10:00 AM | Permalink

          Re: Gary Strand (#119),

          I do have a problem with people dismissing Fortran.

          All right, Gary, I’m willing to be informed. Do something constructive and give us some genuine reasons regarding what is so wonderful (and presumably better than any other coding platform) about Fortran and why, in this day and age, we should all be using it instead of R, for example.

        • Gary Strand
          Posted Jun 9, 2009 at 11:38 AM | Permalink

          Re: RomanM (#126),

          give us some genuine reasons regarding what is so wonderful (and presumably better than any other coding platform) about Fortran

          “R” depends on it, for one example.

          I’m amused by folks who claim Fortran is “medieval” or on its last legs – they’ve been saying that for 20+ years, and it’s still around.

  50. RomanM
    Posted Jun 9, 2009 at 12:06 PM | Permalink

    The same might be said about assembly languages too since Fortran “depends” on them.

    My personal use of R doesn’t “depend” on my knowing how to use Fortran. R may use some subroutines originally implemented in Fortran because they were written well and effiectively that way, but I wouldn’t say that it makes the general use of Fortran for original programming purposes desirable today. I taught some Fortran to engineers back in 1969 (it was the first language I learned), but when I was introduced to APL, I had to forget the loops of Fortran to effectively move on to a more efficient way of doing computing.

    I wonder what a plot of the proportion of computing done in Fortran versus time would look like. My suspicion is that most of the users would be still programming that way from the inertia of habit. Why learn a new language when all of my routines are written in the old one?

    A computer language is an interface between the user and the machine and its value is in the utility for the user. If another language gets the job done better for me, then it is preferable. I asked you to give me an example where a new application would be written in Fortran better than R or any other current programming language. I can think of one or two, but they are rare. Your example was not relevant.

    • Gary Strand
      Posted Jun 9, 2009 at 1:25 PM | Permalink

      Re: RomanM (#129),

      If another language gets the job done better for me, then it is preferable. I asked you to give me an example where a new application would be written in Fortran better than R or any other current programming language.

      Fortran is a current programming language. You don’t even need as many loops as you used to.

  51. Posted Jun 9, 2009 at 2:06 PM | Permalink

    Sure are a lot of comments, so I’m sorry if someone already raised this point.

    R is great, but it is still an interpreted language. Fortran and Matlab code can be compiled into machine code executables.

    There are ways to optimize R performance, but the best would be if the R project would be extended to have a fullfledged compiler (and other tools would be great too).

    Long ago (seems almost like another life) I did a fair amount of FoxPro (PC database) programming, a variant of dBase. You could use it in interpreted mode, or build compiled programs for maximum speed. I think Matlab has that capability, giving it a significant advantage over R.

    • Steve McIntyre
      Posted Jun 9, 2009 at 3:03 PM | Permalink

      Re: Deep Climate (#134),

      For statistical applications, the speed of execution is pretty much irrelevant.

      The costs are programming and analysis, where the gains from using high-level languages such as R or Matlab, vastly outweigh any slight benefits in execution speed from using assembly language or Cobol or Fortran.

      • Gary Strand
        Posted Jun 9, 2009 at 3:17 PM | Permalink

        Re: Steve McIntyre (#136),

        For statistical applications, the speed of execution is pretty much irrelevant.

        That would depend on the nature of the statistics being sought.

  52. Mark T
    Posted Jun 9, 2009 at 3:24 PM | Permalink

    And the skills of the programmer. As I recall, one of Mann’s RegEM routines was terribly written and excruciatingly slow. At the time, Jean S and I were looking at it, and maybe even UC. I don’t recall the specifics, but it was tied to a failure to understand the vectorization available with MATLAB, i.e., he often used loops instead of vector commands, etc. He also did not understand that scalars interact with arrays rather easily, but that’s not a huge performance impact unless a loop is being used to add a scalar to every element in an array.

    But the context, obviously, is the sort of statistics being done in here, for which, Steve is correct. None of these sorts of analyses require real-time execution speeds.

    Mark

    • Steve McIntyre
      Posted Jun 9, 2009 at 3:34 PM | Permalink

      Re: Mark T (#138),

      The Mann analyses only take time because the code was beyond horrendous.

      But because it was done in Matlab – you could sort of follow what they were trying to do. Now that we’ve pretty much figured out RegEM, we need to revisit it. If they’d done it in the Fortran of MBH98, it would have been totally unintelligible.

  53. Mark T
    Posted Jun 9, 2009 at 3:54 PM | Permalink

    Hehe, yes, that was my point. We determined his code was a brain landmine: walk in and your brain might not come out the same.

    I had a full copy at the time, though I do not recall where it was. I had a rather catastrophic HDD crash around December, 2007, so I lost a lot of that peripheral stuff I had been working on. From what I understand, the code as it was at the time, is no longer available. You and Jean S have copies, right?

    If they had done it in the Fortran of MBH98, I’m guessing an obfuscated code contest contribution suggestion would have been in order? :)

    Mark

    Steve: see http://holocene.meteo.psu.edu/shared/research/MANNETAL98/. I’ve copied it as well/

  54. Posted Aug 15, 2009 at 3:17 PM | Permalink

    Humf… Yet Another Language Food Fight… YALFF.

    I could say something snarky like ” Yer all pansies, I used to toggle binary programs into an Altair MITS on the front panel and read the output from the blinky lights”… or mention the time I learned “APL” out of curiosity (and wrote a program about 40 characters long that did about 5 pages worth of work – but they don’t call it a “write only” language for nothing, the next day I couldn’t read it… 8-) Yeah, that APL, the one that needs a custom keyboard with hieroglyphics on it … Ah, the days… I’m SO glad they are gone ;-)

    After that stuff, FORTRAN is positively God’s Gift.

    OK, with that out of the way, I took the approach of minimal change and just made GIStemp go. I also wanted to be able to throw rocks at it without having to defend too much against the “You must have changed it!” accusation.

    It’s up. It’s running. You need a LINUX or UNIX box (I’m on Red Hat 7.2 with a 400 Mhz AMD chip and about 128 MB of memory IIRC) and the changes needed are minimal. (Mostly just not using one non-standard data declaration – i.e. move the data initialization out of the variable declarations and into a DATA statement). I’m happy with that choice. And matching each chunk to the compiler for it’s era. f77 for some, g95 for others.

    Now I can, at my leisure, rewrite any part of it into C (or whatever), do an A/B compare, and proceed with confidence that I have not changed anything enough to matter or to need defending.

    If anyone wants a working copy, let me know at my site. I’ve only translated one small test program to C to check how hard it is (not too bad). The biggest issue I’ve run into is that the Hadley SST files are “Bigendian” and a PC is “littleendian” and FORTRAN “unformatted” files are actually quite formatted and in an odd way that endian swaps make hard to handle in C. (It writes a byte count, the sometimes variable length bytes, then the byte count AGAIN. You can’t just blanket unswap the bytes… you must know the FORTRAN layout of the data. i.e. an INT takes a different byte count that a LONG INT to be reversed.) See:

    http://local.wasp.uwa.edu.au/~pbourke/dataformats/fortran/

    then weep… “what were they thinking” indeed… and do you really want to try to unscramble that in C ?

    A decent starting point is at:

    http://chiefio.wordpress.com/2009/07/29/gistemp-a-cleaner-approach/

    Where I talk about the changes I made to the directory structure, the change to a Makefile design, and that it works with less confusion this way.

    Oh, and I’d suggest NOT dredging your brain through their code if possible. The damage it has done to my coding style will take a couple of years to purge. You can’t steep yourself in someone else’s stuff for months on end without starting to code in their “accent”… A major part of my goal has just been to get most of it so that folks can look at a chunk, re-write it sanely, do an A/B, and then move on. The whole STEP0 ought to just be a few data load scripts into a decent relational database.

    E. M. Smith

  55. Posted Jan 6, 2010 at 10:49 PM | Permalink

    Getting back to the original topic: the problem with translating station to gridded and zonal data is obvious.

Follow

Get every new post delivered to your Inbox.

Join 3,379 other followers

%d bloggers like this: