R – the choice for serious analysis

While Steve is a little “under the weather” (it must be all the snow that Al Gore sent him), I thought I’d mention an interesting article in the New York Times which sings the praises of the programming language R.

R is similar to other programming languages, like C, Java and Perl, in that it helps people perform a wide variety of computing tasks by giving them access to various commands. For statisticians, however, R is particularly useful because it contains a number of built-in mechanisms for organizing data, running calculations on the information and creating graphical representations of data sets.

Some people familiar with R describe it as a supercharged version of Microsoft’s Excel spreadsheet software that can help illuminate data trends more clearly than is possible by entering information into rows and columns.

What makes R so useful — and helps explain its quick acceptance — is that statisticians, engineers and scientists can improve the software’s code or write variations for specific tasks. Packages written for R add advanced algorithms, colored and textured graphs and mining techniques to dig deeper into databases.

So there you have it: R is the future.

Give up the Fortran, Mike and the Hockey Team, and join the 21st Century. Uncle Steve does know best.

R can be downloaded at http://www.r-project.org/ and is available for Windows, Linux and Mac OSX. And its free.

Unthreaded #37

It seems a bit mean that Dan Hughes had to post an interesting Unthreaded comment on a year-old thread with nearly 1000 comments.

The Supra-Long Finnish Chronology #2

This post is going to be a bit more interesting than my last one, where I was getting some preliminary information out of the way. The post here draws heavily on Mauri Timonen’s 2009 PPT, which explains how their 7500-year long chronology is obtained. I’m also going to get to some results which will intrigue many readers.

In small lakes in Finnish Lapland, there are hundreds of submerged Scots pine logs that can be thousands of years old. (BTW the countryside and topography of these photographs looks very familiar to a Canadian). On the left below is a picture of an underwater logs (later determined to be from about 2000 BC); on the right is a montage of pictures showing the recovery of the logs and the taking of sample disks: these obviously give better data than pencil thin drill holes where perpendicularity is hard to maintain and where the pith is not known for certain.

   

Continue reading →

The Finnish "Supra-Long" Chronology #1

Over the past 30 years, Finnish dendrochronologists have developed a “supra-long” tree ring chronology now stretching back (Helama et al 2008) to 7600BP, well into the Holocene Optimum. Earlier versions of this chronology have been reported in many articles (e.g. Helama et al 2002, Eronen et al 2002 and references back into the 1990s plus articles here. Among other things, the chronology has been used to date changes in the treeline, which, in turn, has been used to estimate past temperatures. Continue reading →

Royal Meteorological Society Considers Data Archiving

A little progress on this front since my last post on this issue.

I received a pleasant note from Glenn McGregor acknowledging the changes to my note, in which he said that he would ask the publisher of IJC, the Royal Meteorological Society, to provide a statement of their data archiving policy. Given that the policy is set by the Royal Meteorological Society, rather than the editor, our issue is really with the Royal Meteorological Society.

Bishop Hill (and a couple of others) wrote to the Royal Meteorological Society about their lack of policy and it is now on the agenda of their next meeting. He reports the following answer:

Thanks for your note. I’ve had a couple of emails relating to this discussion and the position currently is as Prof McGregor mentioned. As I have mentioned to others who have emailed in, I’m very happy to consider the requirement for a clear policy statement and as such I have put this on the agenda for the next meeting of the Society’s Scientific Publishing Committee, at which all the Editors of the Society’s journals are members. 

It seems odd that they have no policy, but they don’t. So let’s hope that they develop one. For what it’s worth, Climatic Change instituted a data policy as a result of my acting as a reviewer.

In early 2004, in my capacity as a reviewer, I asked for supporting data and code. Schneider said that, in 28 years of the journal, no reviewer had ever made such a request. Needless to say, that didn’t impress me as a reason not to make the request. He said that it would require a decision of their editorial board, so I asked him to obtain such a decision. They then agreed on data but not code. I then asked for supporting data under the new policy, which the authors (MBH) refused to provide and the manuscript disappeared from sight (in the mean-time, they made a check-kiting citation to it in Jones and Mann 2004, so that accomplished what they wanted.) It’s such a stupid game.

However, the exercise was not entirely pointless. I asked for data from Thompson and this resulted in the scraps of information on Dunde, Dasuopu and Guliya now available digitally.

A few more letters to the Royal Meteorological Society wouldn’t hurt. And give a little support to Bishop Hill as well.

Briffa and Sodankyla Church

The old Sodankyla church was built in 1689. Nine cores from beams at this church have been measured and archived. Briffa 2008 used one of 9 cores. Why only one of 9? Your guess is as good as mine. Continue reading →

M&M Return

PNAS writes:

Dear Mr. McIntyre,

We are pleased to inform you that the PNAS Editorial Board has given final approval of your letter to the Editor for online publication. The author(s) of the published manuscript have been invited to respond to your feedback. If they provide a response, it may appear online concurrently with your letter.

This refers, of course, to our comment on Mann et al 2008.

Phil. Trans. B

I have some happier news to report from Phil Trans B, which, unlike the International Journal of Climatology, has a data policy and takes it seriously. Phil Trans B is a science journal published by the Royal Society as opposed to a climate science journal published by the Royal Meteorological Society.

Last summer, I reported that I had requested that Briffa archive data for Briffa et al 2008, pursuant to policies of Phil Trans B. I received a cordial note at the time from the editor that they took their policies seriously and would follow up on it. They responded cordially when I followed up on several occasions and said that they were working on it (and, given that they were dealing with core Team members, this is no small undertaking) and hoped to have the data by the end of the year.

A couple of days ago, I was notified by Phil Trans B that the first installment (Scandinavian data) was online here and that the authors had undertaken to have the balance online “in the New Year”.

All in all, a very professional response from Phil Trans B, placing the surly response from International Journal of Climatology in rather stark contrast and perhaps highlighting rather neatly the questionable data standards that are deemed acceptable by the Royal Meteorological Society and some climate scientists.

I’ve done a quick look at the data available so far: there are spline and RCS chronologies, which appear to match the figure, and measurement data for Tornetrask/Finland combined. I’ll discuss the data on another occasion.

Glenn McGregor: Data Archiving not required by the International Journal of Climatology

After nearly 2 months and several inquiries, the editor of the International Journal of Climatology has finally said that they do not require authors to provide supporting data.

Given that funding agencies rely on academic journals to ensure that authors archive data (improperly abdicating their own responsibilities), the moral of this should be that the National Science Foundation and similar agencies should no longer consider the International Journal of Climatology as a qualified publication for the purposes of completing publication obligations under the terms of NSF grants. Continue reading →

Gavin and the Big Red Dog

A few posts ago, I used the term “‘fingerprint'” (in quotations) in connection with the big red spot that one commonly sees in the upper tropical troposphere in contour plots of projected temperature trends by latitude. On an earlier occasion, we’d talked about Ross McKitrick’s “T3” ( here.) Continue reading →