With the recent interest taken by the House Commitee in data archiving, I’d like to review some of my past thoughts on data policy. An audience seems to be developing for these issues. First and most importantly, here is some information on U.S. federal government policy on archiving of data. There are definite and long-standing policies on archiving data which are being flouted by climate scientists and not being enforced by NSF.
After publication of Antonio Regalado’s article in the Wall Street Journal on Feb. 14, 2005, where Mann said that he would not be "intimidated" into releasing his algorithm, I posted the following overview on our requests for source code to show that there had been no "intimidation". Such language from these guys – Mann using the word "intimidation", Crowley "threatening". I included accounts of my requests to Mann for information before MM03 here and after MM03 here .
I also posted up my correspondence with National Science Foundation here. ( sequential – follow links). One action of the NSF that particular surprised me was in connection with my request to Mann for the residual series, which I copied to NSF. While Mann had refused various other requests and I did not expect him to provide me with the information on the residuals, I had not specifically asked him for this information before and, unless you ask, you can’t say for certain what he would do. BEFORE Mann could refuse, Verardo of NSF responded that Mann did not have to provide the information. I was really surprised for this – for all Verardo knew, Mann could quixotically have decided to provide the residual series. Verardo had no business interfering.
In mid-2004, I sent NSF a detailed letter asking for information from various climate scientists that was not archived. In response, I got blown off by NSF saying that the requested information was at WDCP and I should look there. The information wasn’t there or I wouldn’t have asked. (A couple of data sets were archived after this letter – Jacoby archived some Mongolian data, Hughes some Russian data and Thompson for the first time a very coarse (decadal average) data on Dunde, Guliya and Dasuopu. I would like to think that it had some connection with my inquiries – in the case of Thompson, it definitely was because of the involvement of Climatic Change.) The administrator seemed to have no idea that guys like Jacoby and Thompson were not adequately archiving their information. (I’ve attempted to make it clear from time to time that, however frustrated I may be with Mann, his disclosure is better than most climate scientists. I’m sure that it’s a considerable source of frustration to him that the people who are worse have avoided publicity so far.)