David Bader, PhD, the Director, Program for Climate Model Diagnosis and Intercomparison, writes today seeking to “clarify several mis-impressions on your “climateaudit.org” web site” regarding the archiving of Santer’s data, the correspondence being shown below.
Readers may recall an earlier post here in which I requested data from Santer et al 2008, in response to which Santer refused to provide the data, circulating his discourteous refusal to 17 coauthors (none of whom had been copied in the original inquiry) and the journal editor. Santer:
… I see no reason why I should do your work for you, and provide you with derived quantities (zonal means, synthetic MSU temperatures, etc.) which you can easily compute yourself. I am copying this email to all co-authors of the 2008 Santer et al. IJoC paper, as well as to Professor Glenn McGregor at IJoC.
I gather that you have appointed yourself as an independent arbiter of the appropriate use of statistical tools in climate research. Rather that “auditing” our paper, you should be directing your attention to the 2007 IJoC paper published by David Douglass et al., which contains an egregious statistical error. Please do not communicate with me in the future.
I have reviewed this post and fail to see any statements by me that could possibly be construed as contributing to a “mis-impression”. On Nov 24, I reported on my unsuccessful FOI request to NOAA for the data, in which all of the NOAA coauthors claimed to have never seen the requested data. Again, I am unable to see any statements by me that could be construed as creating a “mis-impression”. The facts are what they are.
On Dec 28, I reported on efforts to obtain the data through the journal, reporting that these efforts had also been unsuccessful, as the publisher of the journal, the Royal Meteorological Society lacked a data archiving policy. A positive outcome of this effort was that the Society plans to review their lack of policy at a forthcoming editorial meeting. Again, I am unable to see any statements by me that could be construed as creating a “mis-impression”. The facts are what they are.
Around January 18th, I received a snail mail letter from the U.S. government dated Dec 10 (snail mail indeed), advising me that the FOI request had been placed in a queue and would be responded to when it got to the top of the pile. I didn’t plan to hold my breath.
On Jan 26, Ross and I submitted an article on Santer et al 2008, noted up the next day here ; I reported in the post that our submission included comments on the data refusal. On January 27, I received an email from a reader notifying me that the reader had just been notified by the U.S. Department of Energy that the data had been placed at a public archive. I promptly communicated this information to readers. I noted that, while the reader had been so notified, I had not received equivalent notice, again, a matter of fact. I am unable to see any statements by me that could be construed as creating a “mis-impression”. Later that day, I consulted the new archive and noted with some amusement that the file unzipped to a directory entitled “FOIA”. [Update Jan 31 - for "clarification", I do not imply that the data was released "because" of our journal submission on Jan 26. The CA reader in question had been on a lengthy business trip. As noted below, he had been informed on Jan 14 that Livermore was planning to release the data and that he would be informed of the url when available; he followed up upon return from his business trip on Jan 26 and obtained the url on Jan 27, whereupon he informed me. Had he not been traveling, he might have learned the url on an earlier data. Livermore did not inform me of either their plans or the actual archiving and my knowledge of the situation came only from from this CA reader.]
Earlier today, Dr Bader wrote as follows:
Dear Mr. McIntyre;
I want to clarify several mis-impressions on your “climateaudit.org” web site with respect to the Synthetic MSU data sets on the PCMDI website.
1. The data were released publicly on 14 January 2009, at which time our Department of Energy sponsors and NNSA Freedom of Information Act officials were notified. These data were released voluntarily by the Lawrence Livermore National Laboratory and we were never directed to do so as a result of a Freedom of Information Act (FOIA) request. Furthermore, preparation of the datasets and documentation for them began before your FOIA request was received by us.
2. The long disclaimer on the web site beginning with the sentence, “This data available on this site was prepared as an account of work sponsored by an agency of the United States government. …..” is standard language on all published material from Lawrence Livermore National Laboratory and is not specific to this dataset.
David Bader, PhD
Director, Program for Climate Model Diagnosis and Intercomparison
Needless to say, I was extremely surprised by this letter. If Lawrence Livermore was actively preparing the data sets for public release “before” my request for the data, then surely Santer had an obligation to simply say so, rather than withholding the information that a public release was planned in the near future and challenging me to recreate their monthly results from first principles. Secondly, I was surprised to learn that the directory entitled FOIA had nothing to do my Freedom of Information Act request. Perhaps FOIA in this context stands for something else.
I replied to Bader, citing verbatim my original request for data and Santer’s refusal and asking:
If, as you say, preparations for the release of this data had already begun, could you explain why your employee failed to advise me of this at the time. In addition, I received no notice of these plans pursuant to my FOI request (to which I have received two separate acknowledgements).
You say that, on Jan 14, 2009, the data was “released publicly” on Jan 14, 2009 and DOE were notified of this. On Jan 14, 2009, one of my readers received an email from DOE saying that the data was being finalized that week in preparation for posting and the lab was seeking final approval from their site office to post the data, undertaking to send you the url when it was available, providing notice of availability on Jan 26. I received notice from the reader on the following day, Jan 27, and promptly recorded this notice on my blog.
At no point prior to your email did anyone from your organization notify me that the data was now “publicly available” despite my outstanding request.
I will post a notice at my blog of your position, but I’m sure that you will understand if I make editorial comments on them.
Given the information in your email, I hereby file a complaint about the handling of my request for data and request that you investigate how it was handled.
I also observed:
if you unzip the data sets placed online, they unzip into a folder entitled FOIA. Does FOIA in this context have another interpretation other than Freedom of Information Act that I should be aware of?
My previous email was to clarify certain factual information with respect to the release of the Synthetic MSU data. Respectfully, I believe it to be counter productive to engage in an exchange regarding the history of your correspondence with Dr. Santer or Department of Energy officials on a matter I believe to be completely resolved.
I wrote back to him:
What is the factual information that you wish to correct? I am quite prepared to correct any errors: could you please provide me with specific points of fact that you believe to be in error, as I have reviewed the posts in question and cannot any “errors” in what I posted.
This request failed to elicit any details on any factual errors that could have contributed to any “mis-impression”. Bader replied:
I will make a sincere attempt to advise you of our good faith efforts to release the data in question. What you may not realize is that there are time consuming, but legitimate and typical review processes at most laboratories. For example, it takes 2-4 weeks from the time a manuscript is completed at my lab before it is transmitted to a journal for submission.
1. I was not aware of your FOIA request to the NNSA until sometime in early to mid December. At that time, we had already begun the process of preparing the Synthetic MSU datasets and their documentation for public release as a low priority activity. In their original format and without documentation, they were of little value to the broader scientific community for which they were intended.
2. When contacted about the FOIA request by LLNL FOIA officials, I inquired as to whether our plans for data release met their needs, and was advised that it would.
3. Given the demands on my staff’s time, the length of time required for the official laboratory “Review and Release” process and the upcoming holidays and vacations, I asked LLNL officials who handle FOIA requests if January 15 was a reasonable deadline for the release of data. I was advised that the date was acceptable.
4. The data were made available on January 14, at which time I notified LLNL staff who deal with FOIA requests and Department of Energy officials.
Should you intend to publish my correspondence with you on your site, I request that you post this entire email, without embedded editorial comment. Since you have clearly stated your intention to file a formal complaint in a previous message, I think you can understand that further correspondence with me on this issue would not be productive.
Santer et al 2008 was submitted on 25 March 2008, revised 18 July 2008 and accepted 20 July 2008.
Obviously, I have a number of questions about this matter, most of which will undoubtedly occur to readers. So I’ll defer editorial comment for now.