Geoffrey Boulton, who did an execrable job on the Muir Russell “inquiry”, has written a good editorial in Nature here reporting on the recent Royal Society report that he chaired.
There have been a number of reports over the years, urging improved data archiving, and yet the problems persist. Boulton’s report is merely one more. Whether it will have an impact when past reports have failed remains to be changed. In the U.S., there are quite sensible high-level senior policies on data archiving, but these are flouted in paleoclimate by the relevant NSF division. The AGU has sensible policies, but these are ignored by editors and journalists. In the past, as evidenced in Climategate emails, members of the climate “community” have sneered at my efforts to ask AGU editors to enforce these policies, confident in the solidarity of the editor, and such efforts have proved fruitless.
Boulton’s report and editorial merely add one more editorial, but one more editorial isn’t going to affect someone like Lonnie Thompson.
The missing link in Boulton’s report is putting teeth into such recommendations. Enough teeth that recalcitrant journals like The Holocene (which proudly has no data archiving policy) have to adopt and enforce policies and recalcitrant scientists like Lonnie Thompson have to pay attention.
I’ve observed for many years that the funding agencies, e.g. some divisions of the US. National Science Foundation, have much of the responsibility. One of the problems is that some NSF divisions have failed to ensure compliance with existing federa policy, instead leaving matters up to academic journals (which have no obligation to enforce US federal policy.)
As sometimes happens with regulatory agencies, the NSF climate agencies have become a cheerleader for the industry that they are supposed to regulate (in the sense of at least requiring compliance with data archiving). When challenged in the past, NSF administrators have said that authors have met journal standards and that was the end of the story. Similarly with IPCC, which should also have standards for articles cited in IPCC assessment reports. I challenged Susan Solomon on this in 2005 and she said that the establishment of data policies by IPCC would be interfering with academic journals.
However, both funding agencies and IPCC have different obligations than academic journals. If they wish to rely on academic journal policies, then they need to ensure that such journals have data policies that are sufficient to ensure compliance with the funding agency and/or IPCC obligations. To put some teeth on this, if the journal (e.g. The Holocene) does not have such standards, then its eligilibity for citation as a publication of record for an NSF funded project or by IPCC should be withdrawn. If The Holocene, for example, was reduced to grey literature for NSF and/or IPCC purposes, maybe editor John Mathews would pay attention.
Similarly, the funding agencies need to pay some attention to their obligations to ensure data archiving. While they enjoy being cheerleaders for the scientists that they are funding – and are entitled to take some satisfaction in their accomplishments – they cannot become so close to the science industry that they abnegate their regulatory responsibilities.
In passing, Boulton endorses a longstanding issue at Climate Audit – that scientists should archive all the data, not just the data “used” in the final calculations.
Too often, we scientists seek patterns in data that reflect our preconceived ideas. And when we do publish the data, we too frequently publish only those that support these ideas. This cherry-picking is bad practice and should stop.
For example, there is strong evidence that the partial reporting of the results of clinical trials, skewed towards those with positive outcomes, obscures relationships between cause and effect. We should publish all the data, and we should explore them not just for preconceived relationships, but also for unexpected ones. Without rigorous use and manipulation of data, science merely creates myths.
I raised this issue originally with Jacoby at Climatic Change and my requests were rebuffed. Little has changed over the years and my recent similar request with Gergis to four different journals (Journal of Climate, Climate Dynamics, Holocene and GRL) likewise had no success.
This is, of course, the issue that was at the heart of the ongoing Yamal controversy. Gavin Schmidt and Real Climate sneered at the idea that CRU should have any obligation to do anything other than what Boulton describes as “partial reporting”. Perhaps Boulton’s new report will help change perceptions on this point. It’s too bad that Boulton’s epiphany came after his participation in the Muir Russell whitewash.