As I surmised, Science has taken a dim view of Kaufman’s failure to provide data that was supposedly “publicly available” and most of the problems have now been dealt with. There are still a couple of issues though.
The three Finnish sediment series and one Canadian series have now been archived:
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/paleolimnology/europe/finland/nautajarvi2005.txt
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/paleolimnology/europe/finland/korttajarvi2003.txt
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/paleolimnology/europe/finland/lehmilampi2007.txt
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/paleolimnology/northamerica/canada/ellesmere/c2-1996.txt
In respect to the Canadian series, I’d like to specially note that Scott Lamoureux of Queen’s voluntarily sent me the data without being asked, when he learned that it had become an issue. I intended to comment favorably on this at the time; I regret that I didn’t do so right away, but do so now a few weeks later. He also said that he had sent the data to the paleo data bank a number of years ago, thought that it had been archived at one time and was surprised that it was not presently available (which he undertook to correct and has.)
The versions of four ice core series used in the Corrigendum are available in the pdf format increasingly used by paleos to prevent the use of turnkey scripts to access data. At least, they didn’t use the photo format of Esper et al 2009. The original data is here:
http://www.iceandclimate.nbi.ku.dk/data/Kaufman_etal_2009_data_29sep2009.pdf/
An ASCII collation is at CA here:
http://data.climateaudit.org/data/ice/kaufmansep09.dat
The D’Arrigo et al 2006 Gulf of Alaska chronology used by Kaufman remains unavailable. In addition, the Renland version as used in the original article remains unavailable – only the version used in the Corrigendum. The Corrigendum removes an adjustment reported in the peer reviewed article cited in Kaufman et al 2009 and it seems to me that Science should require the original version. Also that, if the Kaufman authors wish to alter the Renland series from the version presented in the original article, this should be presented to the referees and properly described in the Amended Supplementary Information.
At the time, I thought that it was pointless for Kaufman to think that Science would regard my data requests as anything other than well within their policies and that it was imprudent of him to force them to open the file. Sciencemag has done exactly what I anticipated. I’m sure that Kaufman is sulking a little about events, but he and other authors would be better off sulking less and archiving more. I sent a note thanking Science for their prompt attention to the above, reminding them that the D’Arrigo version is still unavailable and asking for the original Renland version.
I’ve also taken another crack at trying to get the D’Arrigo data, something that I originally attempted to obtain in 2005 as an IPCC reviewer. I sent a request to Colin O’Dowd, editor of JGR a couple of weeks ago, reminding him of my original 2005 request and reiterating my request for data, referring to very clear AGU policies on the matter. No answer or acknowledgement. Just as in 2005, when the only response to my similar request was IPCC threatening to remove me as a reviewer. This is a completely different attitude than Science, who are amazingly prompt in replying to my emails. I refreshed my request to O’Dowd, this time copying two members of the AGU Publications Committee, one of whom acknowledged the inquiry within minutes.
Climate Audit 
51 Comments
I must admit I am impressed. The influential journals, especially Science and Nature have been pretty belligerent in their attitude towards criticism of these issues. That Science would take this step is reassuring insofar as their integrity is concerned.
Re: Andrew (#1),
The tide does seem to be turning. They are clearly sick of getting burned. I wonder when they’ll get sick of 27-page SIs that are undecipherable and require their own Corrigenda?
I agree with Andrew. Good on Science for doing this and maybe it will make authors more diligent in their overall approach to archiving. Also good on Scott Lamoureux for offering up his data once he became aware of the situation. (And, as ever, well done Steve for asking!)
Great job Steve.
That’a boy, Steve! It’s good to see Science upholding a policy that’s been long overdue. Patience is virturious! It will be interesting to see what other Publications will follow up on “their” standards. It’s been an “amazing” ride here and with fellow “steadfast” team members at your side! Keep up the great work to “all” of you that care!
We’ll have to coin the term ‘churchillian’ to describe your very admirable persistence in these matters, Steve, and civil all the way. There are few or none other with the fortitude to have achieved it.
–
I looked for an authentic video of Churchill’s final ‘never give in — never, never, never.’ speech to post here, but didn’t find one, more’s the pity.
Re: Pat Frank (#6), Re: Mike Bryant (#11),
This seems to have the authentic audio of the most famous part of the speech in the very beginning:
Re: Jean S (#20),
Thank you for that link to Churchill. I’m using it in a tribute to Peter Bocking, the wizardly Manchester guitarist, who just recently passed away.
Oh, and good show, Steve. You are as unique as Peter Bocking.
==============================
Nice job Steve, and to all and any science publication editors reading here – do not be caught on the wrong side of history. It is a “no brainer” to require authors publishing in journals under your editorship that the data be available to anyone interested.
Fantastic job Steve- I will be renewing (after a three year hiatus) my subscription to Science immediately. Thanks again.
Good. Now I wonder whether Science is going to demand the same thing of their (subset of) social neuroscience authors who have been stonewalling various friends of mine about making their data available.
Great to see how the openness of informed and very persistent public inquiry is shaping or, better to say, remedies (I agree with nvw #7) the data handling standards of a mainstream peer-review literature enterprise like Science.
It is very encouraging and gives one hope that one day the standards of data analysis and other standards will also raise up to the level.
The remarkable progress in physics in the last century rests greatly on the division between experimentalists and theoreticians. Competent experimental data gathering is extremely tedious and leaves little time for competent analysis which demands different skills. It is normal that data collected by some are analyzed by all members of scientific community who are willing to give their time for that, each contributing a bit of his or her competence. Hiding experimental data is a crime against science. Shame on JGR!
Best slightly shortened version of Never Give In speech I found…
Embedding Mike Bryant’s link.
Bulldog !! Once teeth enter you can’t pull them out until the dog is ready
thanks Steve for that 24/365/10/**** persistence of yours, with a little help from friends and Clapton et al. The world would have been a poorer place without you.
It’s another battle, but it’s not yet the war.
OK, this is nice. Now let’s see if it will turn the tables officially around with regard to the AGW discussion.
“..We’ll have to coin the term ‘churchillian’ to describe your very admirable persistence in these matters, Steve, and civil all the way. There are few or none other with the fortitude to have achieved it..”
Churchill’s advice was to ‘Keep buggering on’, or KBO for short.
It seems to work….
This is excellent news and I would like to join with the others in commending Steve M’s extraordinary work. The comparisons with Churchill are apt. KBO seems spot on. By the way, the BBC just showed an excellent docu-drama on Churchill during the war years, entitled ‘Into the Storm’. Well worth watching.
.
One thought occurred to me during the recent Briffa episode. It’s easy to think that these people don’t archive their data because they have a lot to hide. This seems to be the case with Briffa, because it took Steve just a few days to find very serious and obvious shortcomings. The question is: why did Briffa suddenly archive the data after publishing a paper at a journal that unexpectedly enforced its own rules? Did the editor of the journal make any threats?
.
Was it possible that the editor said something like this? “If you do not archive your data as required by our rules I will be forced to print a retraction of your paper”.
.
Was it possible that Briffa decided that archiving his data, with all its obvious problems, was the lesser of two evils?
Chris
Good work Steve. Step by step, inch by inch………………..
Congratulations Steve, you are having a material impact on the openness of paleo science.
Never before in the history of science blogging has so much been achieved for so many by so few.
Some of the climate scientists would benefit from taking a more Churchillian attitude. I am thinking of the incident (discreetly filmed in the excellent recent docudrama) where Churchill discussed strategy with Roosevelt while in the bath, stepped out wearing a towel that fell off and said
My estimate is that it will take five years of vigilance before this proper behavior becomes habitual with journal editors. Look for a slip-up or two in the near term and the need for a firm reminder. The battle is not won; only the tide is turning.
Congratulations, Steve! It is reassuring, though long overdue, to see the Science community finally deal with this issue.
snip – venting
Folks, while I appreciate the kind comments and flattery, they vastly over-value a very small enterprise.
Re: Steve McIntyre (#25),
Seriously, you are being too modest here, Steve. We are seeing more and more that the standards of science are being upheld by the journals and you are the one responsible. Yes, we know there is still a great deal of work to be done. It will be much easier when the work can be done in the light of day.
Re: Ron Cram (#26),
I have to agree. It is not just paleoclimatology that suffers from a problem of irreplicability and non-disclosure. These persistent efforts are starting to have an impact in the medical sciences as well.
Re: Steve McIntyre (#25),
Since when was the importance of an enterprise determined by it’s size?
See PaulM on Churchill’s bath towel above.
I, too, will consider renewing my subscription to Science that I had discontinued about a year ago for what I considered poor editing ethics.
Steve, I’ll point out that small enterprise is the majority of the world’s business. Don’t underestimate its value.
Applause!
I will not bother Steve M with accolades for his recent success, but instead will repeat my mantra that whether Steve M is successful or not the major good to come out of what he is doing is to reveal, for all to see, some of these weaknesses and contradictions in the peer review processes – and hit the tip jar.
Steve,
Is there any place (either here or somewhere else on the ‘Net) that provides a comprehensive list of all of the major articles published on AGW, including their relationships to each other and links to their datasets (or lack thereof)? I’ve seen you reference the “web of relationships” (for lack of a better phrase) a couple of times recently, but it seems like a page detailing the papers–their methods, results, data collection techniques, data sets, etc.–would be immensely helpful for a reference point. Especially if a relationship graph could be displayed. Tufte would approve.
I am interested in doing this myself, but frankly, don’t know where to begin. It would obviously have to be kept up-to-date (with previous versions being archived for purposes of integrity) as new results came out, and updates (corrigenda?) to previous outstanding issues brought to light. Like this data being finally published, for example.
Just a thought. If there is anyone here who could take this on, I think it would bring a great deal of sunlight to the subject. Or, if something like this exists, I’d love to see it. And if I can help at all, just drop me a line at my email address in my name.
Good work, Steve. Your modesty is appreciated, and you have every right to be proud of the results your efforts have brought forth.
Re: NukemHill (#33),
There have been some articles here which did a bit of that sort of thing, but not everything. You might want to look, or relook, at the Wegman Report which is linked in the top left-hand margin here to see which articles he used and how he displayed the info. Also there’s a wiki for CA which can be found via the Acronyms link. I believe there’s a number of articles listed there and linked to the corresponding articles here.
Re: Dave Dardinger (#35),
I’d had Wegman’s “social network” chart in mind when I was asking about an overall summation page. I’ll have to take another look at the Wegman report. It certainly was fascinating. And the interdependencies he talks about (and that Steve addresses in some of his posts) are an important component of what I’m thinking of.
Part of what I’ve got in my mind would be some sort of diagram (tree or graph) that displays the timeline over which these papers have been published, with links displaying data, author, methodology re-use, etc. Also, links to different blog/article discussions of such. Including both sides of the argument, of course. I’d have no problem seeing links to RC as well as CA. It might prove to be enlightening to some people who think there is equal transparency and tone on both sides….
If I get a chance, I’ll dig back through Steve’s archives and try to mock something up. If I think it’s worth something, I’ll let you know. I’m pretty pressed for time, so I don’t know when I’ll actually be able to do this. I don’t know where Steve finds the time to do all the work he has done. It’s really remarkable to me how dedicated he is to uprooting the truth buried under all the crap.
Steve: Wegman’s network analysis missed the much more obvious central connections: Bradley and Jones were much more central than Mann as at 1998.
Re: NukemHill (#37),
I think the Wegman social networking matrix plus Steve’s chart on the use of proxies in papers from “Briffa on Yamal” (link below) provides a lot of perspective.
http://www.climateaudit.org/?p=7588
Re: David Jay (#40),
Ding! That’s a great start. Figures. I hadn’t dug into that posting yet, and there’s the beginning of what I’ve been thinking about. Now, the blanks need to be filled in with references to the data sets they use (since they’re not Briffa). And, of course, details on the claims, methodologies, etc.
Thanks. I’m going to start archiving all of the posts that I find that give details like this. If only for my own benefit, as I tend to get lost in the names, dates and data sources. The claims and counter-claims get really overwhelming after a while.
Steve–interesting point about Bradley/Jones vs Mann. Do you have that documented somewhere? I see Jones referenced several times in the table to which David Jay refers, and lots of et als. If you have easy access to a more complete list, I’d love to see it.
Which actually raises another question for me–is there a comprehensive list (yeah, yeah–make it yourself, Greg!) of biographies/CVs of all the dramatis personae in this ongoing soap opera? Wagner’d be proud!
It’s interesting. The more that I think about this, the more I’m thinking this might make for a fascinating (if only to the digital cognisenti) book. An in-depth look at all of the players and the politics of AGW. I’m thinking the lack of transparency in one camp might make it a difficult endeavor, though…. Sorry, Steve, if that’s a little too political for you.
A potential glitch here is that in both files, the “decimal points” are continental-style commas instead of periods. Does R have an option to work around this automatically?
Steve: It seems like there should be one, but I don’t know it. A simple patch is to do the following:
I’ll edit my online rendering accordingly and save it in that form.
Oh, and a definitive set of links to the data that is actually available is paramount. There have been so many claims over the years about data being available for this and that. It’s hard to keep track of what the reality is. I guess I’m kind of thinking of having a page that is a clearing house for all of the relevant discussions regarding the major issues.
*sigh*
I know, I know. If you want something done…. 😉 I’ll see what I can come up with.
Matlab (which I assume they are using) has a publish command which lets you turn an m file into a variety of formats. One of them is pdf. They’re probably just publishing to pdf because they are lazy, and it is actually easier to do than publish to a text file.
In any case, you can still turnkey data like this, with something like pdf2html, a unix library. A little bit of code to find where tables begin and end, and you can extract all tables from a given file. They will still have idiosyncrasies, of course.
Steve: What is the basis of archiving data as photo-images within a pdf? Puh-leeze. I have a lot of experience in trying to make data accessible. If I made data available only as a photo-image, you can be sure that I was trying to make replication as onerous as possible within what the journal had required me to do. If you want to make data accessible in 2009, you make it available digitally.
“…better off sulking less and archiving more.”: That’s my quote of the week. And I too applaud the tenacity, the persistence, of the creator of this blog. (Can I do that without flattering? Well, I hope so: Humility in everyone is vital.)
♫ And the times they are a-chaingin’….. ♫
Perhaps the Editor of “Nature” should consider the words of Marshal Foch, suitably modified.
The Royal Society has given way, “Science” has retreated, situation dreadful, I shall surrender.
The whole issue of not publishing such data is just amazing to me. Where is the up side, ethically or long term? The reasons for not doing so run the gamut from deliberate fraud or incompetence on one side to being so arrogant, or disorganized and befuddled you shouldn’t be trusted to come to the right conclusions with your research on the “good” end of that spectrum. I mean, there’s no “good” fight to be fought there, covering up things that should be freely available has no real up side as far as I can see.
But without the efforts of our host here and scores of others, chipping away at this edifice, so much would still be hidden. We are witnessing a new paradigm of how peer review and science operate today with blogs and such. While it does have a Wild West side at times, it also brings the largest number of serious eyes onto a topic possible, and is egalitarian and promotes what I’d call a true meritocracy of ideas, where rough and tumble discourse peels back the layers and gets down to the real point quickly. Which apparently scares a lot of people invested in the status quo. Same is happening with a lot of the old information channels, from news to entertainment to communication, everything gets more amorphous and interconnected. I do find it an interesting ride, and kudos again to Steve and all the others (including Science) for freeing the data.
Should we be sending Steve bottles of Johnnie Walker Black then? Churchill won the war on a quart a day, it is claimed. . .No wonder he had that lisp!
“It was the nation and race dwelling all around the globe that had the lion’s heart. I had the luck to be called upon to give the roar.”
From nobelprize.org: “Each year the respective Nobel Committees send individual invitations to thousands of members of academies, university professors, scientists from numerous countries, previous Nobel Laureates, members of parliamentary assemblies and others, asking them to submit candidates for the Nobel Prizes for the coming year. These nominators are chosen in such a way that as many countries and universities as possible are represented over time.”
I would like to propose to any reader of this blog who receives an invitation to nominate people for the nobel prizes, to consider nominating Steve (don’t you dare delete this Steve) for his dogged persistence in leading the charge to demand that the science community, and particularly science publishing, tighten up its ship and live up to the principles and standards of the scientific method.
I’d love to see Al Gore have to attend Steve’s award ceremony…..
The small enterprises can’t be ignored for long if the Government is serious about the economy recovering. Small entrepreneurs make the economy tick.
Is this Science’s changing its policies, or is there a difference between Kaufman 09 and Briffa&Osborn 06 that requires Kaufman to provide extra data?
>Wegman’s network analysis missed the much more obvious
This network analysis absolutely dominates the report, and he missed something?