Excellent post by Shub Niggurath at his blog here discussing replication problems. It’s interesting to see how the same excuses play themselves out in different fields. Statisticians criticize authors for non-replicability of their results. The authors complain that the statisticians failed to replicate a previously unreported (and usually questionable) methodological procedure. We’ve seen this movie before.
Shub reports that Hothorn and Leisch, Case studies in reproducibility, in Briefings in Bioinformatics noted that one of our papers (MM 2005, EE) even included code in the running text of the paper to clarify certain points:
Acknowledging the many subtle choices that have to be made and that never appear in a ‘Methods’ section in papers, McIntyre and McKitrick go as far as printing the main steps of their analysis in the paper (as R code).