Data and Source Code: American Economic Review Policy

In our E&E article, we referred to the fact that the American Economic Review (AER) has adopted a policy for requiring authors to archive source code and data at the time of submission. These policies are online and worth looking at in detail. My attention to this online URL is due to Joseph Potts, who has made frequent comments on posts here and who has written a comment about our work and data/code policy which explicitly links to the AER policies.

The AER policy states:

It is the policy of the American Economic Review to publish papers only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication. Authors of accepted papers that contain empirical work, simulations, or experimental work must provide to the Review, prior to publication, the data, programs, and other details of the computations sufficient to permit replication. These will be posted on the AER Web site.

The detailed policy goes on to say:

For econometric and simulation papers, the minimum requirement should include the data set(s) and programs used to run the final models, plus a description of how previous intermediate data sets and programs were employed to create the final data set(s). Authors are invited to submit these intermediate data files and programs as an option; if they are not provided, authors must fully cooperate with investigators seeking to conduct a replication who request them. The data files and programs can be provided in any format using any statistical package or software, but a Readme PDF file documenting the purpose and format of each file provided, and instructing a user on how replication can be conducted, should also be provided.

Compare that with Mann’s comments to the Wall Street Journal:

Dr. Mann refuses to release [the source code]. "Giving them the algorithm would be giving in to the intimidation tactics that these people are engaged in," he says.

I see no reason why journals publishing paleoclimate articles should not adopt a policy identical to AER. We voluntarily complied with an AER-type policy by submitting source code and data together with submission, which are archived here.


3 Comments

  1. Louis Hissink
    Posted Mar 4, 2005 at 7:28 AM | Permalink | Reply

    On the other hand sensitive mineral exploration data, which might fall into this category should not be archived- otherwise I hand over to total incompetents and otherwise undeserving types the fruits of my labour.

    Therefore the policy must be that if you publish in a peer reviewed journal, then all the data are archived.

    If the data are commercially sensitive, then what idiot publishes these in the first place?

    No Brainer.

  2. John A
    Posted Mar 4, 2005 at 2:54 PM | Permalink | Reply

    Steve, were these policies at the AER changed as the result of something like MBH98?

    Steve: There have been periodic attempts in econometrics to replicate empirical studies. If you look towards the end of MM05 (EE) you’ll see some references, the most recent of which was McCullough and Vinod. But it arose out of attempts to replicate non-controversial papers. Many of the problems of uncooperativeness noted by McC-V are similar to ones that I’ve encountered, although probably not to the same degree. Ross got a very nice email from McCullough recently.

  3. TCO
    Posted Sep 15, 2005 at 9:59 PM | Permalink | Reply

    This is the way to go, but it will take time…

Post a Comment

Required fields are marked *

*
*

Follow

Get every new post delivered to your Inbox.

Join 3,118 other followers

%d bloggers like this: