[Limor Peer]

The “Social Science Data Editors” group is an ad-hoc collective of (mostly) economics or economics-adjacent active data and reproducibility editors. Most, but not all, of our members work for journals that conduct active computational reproducibility checks.

Lars Vilhuber has provided additional information on their vision for reproducibility and its challenges in the thread below. Please feel free to continue the conversation within the thread!

Briefly, tell us about your work/organization and how it’s related to computational reproducibility; What are you trying to address and how?

<div>The “Social Science Data Editors” group is an ad-hoc collective of (mostly) economics or economics-adjacent active data and reproducibility editors. Most, but not all, of our members work for journals that conduct active computational reproducibility checks. We pragmatically address making standards, policies, tools, and expectations common across many different journals, recognizing that strong standards and policies will depend on the particular discipline, and are driven by and adapted to habits and cultures within disciplines. We differ in our approach to most “data” editors in that we are strongly interested in computational reproducibility, not just data availability, and so tackle the dual problem of making code work, and making data available. Some of our achievements are that many of the top journals in economics, many of which are represented in our working group, now share a common approach to data and code availability, via the Data and Code Availability Standard (https://doi.org/10.5281/zenodo.7436134) we developed, and generally require, or “strongly suggest” a common human-readable descriptive “README” (https://doi.org/10.5281/zenodo.7293838) that has greatly improved the quality of the replication packages that journals receive and publish. We mostly work via a monthly group Zoom (for the active working group) and a more low-volume mailing list. Our members have also consulted with new editors wishing to strengthen their journal’s policies, helping in the choice of tools, repositories, policies, and communications, and the growth of the working group over the past 5 years is reflective of that.
</div>

What is your/your organization’s vision when it comes to computational reproducibility (e.g., all scholarship is computationally reproducible by default)?

We firmly believe, and our journals have in some form or other stated officially (see AEA, Econometrica, EJ, JEEA), that data and code used in the analysis of empirical articles must be available, transparent, and openly available, where possible. When data are not easily shareable, due to ethical or unavoidable proprietary rights, authors must be very clear in how others can access such data as well.

What are some of the challenges you see to achieving this vision?

The core challenges we attempt to address through the group are (a) consistency in expectations and application at journals, reflective of evolving community consensus (b) knowledge within the community about modern computational methods that make reproducibility easier – use of old-fashioned methods is detrimental to some aspects of reproducibility, even if it may not be perceived as such within the community (c) broadening access to restricted data for reproducibility checks by at least one set of outside experts – building in access within data use agreements and other access protocols (d) training of the next generation.

What would you like to ask the members of our Interest Group?

Please reach out with any questions, join the mailing list if you want to lurk, or join the monthly call, where we discuss practical questions.

Discussion