During January 1996, the Division on Addiction created The WAGER, which was the precursor to The BASIS. We created The WAGER because academic research, including addiction research, often is locked behind paywalls. This history suggests that The WAGER was an early open science initiative. We have carried this support for open science forward into our work for The BASIS. The BASIS’s mission is to minimize addiction’s harmful effects by providing the general public, treatment providers, policy makers and others with access to addiction research. Today, interest in open science is growing, in part due to the replication crisis first observed by the Many Labs research initiative (Nosek & Lakens, 2014; Open Science Collaborative, 2015). This month’s Special Series about contemporary open science practices discusses ways that researchers and academic journals can support open science and features research that has used such strategies.
Researchers have implicated a variety of research practices in the development of the replication crisis. Many of these relate to so-called researcher degrees of freedom (Wicherts et al., 2016), which include research practices and decisions that intentionally or unintentionally increase the likelihood of observing and reporting statistically significant findings. However, the peer review system itself might play a role in the replication crisis through publication bias (e.g., publication preferences for novel significant findings; Ferguson & Heene, 2012). This is problematic because as Asendorpf (2013) noted: “When a paper contains only one perfect but underpowered demonstration of an effect, high powered replication studies are needed before much credibility can be given to the observed effect.” This suggests that although researchers can act in ways that increase the likelihood of replicable research (by pre-registering their studies, for example), changes in the editorial process can facilitate the growth of a replicable literature as well. Some possibilities for the peer review publishing process discussed by open science supporters include the following ideas:
- Require a pre-registration statement. In a Methods or Limitations section, researchers would provide the enduring link to their pre-registration, or explain that the study was not pre-registered and therefore findings should be considered exploratory until replicated.
- Require a data availability statement. In a Methods or Limitations section, researchers would report where data are available upon publication, or include a statement that explains why data are not publicly available.
- Allow pre-prints on the Open Science Framework, psyarxiv, or similar services.
- Allow high-powered, high-quality replication studies to be published, even though they lack the novelty of new discoveries.
- Proactively solicit high-powered research replications of important but underpowered, unexpected, or controversial findings.
- Certify open science practice using open science badges, which research shows increase data sharing (Kidwell et al., 2016; Rowhani-Farid et al., 2017).
To promote awareness of the use of contemporary open science research practices for addiction research, this Special Series highlights published papers that have employed some of these techniques. Each featured open access paper has available research pre-registration documents. First up, The DRAM features a pre-registered study related to self-affirmation and receptiveness to alcohol warnings. Next, The WAGER features a pre-registered study concerning gambling-related judgments and graphical depictions of payback information. Following this, ASHES features pre-registered research about the effectiveness and believability of tobacco health warnings. Finally, STASH features pre-registered research that addresses patient-centered care for addiction treatment. We will round out this Special Series with guest editorials from Drs. Rob Heirene & Sally Gainsbury, Can the Open Science Revolution Revolutionise Gambling Research?, and Dr. David Mellor, Open Science Practices to Support Addiction Research.
We hope you will enjoy and learn from this Special Series. Please share your feedback using the comment link below or within the individual posts. Don’t miss our earlier announcements related to the Division on Addiction’s open science practices!
— Debi A. LaPlante, Director, Division on Addiction at Cambridge Health Alliance, a Harvard Medical School teaching hospital; Assistant Professor, Harvard Medical School
References
Asendorpf, J. B., Conner, M. De Fruyt, F., De Houwer, J., … Wicherts, J. M. (2013). Recommendations for Increasing Replicability in Psychology. European Journal of Personality, 27(2), 108-119.
Ferguson, C. J., & Heene, M. (2012). A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null. Perspective on Psychological Science, 7(6), 555-561. doi:10.1177/1745691612459059
Kidwell MC, Lazarević LB, Baranski E, Hardwicke TE, Piechowski S, Falkenberg L-S, et al. (2016) Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biol 14(5): e1002456. https://doi.org/10.1371/journal.pbio.1002456
Nosek, B. A., & Lakens, D. (2014). Registered Reports: A Method to Increase the Credibility of Published Results. Social Psychology, 45(3), 137-141. doi:10.1027/1864-9335/a000192
Open Science Collaborative. (2015). Estimating the Reproducibility of Psychological Science. Science, 349(6251), aac4716. doi:10.1126/science.aac4716
Rowhani-Farid, A., Allen, M. & Barnett, A.G. What incentives increase data sharing in health and medical research? A systematic review. Res Integr Peer Rev 2, 4 (2017) doi:10.1186/s41073-017-0028-9
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A., L. M. (2016). Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking. Frontiers in Psychology, 7, 1-12. doi:10.3389/fpsyg.2016.01832
Acknowledgements
I completed this editorial with support from GVC Holdings, PLC via a research grant to the Division on Addiction. GVC Holdings, PLC did not contribute to the development of this editorial.
The Division on Addiction currently receives funding from DraftKings, Inc., The Foundation for Advancing Alcohol Responsibility (FAAR), The Healing Lodge of the Seven Nations via the Indian Health Service with funds approved by the National Institute of General Medical Sciences, National Institutes of Health; The Integrated Centre on Addiction Prevention and Treatment of the Tung Wah Group of Hospitals; the Gavin Foundation via the Substance Abuse and Mental Health Services Administration (SAMHSA); University of Nevada, Las Vegas via MGM Resorts International; and GVC Holdings, PLC.
During the past 5 years, I have received speaker honoraria and travel support from the National Center for Responsible Gaming (NCRG) and the National Collegiate Athletic Association. I have served as a paid grant reviewer for NCRG and received honoraria funds for preparation of a book chapter from Universite Laval. I am a non-paid board member of the New Hampshire Council on Problem Gambling.