The Reproducibility Project is a series of crowdsourced collaborations aiming to reproduce published scientific studies, finding high rates of results which could not be replicated. It has resulted in two major initiatives focusing on the fields of
psychology
Psychology is the scientific study of mind and behavior. Its subject matter includes the behavior of humans and nonhumans, both consciousness, conscious and Unconscious mind, unconscious phenomena, and mental processes such as thoughts, feel ...
and
cancer
Cancer is a group of diseases involving Cell growth#Disorders, abnormal cell growth with the potential to Invasion (cancer), invade or Metastasis, spread to other parts of the body. These contrast with benign tumors, which do not spread. Po ...
biology. The project has brought attention to the
replication crisis
The replication crisis, also known as the reproducibility or replicability crisis, refers to the growing number of published scientific results that other researchers have been unable to reproduce or verify. Because the reproducibility of empir ...
, and has contributed to shifts in scientific culture and publishing practices to address it.
The project was led by the
Center for Open Science
The Center for Open Science is a non-profit technology organization based in Charlottesville, Virginia with a mission to "increase the openness, integrity, and reproducibility of scientific research." Brian Nosek and Jeffrey Spies founded the o ...
and its co-founder,
Brian Nosek, who started the project in November 2011.
Results
Brian Nosek of
University of Virginia
The University of Virginia (UVA) is a Public university#United States, public research university in Charlottesville, Virginia, United States. It was founded in 1819 by Thomas Jefferson and contains his The Lawn, Academical Village, a World H ...
and colleagues sought out to replicate 100 different studies, all published in 2008. The project pulled these studies from three different journals'',
Psychological Science
''Psychological Science'', the flagship journal of the Association for Psychological Science, is a monthly, peer-reviewed scientific journal published by SAGE Publications. The journal publishes research articles, short reports, and research repor ...
'', the ''
Journal of Personality and Social Psychology
The ''Journal of Personality and Social Psychology'' is a monthly peer-reviewed scientific journal published by the American Psychological Association that was established in 1965. It covers the fields of social and personality psychology. The edi ...
'', and the ''
Journal of Experimental Psychology: Learning, Memory, and Cognition'', published in 2008 to see if they could get the same results as the initial findings. 97 of the original studies had significant effects, but of those 97, only 36% of the replications yielded significant findings (''p'' value below 0.05),
and the effects were often smaller than those in the original papers. The authors emphasized that the findings reflect a problem that affects all of science and not just psychology, and that there is room to improve reproducibility in psychology.
In 2021, the project showed that of 193 experiments from 53 top papers about cancer published between 2010 and 2012, only 50 experiments from 23 papers could be replicated. Moreover, it showed that the effect sizes of that fraction were 85% smaller on average than the original findings. None of the papers had its experimental
protocols fully described and 70% of experiments required asking for key reagents.
Impact
The project, along with broader action in response to the
replication crisis
The replication crisis, also known as the reproducibility or replicability crisis, refers to the growing number of published scientific results that other researchers have been unable to reproduce or verify. Because the reproducibility of empir ...
, has helped spur changes in scientific culture and publishing practices.
The results of the Reproducibility Project might also affect public trust in psychology.
Lay people who learned about the low replication rate found in the Reproducibility Project subsequently reported a lower trust in psychology, compared to people who were told that a high number of the studies had replicated.
See also
*
Invalid science
*
John Ioannidis
John P. A. Ioannidis ( ; , ; born August 21, 1965) is a Greek-American physician-scientist, writer and Stanford University professor who has made contributions to evidence-based medicine, epidemiology, and clinical research. Ioannidis studies sc ...
*
Meta-analysis
Meta-analysis is a method of synthesis of quantitative data from multiple independent studies addressing a common research question. An important part of this method involves computing a combined effect size across all of the studies. As such, th ...
*
Metascience
Metascience (also known as meta-research) is the use of scientific methodology to study science itself. Metascience seeks to increase the quality of scientific research while reducing inefficiency. It is also known as "research on research" and ...
*
Proteus phenomenon
*
Publication bias
In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a Statistical significance, significant find ...
*
Replication crisis
The replication crisis, also known as the reproducibility or replicability crisis, refers to the growing number of published scientific results that other researchers have been unable to reproduce or verify. Because the reproducibility of empir ...
*
Scientific method
The scientific method is an Empirical evidence, empirical method for acquiring knowledge that has been referred to while doing science since at least the 17th century. Historically, it was developed through the centuries from the ancient and ...
External links
Official website for the Reproducibility Project: PsychologyOfficial website for the Reproducibility Project: Cancer Biology
References
{{Use dmy dates, date=October 2019
Validity (statistics)
Scientific method
Criticism of academia
Metascience