"Why Most Published Research Findings Are False" is a 2005 essay written by
John Ioannidis
John P. A. Ioannidis (; el, Ιωάννης Ιωαννίδης, ; born August 21, 1965) is a Greek-American physician-scientist, writer and Stanford University professor who has made contributions to evidence-based medicine, epidemiology, and c ...
, a professor at the
Stanford School of Medicine
Stanford University School of Medicine is the medical school of Stanford University and is located in Stanford, California. It traces its roots to the Medical Department of the University of the Pacific, founded in San Francisco in 1858. This ...
, and published in ''
PLOS Medicine
''PLOS Medicine'' (formerly styled ''PLoS Medicine'') is a peer-reviewed weekly medical journal covering the full spectrum of the medical sciences. It began operation on October 19, 2004, as the second journal of the Public Library of Science (PLO ...
''. It is considered foundational to the field of
metascience
Metascience (also known as meta-research) is the use of scientific methodology to study science itself. Metascience seeks to increase the quality of scientific research while reducing inefficiency. It is also known as "''research on research''" ...
.
In the paper, Ioannidis argued that a large number, if not the majority, of published
medical research
Medical research (or biomedical research), also known as experimental medicine, encompasses a wide array of research, extending from " basic research" (also called ''bench science'' or ''bench research''), – involving fundamental scienti ...
papers contain results that cannot be
replicated. In simple terms, the essay states that scientists use
hypothesis testing
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Hypothesis testing allows us to make probabilistic statements about population parameters.
...
to determine whether scientific discoveries are significant.
"Significance" is formalized in terms of probability, and one formalized calculation ("''
P value
In statistical hypothesis testing, null-hypothesis significance testing, the ''p''-value is the probability of obtaining test results at least as extreme as the Realization (probability), result actually observed, under the assumption that the nu ...
''") is reported in the scientific literature as a screening mechanism. Ioannidis posited assumptions about the way people perform and report these tests; then he constructed a statistical model which indicates that most published findings are
false positive results.
Argument
Suppose that in a given scientific field there is a known baseline probability that a result is true, denoted by
. When a study is conducted, the probability that a positive result is obtained is
. Given these two factors, we want to compute the
conditional probability
In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occu ...
, which is known as the
positive predictive value
The positive and negative predictive values (PPV and NPV respectively) are the proportions of positive and negative results in statistics and diagnostic tests that are true positive and true negative results, respectively. The PPV and NPV de ...
(PPV).
Bayes' theorem
In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule), named after Thomas Bayes, describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For exa ...
allows us to compute the PPV as:
where
is the
type I error rate
In statistical hypothesis testing, a type I error is the mistaken rejection of an actually true null hypothesis (also known as a "false positive" finding or conclusion; example: "an innocent person is convicted"), while a type II error is the fa ...
and
is the
type II error rate; the
statistical power
In statistics, the power of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis (H_0) when a specific alternative hypothesis (H_1) is true. It is commonly denoted by 1-\beta, and represents the chances ...
is
. It is customary in most scientific research to desire
and
. If we assume
for a given scientific field, then we may compute the PPV for different values of
and
:
However, the simple formula for PPV derived from Bayes' theorem does not account for
bias
Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is closed-minded, prejudicial, or unfair. Biases can be innate or learned. People may develop biases for or against an individual, a group ...
in study design or reporting. Some published findings would not have been presented as research findings if not for researcher bias. Let