false positive rate
   HOME

TheInfoList



OR:

In
statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, when performing
multiple comparisons In statistics, the multiple comparisons, multiplicity or multiple testing problem occurs when one considers a set of statistical inferences simultaneously or infers a subset of parameters selected based on the observed values. The more inferences ...
, a false positive ratio (also known as fall-out or false alarm ratio) is the
probability Probability is the branch of mathematics concerning numerical descriptions of how likely an Event (probability theory), event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and ...
of falsely rejecting the
null hypothesis In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
for a particular
test Test(s), testing, or TEST may refer to: * Test (assessment), an educational assessment intended to measure the respondents' knowledge or other abilities Arts and entertainment * ''Test'' (2013 film), an American film * ''Test'' (2014 film), ...
. The false positive rate is calculated as the ratio between the number of negative events wrongly categorized as positive (false positives) and the total number of actual negative events (regardless of classification). The false positive rate (or "false alarm rate") usually refers to the expectancy of the false positive ratio.


Definition

The false positive rate is FPR=\frac where \mathrm is the number of false positives, \mathrm is the number of true negatives and N=\mathrm+\mathrm is the total number of ground truth negatives. The level of significance that is used to test each hypothesis is set based on the form of inference ( simultaneous inference vs. selective inference) and its supporting criteria (for example FWER or
FDR Franklin Delano Roosevelt (; ; January 30, 1882April 12, 1945), often referred to by his initials FDR, was an American politician and attorney who served as the 32nd president of the United States from 1933 until his death in 1945. As the ...
), that were pre-determined by the researcher. When performing
multiple comparisons In statistics, the multiple comparisons, multiplicity or multiple testing problem occurs when one considers a set of statistical inferences simultaneously or infers a subset of parameters selected based on the observed values. The more inferences ...
in a
statistical Statistics (from German: ''Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industria ...
framework such as above, the false positive ratio (also known as the false alarm ratio, as opposed to false positive rate / false alarm rate ) usually refers to the probability of falsely rejecting the
null hypothesis In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
for a particular
test Test(s), testing, or TEST may refer to: * Test (assessment), an educational assessment intended to measure the respondents' knowledge or other abilities Arts and entertainment * ''Test'' (2013 film), an American film * ''Test'' (2014 film), ...
. Using the terminology suggested here, it is simply V/m_0. Since ''V'' is a random variable and ''m_0'' is a constant ( V \leq m_0 ), the false positive ratio is also a random variable, ranging between 0-1.
The false positive rate (or "false alarm rate") usually refers to the expectancy of the false positive ratio, expressed by E(V/m_0). It is worth noticing that the two definitions ("false positive ratio" / "false positive rate") are somewhat interchangeable. For example, in the referenced article V/m_0 serves as the false positive "rate" rather than as its "ratio".


Classification of multiple hypothesis tests


Comparison with other error rates

While the false positive rate is mathematically equal to the
type I error rate In statistical hypothesis testing, a type I error is the mistaken rejection of an actually true null hypothesis (also known as a "false positive" finding or conclusion; example: "an innocent person is convicted"), while a type II error is the fa ...
, it is viewed as a separate term for the following reasons: * The type I error rate is often associated with the a-priori setting of the
significance level In statistical hypothesis testing, a result has statistical significance when it is very unlikely to have occurred given the null hypothesis (simply by chance alone). More precisely, a study's defined significance level, denoted by \alpha, is the ...
by the researcher: the significance level represents an acceptable error rate considering that all null hypotheses are true (the "global null" hypothesis). The choice of a significance level may thus be somewhat arbitrary (i.e. setting 10% (0.1), 5% (0.05), 1% (0.01) etc.) : As opposed to that, the false positive rate is associated with a post-prior result, which is the expected number of false positives divided by the total number of hypotheses under the real combination of true and non-true null hypotheses (disregarding the "global null" hypothesis). Since the false positive rate is a parameter that is not controlled by the researcher, it cannot be identified with the significance level. * Moreover, false positive rate is usually used regarding a medical test or diagnostic device (i.e. "the false positive rate of a certain diagnostic device is 1%"), while type I error is a term associated with statistical tests, where the meaning of the word "positive" is not as clear (i.e. "the type I error of a test is 1%"). The false positive rate should also not be confused with the
family-wise error rate In statistics, family-wise error rate (FWER) is the probability of making one or more false discoveries, or type I errors when performing multiple hypotheses tests. Familywise and Experimentwise Error Rates Tukey (1953) developed the concept of a ...
, which is defined as \mathrm{FWER} = \Pr(V \ge 1)\,. As the number of tests grows, the familywise error rate usually converges to 1 while the false positive rate remains fixed. Lastly, it is important to note the profound difference between the false positive rate and the
false discovery rate In statistics, the false discovery rate (FDR) is a method of conceptualizing the rate of type I errors in null hypothesis testing when conducting multiple comparisons. FDR-controlling procedures are designed to control the FDR, which is the expec ...
: while the first is defined as E(V/m_0), the second is defined as E(V/R).


See also

*
False positives and false negatives A false positive is an error in binary classification in which a test result incorrectly indicates the presence of a condition (such as a disease when the disease is not present), while a false negative is the opposite error, where the test result ...
*
False coverage rate In statistics, a false coverage rate (FCR) is the average rate of false coverage, i.e. not covering the true parameters, among the selected intervals. The FCR gives a simultaneous coverage at a (1 − ''α'')×100% level for all ...
*
False discovery rate In statistics, the false discovery rate (FDR) is a method of conceptualizing the rate of type I errors in null hypothesis testing when conducting multiple comparisons. FDR-controlling procedures are designed to control the FDR, which is the expec ...
*
Sensitivity and specificity ''Sensitivity'' and ''specificity'' mathematically describe the accuracy of a test which reports the presence or absence of a condition. Individuals for which the condition is satisfied are considered "positive" and those for which it is not are ...


References

Multiple comparisons Statistical tests Analysis of variance Statistical ratios