The base rate fallacy, also called base rate neglect or base rate
bias, is a formal fallacy. If presented with related base rate
information (i.e. generic, general information) and specific
information (information pertaining only to a certain case), the mind
tends to ignore the former and focus on the latter.[1]
Contents 1 Examples 1.1 Example 1: Drunk drivers 1.2 Example 2: Terrorist identification 2 Findings in psychology 3 See also 4 References 5 External links Examples[edit] Example 1: Drunk drivers[edit] A group of police officers have breathalyzers displaying false drunkenness in 5% of the cases in which the driver is sober. However, the breathalyzers never fail to detect a truly drunk person. One in a thousand drivers is driving drunk. Suppose the police officers then stop a driver at random, and force the driver to take a breathalyzer test. It indicates that the driver is drunk. We assume you don't know anything else about him or her. How high is the probability he or she really is drunk? Many would answer as high as 95%, but the correct probability is about 2%. An explanation for this is as follows: on average, for every 1,000 drivers tested, 1 driver is drunk, and it is 100% certain that for that driver there is a true positive test result, so there is 1 true positive test result 999 drivers are not drunk, and among those drivers there are 5% false positive test results, so there are 49.95 false positive test results Therefore, the probability that one of the drivers among the 1 + 49.95 = 50.95 positive test results really is drunk is 1 / 50.95 ≈ 0.019627 displaystyle 1/50.95approx 0.019627 . The validity of this result does, however, hinge on the validity of the initial assumption that the police officer stopped the driver truly at random, and not because of bad driving. If that or another non-arbitrary reason for stopping the driver was present, then the calculation also involves the probability of a drunk driver driving competently and a non-drunk driver driving (in-)competently. More formally, the same probability of roughly 0.02 can be established using Bayes's theorem. The goal is to find the probability that the driver is drunk given that the breathalyzer indicated he/she is drunk, which can be represented as p ( d r u n k
D ) displaystyle p(mathrm drunk D) where "D" means that the breathalyzer indicates that the driver is
drunk.
p ( d r u n k
D ) = p ( D
d r u n k ) p ( d r u n k ) p ( D ) displaystyle p(mathrm drunk D)= frac p(Dmathrm drunk ),p(mathrm drunk ) p(D) We were told the following in the first paragraph: p ( d r u n k ) = 0.001 displaystyle p(mathrm drunk )=0.001 p ( s o b e r ) = 0.999 displaystyle p(mathrm sober )=0.999 p ( D
d r u n k ) = 1.00 displaystyle p(Dmathrm drunk )=1.00 p ( D
s o b e r ) = 0.05 displaystyle p(Dmathrm sober )=0.05 As you can see from the formula, one needs p(D) for Bayes' theorem, which one can compute from the preceding values using p ( D ) = p ( D
d r u n k ) p ( d r u n k ) + p ( D
s o b e r ) p ( s o b e r ) displaystyle p(D)=p(Dmathrm drunk ),p(mathrm drunk )+p(Dmathrm sober ),p(mathrm sober ) which gives p ( D ) = ( 1.00 × 0.001 ) + ( 0.05 × 0.999 ) = 0.05095 displaystyle p(D)=(1.00times 0.001)+(0.05times 0.999)=0.05095 Plugging these numbers into Bayes' theorem, one finds that p ( d r u n k
D ) = 1.00 × 0.001 0.05095 = 0.019627 ⋅ displaystyle p(mathrm drunk D)= frac 1.00times 0.001 0.05095 =0.019627cdot Example 2: Terrorist identification[edit] In a city of 1 million inhabitants let there be 100 terrorists and 999,900 non-terrorists. To simplify the example, it is assumed that all people present in the city are inhabitants. Thus, the base rate probability of a randomly selected inhabitant of the city being a terrorist is 0.0001, and the base rate probability of that same inhabitant being a non-terrorist is 0.9999. In an attempt to catch the terrorists, the city installs an alarm system with a surveillance camera and automatic facial recognition software. The software has two failure rates of 1%: The false negative rate: If the camera scans a terrorist, a bell will ring 99% of the time, and it will fail to ring 1% of the time. The false positive rate: If the camera scans a non-terrorist, a bell will not ring 99% of the time, but it will ring 1% of the time. Suppose now that an inhabitant triggers the alarm. What is the chance
that the person is a terrorist? In other words, what is P(T B), the
probability that a terrorist has been detected given the ringing of
the bell? Someone making the 'base rate fallacy' would infer that
there is a 99% chance that the detected person is a terrorist.
Although the inference seems to make sense, it is actually bad
reasoning, and a calculation below will show that the chances he/she
is a terrorist are actually near 1%, not near 99%.
The fallacy arises from confusing the natures of two different failure
rates. The 'number of non-bells per 100 terrorists' and the 'number of
non-terrorists per 100 bells' are unrelated quantities. One does not
necessarily equal the other, and they don't even have to be almost
equal. To show this, consider what happens if an identical alarm
system were set up in a second city with no terrorists at all. As in
the first city, the alarm sounds for 1 out of every 100 non-terrorist
inhabitants detected, but unlike in the first city, the alarm never
sounds for a terrorist. Therefore, 100% of all occasions of the alarm
sounding are for non-terrorists, but a false negative rate cannot even
be calculated. The 'number of non-terrorists per 100 bells' in that
city is 100, yet P(T B) = 0%. There is zero chance that a terrorist
has been detected given the ringing of the bell.
Imagine that the first city's entire population of one million people
pass in front of the camera. About 99 of the 100 terrorists will
trigger the alarm—and so will about 9,999 of the 999,900
non-terrorists. Therefore, about 10,098 people will trigger the alarm,
among which about 99 will be terrorists. So, the probability that a
person triggering the alarm actually is a terrorist, is only about 99
in 10,098, which is less than 1%, and very, very far below our initial
guess of 99%.
The base rate fallacy is so misleading in this example because there
are many more non-terrorists than terrorists, and the number of false
positives (non-terrorists scanned as terrorists) is so much larger
than the true positives (the real number of terrorists).
Findings in psychology[edit]
In experiments, people have been found to prefer individuating
information over general information when the former is
available.[2][3][4]
In some experiments, students were asked to estimate the grade point
averages (GPAs) of hypothetical students. When given relevant
statistics about GPA distribution, students tended to ignore them if
given descriptive information about the particular student even if the
new descriptive information was obviously of little or no relevance to
school performance.[3] This finding has been used to argue that
interviews are an unnecessary part of the college admissions process
because interviewers are unable to pick successful candidates better
than basic statistics.
Psychologists
1 out of 1000 drivers are driving drunk. The breathalyzers never fail to detect a truly drunk person. For 50 out of the 999 drivers who are not drunk the breathalyzer falsely displays drunkness. Suppose the policemen then stop a driver at random, and force them to take a breathalyzer test. It indicates that he or she is drunk. We assume you don't know anything else about him or her. How high is the probability he or she really is drunk? In this case, the relevant numerical information—p(drunk), p(D drunk), p(D sober)—is presented in terms of natural frequencies with respect to a certain reference class (see reference class problem). Empirical studies show that people's inferences correspond more closely to Bayes' rule when information is presented this way, helping to overcome base-rate neglect in laypeople[11] and experts.[12] As a consequence, organizations like the Cochrane Collaboration recommend using this kind of format for communicating health statistics.[13] Teaching people to translate these kinds of Bayesian reasoning problems into natural frequency formats is more effective than merely teaching them to plug probabilities (or percentages) into Bayes' theorem.[14] It has also been shown that graphical representations of natural frequencies (e.g., icon arrays) help people to make better inferences.[14][15][16] Why are natural frequency formats helpful? One important reason is that this information format facilitates the required inference because it simplifies the necessary calculations. This can be seen when using an alternative way of computing the required probability p(drunkD): p ( d r u n k
D ) = N ( d r u n k ∩ D ) N ( D ) = 1 51 = 0.0196 displaystyle p(mathrm drunk D)= frac N(mathrm drunk cap D) N(D) = frac 1 51 =0.0196 where N(drunk ∩ D) denotes the number of drivers that are drunk and get a positive breathalyzer result, and N(D) denotes the total number of cases with a positive breathalyzer result. The equivalence of this equation to the above one follows from the axioms of probability theory, according to which N(drunk ∩ D) = N × p (D drunk) × p (drunk). Importantly, although this equation is formally equivalent to Bayes’ rule, it is not psychologically equivalent. Using natural frequencies simplifies the inference because the required mathematical operation can be performed on natural numbers, instead of normalized fractions (i.e., probabilities), because it makes the high number of false positives more transparent, and because natural frequencies exhibit a "nested-set structure".[17][18] It is important to note that not any kind of frequency format facilitates Bayesian reasoning.[18][19] Natural frequencies refer to frequency information that results from natural sampling,[20] which preserves base rate information (e.g., number of drunken drivers when taking a random sample of drivers). This is different from systematic sampling, in which base rates are fixed a priori (e.g., in scientific experiments). In the latter case it is not possible to infer the posterior probability p (drunk positive test) from comparing the number of drivers who are drunk and test positive compared to the total number of people who get a positive breathalyzer result, because base rate information is not preserved and must be explicitly re-introduced using Bayes' theorem. See also[edit] Bayesian probability Data dredging False positive paradox Inductive argument List of cognitive biases Misleading vividness Prosecutor's fallacy Stereotype References[edit] ^ "Logical Fallacy: The Base Rate Fallacy". Fallacyfiles.org.
Retrieved 2013-06-15.
^ Bar-Hillel, Maya (1980). "The base-rate fallacy in probability
judgments". Acta Psychologica. 44: 211–233.
doi:10.1016/0001-6918(80)90046-3.
^ a b c Kahneman, Daniel;
External links[edit] The Base Rate Fallacy The Fallacy Files Psychology of Intelligence Analysis: Base Rate Fallacy The base rate fallacy explained visually (Video) Interactive page for visualizing statistical information and Bayesian inference problems Current ‘best practice’ for communicating probabilities in health according to the International Patient Decision Aid Standards (IPDAS) Collaboration v t e Fallacies of relevance
Appeal to worse problems Two wrongs make a right
Appeals to emotion Fear
Flattery
Novelty
Pity
Ridicule
Think of the children
In-group favoritism
Genetic fallacies Ad hominem Abusive Association reductio ad Hitlerum Godwin's law reductio ad Stalinum Circumstantial Appeal to motive Bulverism Poisoning the well Tone Tu quoque Authority Accomplishment Ipse dixit Poverty / Wealth Etymology Nature Tradition / Novelty Chronological snobbery Appeals to consequences Argumentum ad baculum Appeal to force Wishful thinking Other appeals Argument to moderation Popularity List of fallacies Other types of fallacy |