The raven paradox, also known as Hempel's paradox, Hempel's ravens, or rarely the paradox of indoor ornithology, is a
paradox
A paradox is a logically self-contradictory statement or a statement that runs contrary to one's expectation. It is a statement that, despite apparently valid reasoning from true premises, leads to a seemingly self-contradictory or a logically u ...
arising from the question of what constitutes
evidence
Evidence for a proposition is what supports this proposition. It is usually understood as an indication that the supported proposition is true. What role evidence plays and how it is conceived varies from field to field.
In epistemology, evidenc ...
for the truth of a statement. Observing objects that are neither black nor ravens may formally increase the likelihood that all ravens are black even though, intuitively, these observations are unrelated.
This problem was proposed by the logician
Carl Gustav Hempel
Carl Gustav "Peter" Hempel (January 8, 1905 – November 9, 1997) was a German writer, philosopher, logician, and epistemologist. He was a major figure in logical empiricism, a 20th-century movement in the philosophy of science. He is espec ...
in the 1940s to illustrate a contradiction between
inductive logic
Inductive reasoning is a method of reasoning in which a general principle is derived from a body of observations. It consists of making broad generalizations based on specific observations. Inductive reasoning is distinct from ''deductive'' re ...
and
intuition
Intuition is the ability to acquire knowledge without recourse to conscious reasoning. Different fields use the word "intuition" in very different ways, including but not limited to: direct access to unconscious knowledge; unconscious cognition; ...
.
Paradox
Hempel describes the paradox in terms of the
hypothesis
A hypothesis (plural hypotheses) is a proposed explanation for a phenomenon. For a hypothesis to be a scientific hypothesis, the scientific method requires that one can test it. Scientists generally base scientific hypotheses on previous obse ...
:
: (1) ''All
raven
A raven is any of several larger-bodied bird species of the genus ''Corvus''. These species do not form a single taxonomic group within the genus. There is no consistent distinction between "crows" and "ravens", common names which are assigned t ...
s are black''. In the form of an implication, this can be expressed as: ''If something is a raven, then it is black.''
Via
contraposition
In logic and mathematics, contraposition refers to the inference of going from a conditional statement into its logically equivalent contrapositive, and an associated proof method known as Proof by contrapositive, proof by contraposition. The cont ...
, this statement is
equivalent
Equivalence or Equivalent may refer to:
Arts and entertainment
*Album-equivalent unit, a measurement unit in the music industry
*Equivalence class (music)
*''Equivalent VIII'', or ''The Bricks'', a minimalist sculpture by Carl Andre
*''Equivale ...
to:
: (2) ''If something is not black, then it is not a raven.''
In all circumstances where (2) is true, (1) is also true—and likewise, in all circumstances where (2) is false (i.e., if a world is imagined in which something that was not black, yet was a raven, existed), (1) is also false.
Given a general statement such as ''all ravens are black'', a form of the same statement that refers to a specific observable instance of the general class would typically be considered to constitute evidence for that general statement. For example,
: (3) ''My pet raven is black.''
is evidence supporting the hypothesis that ''all ravens are black''.
The paradox arises when this same process is applied to statement (2). On sighting a green apple, one can observe:
: (4) ''This green apple is not black, and it is not a raven.''
By the same reasoning, this statement is evidence that (2) ''if something is not black then it is not a raven.'' But since (as above) this statement is logically equivalent to (1) ''all ravens are black'', it follows that the sight of a green apple is evidence supporting the notion that all ravens are black. This conclusion seems paradoxical because it implies that information has been gained about ravens by looking at an apple.
Proposed resolutions
Nicod's criterion says that only observations of ravens should affect one's view as to whether all ravens are black. Observing more instances of black ravens should support the view, observing white or coloured ravens should contradict it, and observations of non-ravens should not have any influence.
Hempel's equivalence condition states that when a proposition, X, provides evidence in favor of another proposition Y, then X also provides evidence in favor of any proposition that is logically equivalent to Y.
Realistically, the set of ravens is finite. The set of non-black things is either infinite or beyond human enumeration. To confirm the statement 'All ravens are black', it would be necessary to observe all ravens. This is difficult but possible. To confirm the statement 'All non-black things are non-ravens', it would be necessary to examine all non-black things. This is not possible. Observing a black raven could be considered a finite amount of confirmatory evidence, but observing a non-black non-raven would be an
infinitesimal
In mathematics, an infinitesimal number is a quantity that is closer to zero than any standard real number, but that is not zero. The word ''infinitesimal'' comes from a 17th-century Modern Latin coinage ''infinitesimus'', which originally referr ...
amount of evidence.
The paradox shows that Nicod's criterion and Hempel's equivalence condition are not mutually consistent. A resolution to the paradox must reject at least one out of:
# negative instances having no influence (!PC),
# equivalence condition (EC), or,
# validation by positive instances (NC).
A satisfactory resolution should also explain ''why'' there naively appears to be a paradox. Solutions that accept the paradoxical conclusion can do this by presenting a proposition that we intuitively know to be false but that is easily confused with (PC), while solutions that reject (EC) or (NC) should present a proposition that we intuitively know to be true but that is easily confused with (EC) or (NC).
Accepting non-ravens as relevant
Although this conclusion of the paradox seems counter-intuitive, some approaches accept that observations of (coloured) non-ravens can in fact constitute valid evidence in support for hypotheses about (the universal blackness of) ravens.
Hempel's resolution
Hempel himself accepted the paradoxical conclusion, arguing that the reason the result appears paradoxical is that we possess prior information without which the observation of a non-black non-raven would indeed provide evidence that all ravens are black.
He illustrates this with the example of the generalization "All sodium salts burn yellow," and asks us to consider the observation that occurs when somebody holds a piece of pure ice in a colorless flame that does not turn yellow:
Standard Bayesian solution
One of the most popular proposed resolutions is to accept the conclusion that the observation of a green apple provides evidence that all ravens are black but to argue that the amount of confirmation provided is very small, due to the large discrepancy between the number of ravens and the number of non-black objects. According to this resolution, the conclusion appears paradoxical because we intuitively estimate the amount of evidence provided by the observation of a green apple to be zero, when it is in fact non-zero but extremely small.
I. J. Good
Irving John Good (9 December 1916 – 5 April 2009)The Times of 16-apr-09, http://www.timesonline.co.uk/tol/comment/obituaries/article6100314.ece
was a British mathematician who worked as a cryptologist at Bletchley Park with Alan Turing. Afte ...
's presentation of this argument in 1960
is perhaps the best known, and variations of the argument have been popular ever since,
although it had been presented in 1958 and early forms of the argument appeared as early as 1940.
Good's argument involves calculating the weight of evidence provided by the observation of a black raven or a white shoe in favor of the hypothesis that all the ravens in a collection of objects are black. The weight of evidence is the logarithm of the
Bayes factor
The Bayes factor is a ratio of two competing statistical models represented by their marginal likelihood, and is used to quantify the support for one model over the other. The models in questions can have a common set of parameters, such as a nu ...
, which in this case is simply the factor by which the
odds
Odds provide a measure of the likelihood of a particular outcome. They are calculated as the ratio of the number of events that produce that outcome to the number that do not. Odds are commonly used in gambling and statistics.
Odds also have ...
of the hypothesis changes when the observation is made. The argument goes as follows:
:... suppose that there are
objects that might be seen at any moment, of which
are ravens and
are black, and that the
objects each have probability
of being seen. Let
be the hypothesis that there are
non-black ravens, and suppose that the hypotheses
are initially equiprobable. Then, if we happen to see a black raven, the Bayes factor in favour of
is
:::
:i.e. about 2 if the number of ravens in existence is known to be large. But the factor if we see a white shoe is only
:::
:::
:and this exceeds unity by only about
if
is large compared to
. Thus the weight of evidence provided by the sight of a white shoe is positive, but is small if the number of ravens is known to be small compared to the number of non-black objects.
Many of the proponents of this resolution and variants of it have been advocates of Bayesian probability, and it is now commonly called the Bayesian Solution, although, as
Chihara observes, "there is no such thing as ''the'' Bayesian solution. There are many different 'solutions' that Bayesians have put forward using Bayesian techniques." Noteworthy approaches using Bayesian techniques (some of which accept !PC and instead reject NC) include Earman, Eells, Gibson,
Hosiasson-Lindenbaum,
[ Howson and Urbach, Mackie, and Hintikka,] who claims that his approach is "more Bayesian than the so-called 'Bayesian solution' of the same paradox". Bayesian approaches that make use of Carnap's theory of inductive inference include Humburg, Maher,[ and Fitelson & Hawthorne.][ Vranas introduced the term "Standard Bayesian Solution" to avoid confusion.
]
Carnap approach
Maher[ accepts the paradoxical conclusion, and refines it:
To reach (ii), he appeals to Carnap's theory of inductive probability, which is (from the Bayesian point of view) a way of assigning prior probabilities that naturally implements induction. According to Carnap's theory, the posterior probability, , that an object, , will have a predicate, , after the evidence has been observed, is:
:
where is the initial probability that has the predicate ; is the number of objects that have been examined (according to the available evidence ); is the number of examined objects that turned out to have the predicate , and is a constant that measures resistance to generalization.
If is close to zero, will be very close to one after a single observation of an object that turned out to have the predicate , while if is much larger than , will be very close to regardless of the fraction of observed objects that had the predicate .
Using this Carnapian approach, Maher identifies a proposition we intuitively (and correctly) know is false, but easily confuse with the paradoxical conclusion. The proposition in question is that observing non-ravens tells us about the color of ravens. While this is intuitively false and is also false according to Carnap's theory of induction, observing non-ravens (according to that same theory) causes us to reduce our estimate of the total number of ravens, and thereby reduces the estimated number of possible counterexamples to the rule that all ravens are black.
Hence, from the Bayesian-Carnapian point of view, the observation of a non-raven does not tell us anything about the color of ravens, but it tells us about the prevalence of ravens, and supports "All ravens are black" by reducing our estimate of the number of ravens that might not be black.
]
Role of background knowledge
Much of the discussion of the paradox in general and the Bayesian approach in particular has centred on the relevance of background knowledge. Surprisingly, Maher[ shows that, for a large class of possible configurations of background knowledge, the observation of a non-black non-raven provides ''exactly the same'' amount of confirmation as the observation of a black raven. The configurations of background knowledge that he considers are those that are provided by a ''sample proposition'', namely a proposition that is a ]conjunction
Conjunction may refer to:
* Conjunction (grammar), a part of speech
* Logical conjunction, a mathematical operator
** Conjunction introduction, a rule of inference of propositional logic
* Conjunction (astronomy), in which two astronomical bodies ...
of atomic propositions, each of which ascribes a single predicate to a single individual, with no two atomic propositions involving the same individual. Thus, a proposition of the form "A is a black raven and B is a white shoe" can be considered a sample proposition by taking "black raven" and "white shoe" to be predicates.
Maher's proof appears to contradict the result of the Bayesian argument, which was that the observation of a non-black non-raven provides much less evidence than the observation of a black raven. The reason is that the background knowledge that Good and others use can not be expressed in the form of a sample proposition – in particular, variants of the standard Bayesian approach often suppose (as Good did in the argument quoted above) that the total numbers of ravens, non-black objects and/or the total number of objects, are known quantities. Maher comments that, "The reason we think there are more non-black things than ravens is because that has been true of the things we have observed to date. Evidence of this kind can be represented by a sample proposition. But ... given any sample proposition as background evidence, a non-black non-raven confirms A just as strongly as a black raven does ... Thus my analysis suggests that this response to the paradox .e. the Standard Bayesian onecannot be correct."
Fitelson & Hawthorne[ examined the conditions under which the observation of a non-black non-raven provides less evidence than the observation of a black raven. They show that, if is an object selected at random, is the proposition that the object is black, and is the proposition that the object is a raven, then the condition:
:
is sufficient for the observation of a non-black non-raven to provide less evidence than the observation of a black raven. Here, a line over a proposition indicates the logical negation of that proposition.
This condition does not tell us ''how large'' the difference in the evidence provided is, but a later calculation in the same paper shows that the weight of evidence provided by a black raven exceeds that provided by a non-black non-raven by about . This is equal to the amount of additional information (in bits, if the base of the logarithm is 2) that is provided when a raven of unknown color is discovered to be black, given the hypothesis that not all ravens are black.
Fitelson & Hawthorne][ explain that:
:Under normal circumstances, may be somewhere around 0.9 or 0.95; so is somewhere around 1.11 or 1.05. Thus, it may appear that a single instance of a black raven does not yield much more support than would a non-black non-raven. However, under plausible conditions it can be shown that a sequence of instances (i.e. of n black ravens, as compared to n non-black non-ravens) yields a ratio of likelihood ratios on the order of , which blows up significantly for large .
The authors point out that their analysis is completely consistent with the supposition that a non-black non-raven provides an extremely small amount of evidence although they do not attempt to prove it; they merely calculate the difference between the amount of evidence that a black raven provides and the amount of evidence that a non-black non-raven provides.
]
Disputing the induction from positive instances
Some approaches for resolving the paradox focus on the inductive step. They dispute whether observation of a particular instance (such as one black raven) is the kind of evidence that necessarily ''increases'' confidence in the general hypothesis (such as that ravens are always black).
Red herring
Good gives an example of background knowledge with respect to which the observation of a black raven ''decreases'' the probability that all ravens are black:
:Suppose that we know we are in one or other of two worlds, and the hypothesis, H, under consideration is that all the ravens in our world are black. We know in advance that in one world there are a hundred black ravens, no non-black ravens, and a million other birds; and that in the other world there are a thousand black ravens, one white raven, and a million other birds. A bird is selected equiprobably at random from all the birds in our world. It turns out to be a black raven. This is strong evidence ... that we are in the second world, wherein not all ravens are black.
Good concludes that the white shoe is a "red herring
A red herring is a figurative expression referring to a logical fallacy in which a clue or piece of information is or is intended to be misleading, or distracting from the actual question.
Red herring may also refer to: Animals
* Red herring (fis ...
": Sometimes even a black raven can constitute evidence ''against'' the hypothesis that all ravens are black, so the fact that the observation of a white shoe can support it is not surprising and not worth attention. Nicod's criterion is false, according to Good, and so the paradoxical conclusion does not follow.
Hempel rejected this as a solution to the paradox, insisting that the proposition 'c is a raven and is black' must be considered "by itself and without reference to any other information", and pointing out that it "... was emphasized in section 5.2(b) of my article in ''Mind'' ... that the very appearance of paradoxicality in cases like that of the white shoe results in part from a failure to observe this maxim."
The question that then arises is whether the paradox is to be understood in the context of absolutely no background information (as Hempel suggests), or in the context of the background information that we actually possess regarding ravens and black objects, or with regard to all possible configurations of background information.
Good had shown that, for some configurations of background knowledge, Nicod's criterion is false (provided that we are willing to equate "inductively support" with "increase the probability of" – see below). The possibility remained that, with respect to our actual configuration of knowledge, which is very different from Good's example, Nicod's criterion might still be true and so we could still reach the paradoxical conclusion. Hempel, on the other hand, insists our background knowledge itself is the red herring, and that we should consider induction with respect to a condition of perfect ignorance.
Good's baby
In his proposed resolution, Maher implicitly made use of the fact that the proposition "All ravens are black" is highly probable when it is highly probable that there are no ravens. Good had used this fact before to respond to Hempel's insistence that Nicod's criterion was to be understood to hold in the absence of background information:
:...imagine an infinitely intelligent newborn baby having built-in neural circuits enabling him to deal with formal logic, English syntax, and subjective probability. He might now argue, after defining a raven in detail, that it is extremely unlikely that there are any ravens, and therefore it is extremely likely that all ravens are black, that is, that is true. 'On the other hand', he goes on to argue, 'if there are ravens, then there is a reasonable chance that they are of a variety of colours. Therefore, if I were to discover that even a black raven exists I would consider to be less probable than it was initially.'
This, according to Good, is as close as one can reasonably expect to get to a condition of perfect ignorance, and it appears that Nicod's condition is still false. Maher made Good's argument more precise by using Carnap's theory of induction to formalize the notion that if there is one raven, then it is likely that there are many.
Maher's argument considers a universe of exactly two objects, each of which is very unlikely to be a raven (a one in a thousand chance) and reasonably unlikely to be black (a one in ten chance). Using Carnap's formula for induction, he finds that the probability that all ravens are black decreases from 0.9985 to 0.8995 when it is discovered that one of the two objects is a black raven.
Maher concludes that not only is the paradoxical conclusion true, but that Nicod's criterion is false in the absence of background knowledge (except for the knowledge that the number of objects in the universe is two and that ravens are less likely than black things).
Distinguished predicates
Quine argued that the solution to the paradox lies in the recognition that certain predicate
Predicate or predication may refer to:
* Predicate (grammar), in linguistics
* Predication (philosophy)
* several closely related uses in mathematics and formal logic:
**Predicate (mathematical logic)
**Propositional function
**Finitary relation, o ...
s, which he called natural kind
"Natural kind" is an intellectual grouping, or categorizing of things, in a manner that is reflective of the actual world and not just human interests. Some treat it as a classification identifying some structure of truth and reality that exists wh ...
s, have a distinguished status with respect to induction. This can be illustrated with Nelson Goodman
Henry Nelson Goodman (7 August 1906 – 25 November 1998) was an American philosopher, known for his work on counterfactuals, mereology, the problem of induction, irrealism, and aesthetics.
Life and career
Goodman was born in Somerville, Mas ...
's example of the predicate grue. An object is grue if it is blue before (say) and green afterwards. Clearly, we expect objects that were blue before to remain blue afterwards, but we do not expect the objects that were found to be grue before to be blue after , since after they would be green. Quine's explanation is that "blue" is a natural kind; a privileged predicate we can use for induction, while "grue" is not a natural kind and using induction with it leads to error.
This suggests a resolution to the paradox – Nicod's criterion is true for natural kinds, such as "blue" and "black", but is false for artificially contrived predicates, such as "grue" or "non-raven". The paradox arises, according to this resolution, because we implicitly interpret Nicod's criterion as applying to all predicates when in fact it only applies to natural kinds.
Another approach, which favours specific predicates over others, was taken by Hintikka.[ Hintikka was motivated to find a Bayesian approach to the paradox that did not make use of knowledge about the ]relative frequencies
In statistics, the frequency (or absolute frequency) of an Event (probability theory), event i is the number n_i of times the observation has occurred/recorded in an experiment or study. These frequencies are often depicted graphically or in tabu ...
of ravens and black things. Arguments concerning relative frequencies, he contends, cannot always account for the perceived irrelevance of evidence consisting of observations of objects of type A for the purposes of learning about objects of type not-A.
His argument can be illustrated by rephrasing the paradox using predicates other than "raven" and "black". For example, "All men
are tall" is equivalent to "All short people are women", and so observing that a randomly selected person is a short woman should provide evidence that all men are tall. Despite the fact that we lack background knowledge to indicate that there are dramatically fewer men than short people, we still find ourselves inclined to reject the conclusion. Hintikka's example is: "... a generalization like 'no material bodies are infinitely divisible' seems to be completely unaffected by questions concerning immaterial entities, independently of what one thinks of the relative frequencies of material and immaterial entities in one's universe of discourse."[
His solution is to introduce an ''order'' into the set of predicates. When the logical system is equipped with this order, it is possible to restrict the ''scope'' of a generalization such as "All ravens are black" so that it applies to ravens only and not to non-black things, since the order privileges ravens over non-black things. As he puts it:
:"If we are justified in assuming that the scope of the generalization 'All ravens are black' can be restricted to ravens, then this means that we have some outside information which we can rely on concerning the factual situation. The paradox arises from the fact that this information, which colors our spontaneous view of the situation, is not incorporated in the usual treatments of the inductive situation."][
]
Rejections of Hempel's equivalence condition
Some approaches for the resolution of the paradox reject Hempel's equivalence condition. That is, they may not consider evidence supporting the statement ''all non-black objects are non-ravens'' to necessarily support logically-equivalent statements such as ''all ravens are black''.
Selective confirmation
Scheffler and Goodman took an approach to the paradox that incorporates Karl Popper
Sir Karl Raimund Popper (28 July 1902 – 17 September 1994) was an Austrian-British philosopher, academic and social commentator. One of the 20th century's most influential philosophers of science, Popper is known for his rejection of the cl ...
's view that scientific hypotheses are never really confirmed, only falsified.
The approach begins by noting that the observation of a black raven does not prove that "All ravens are black" but it falsifies the contrary hypothesis, "No ravens are black". A non-black non-raven, on the other hand, is consistent with both "All ravens are black" and with "No ravens are black". As the authors put it:
:... the statement that all ravens are black is not merely ''satisfied'' by evidence of a black raven but is ''favored'' by such evidence, since a black raven disconfirms the contrary statement that all ravens are not black, i.e. satisfies its denial. A black raven, in other words, satisfies the hypothesis ''that all ravens are black rather than not:'' it thus selectively confirms ''that all ravens are black''.
Selective confirmation violates the equivalence condition since a black raven selectively confirms "All ravens are black" but not "All non-black things are non-ravens".
=Probabilistic or non-probabilistic induction
=
Scheffler and Goodman's concept of selective confirmation is an example of an interpretation of "provides evidence in favor of..." which does not coincide with "increase the probability of..." This must be a general feature of all resolutions that reject the equivalence condition, since logically equivalent propositions must always have the same probability.
It is impossible for the observation of a black raven to increase the probability of the proposition "All ravens are black" without causing exactly the same change to the probability that "All non-black things are non-ravens". If an observation inductively supports the former but not the latter, then "inductively support" must refer to something other than changes in the probabilities of propositions. A possible loophole is to interpret "All" as "Nearly all" – "Nearly all ravens are black" is not equivalent to "Nearly all non-black things are non-ravens", and these propositions can have very different probabilities.
This raises the broader question of the relation of probability theory to inductive reasoning. Karl Popper
Sir Karl Raimund Popper (28 July 1902 – 17 September 1994) was an Austrian-British philosopher, academic and social commentator. One of the 20th century's most influential philosophers of science, Popper is known for his rejection of the cl ...
argued that probability theory alone cannot account for induction. His argument involves splitting a hypothesis, , into a part that is deductively entailed by the evidence, , and another part. This can be done in two ways.
First, consider the splitting:
:
where , and are probabilistically independent: and so on. The condition that is necessary for such a splitting of H and E to be possible is , that is, that is probabilistically supported by .
Popper's observation is that the part, , of that receives support from actually follows deductively from , while the part of that does not follow deductively from receives no support at all from – that is, .
Second, the splitting:
:
separates into , which as Popper says, "is the logically strongest part of (or of the content of ) that follows eductivelyfrom ", and , which, he says, "contains all of that goes beyond ". He continues:
:Does , in this case, provide any support for the factor , which in the presence of is alone needed to obtain ? The answer is: No. It never does. Indeed, countersupports unless either or (which are possibilities of no interest). ...
:This result is completely devastating to the inductive interpretation of the calculus of probability. All probabilistic support is purely deductive: that part of a hypothesis that is not deductively entailed by the evidence is always strongly countersupported by the evidence ... There is such a thing as probabilistic support; there might even be such a thing as inductive support (though we hardly think so). But the calculus of probability reveals that probabilistic support cannot be inductive support.
Orthodox approach
The orthodox Neyman–Pearson theory of hypothesis testing considers how to decide whether to ''accept'' or ''reject'' a hypothesis, rather than what probability to assign to the hypothesis. From this point of view, the hypothesis that "All ravens are black" is not accepted ''gradually'', as its probability increases towards one when more and more observations are made, but is accepted in a single action as the result of evaluating the data that has already been collected. As Neyman and Pearson put it:
:Without hoping to know whether each separate hypothesis is true or false, we may search for rules to govern our behaviour with regard to them, in following which we insure that, in the long run of experience, we shall not be too often wrong.
According to this approach, it is not necessary to assign any value to the probability of a ''hypothesis'', although one must certainly take into account the probability of the ''data'' given the hypothesis, or given a competing hypothesis, when deciding whether to accept or to reject. The acceptance or rejection of a hypothesis carries with it the risk of error
An error (from the Latin ''error'', meaning "wandering") is an action which is inaccurate or incorrect. In some usages, an error is synonymous with a mistake. The etymology derives from the Latin term 'errare', meaning 'to stray'.
In statistics ...
.
This contrasts with the Bayesian approach, which requires that the hypothesis be assigned a prior probability, which is revised in the light of the observed data to obtain the final probability of the hypothesis. Within the Bayesian framework there is no risk of error since hypotheses are not accepted or rejected; instead they are assigned probabilities.
An analysis of the paradox from the orthodox point of view has been performed, and leads to, among other insights, a rejection of the equivalence condition:
:It seems obvious that one cannot both ''accept'' the hypothesis that all P's are Q and also reject the contrapositive, i.e. that all non-Q's are non-P. Yet it is easy to see that on the Neyman-Pearson theory of testing, a test of "All P's are Q" is ''not'' necessarily a test of "All non-Q's are non-P" or vice versa. A test of "All P's are Q" requires reference to some alternative statistical hypothesis of the form of all P's are Q,