![Youngronaldfisher2](https://upload.wikimedia.org/wikipedia/commons/a/aa/Youngronaldfisher2.JPG)
In the
design of experiments
The design of experiments (DOE, DOX, or experimental design) is the design of any task that aims to describe and explain the variation of information under conditions that are hypothesized to reflect the variation. The term is generally associ ...
in
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the lady tasting tea is a
randomized experiment
In science, randomized experiments are the experiments that allow the greatest reliability and validity of statistical estimates of treatment effects. Randomization-based inference is especially important in experimental design and in survey sampl ...
devised by
Ronald Fisher
Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who a ...
and reported in his book ''
The Design of Experiments
''The Design of Experiments'' is a 1935 book by the English statistician Ronald Fisher about the design of experiments and is considered a foundational work in experimental design. Among other contributions, the book introduced the concept of the ...
'' (1935). The experiment is the original exposition of Fisher's notion of a
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
, which is "never proved or established, but is possibly disproved, in the course of experimentation".
[OED quote: 1935 R. A. Fisher, '']The Design of Experiments
''The Design of Experiments'' is a 1935 book by the English statistician Ronald Fisher about the design of experiments and is considered a foundational work in experimental design. Among other contributions, the book introduced the concept of the ...
'' ii. 19, "We may speak of this hypothesis as the 'null hypothesis' ..the null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation."
The lady in question (
Muriel Bristol
Blanche Muriel Bristol (21 April 1888 – 15 March 1950) was a British phycologist who worked at Rothamsted Research (then Rothamsted Experimental Station) in 1919. Her research focused on the mechanisms by which algae acquire nutrients.
Stati ...
) claimed to be able to tell
whether the tea or the milk was added first to a cup. Fisher proposed to give her eight cups, four of each variety, in random order. One could then ask what the probability was for her getting the specific number of cups she identified correct, but just by chance.
Fisher's description is less than 10 pages in length and is notable for its simplicity and completeness regarding terminology, calculations and design of the experiment.
The example is loosely based on an event in Fisher's life. The test used was
Fisher's exact test
Fisher's exact test is a statistical significance test used in the analysis of contingency tables. Although in practice it is employed when sample sizes are small, it is valid for all sample sizes. It is named after its inventor, Ronald Fisher, ...
.
The experiment
The experiment provides a subject with eight randomly ordered cups of tea – four prepared by pouring milk and then tea, four by pouring tea and then milk. The subject attempts to select the four cups prepared by one method or the other, and may compare cups directly against each other as desired. The method employed in the experiment is fully disclosed to the subject.
The
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
is that the subject has no ability to distinguish the teas. In Fisher's approach, there was no
alternative hypothesis
In statistical hypothesis testing, the alternative hypothesis is one of the proposed proposition in the hypothesis test. In general the goal of hypothesis test is to demonstrate that in the given condition, there is sufficient evidence supporting ...
, unlike in the
Neyman–Pearson approach.
The test statistic is a simple count of the number of successful attempts to select the four cups prepared by a given method. The distribution of possible numbers of successes, assuming the null hypothesis is true, can be computed using the number of combinations. Using the
combination
In mathematics, a combination is a selection of items from a set that has distinct members, such that the order of selection does not matter (unlike permutations). For example, given three fruits, say an apple, an orange and a pear, there are th ...
formula, with
total cups and
cups chosen, there are
:
possible combinations.
The frequencies of the possible numbers of successes, given in the final column of this table, are derived as follows. For 0 successes, there is clearly only one set of four choices (namely, choosing all four incorrect cups) giving this result. For one success and three failures, there are four correct cups of which one is selected, which by the combination formula can occur in
different ways (as shown in column 2, with ''x'' denoting a correct cup that is chosen and ''o'' denoting a correct cup that is not chosen); and independently of that, there are four incorrect cups of which three are selected, which can occur in
ways (as shown in the second column, this time with ''x'' interpreted as an incorrect cup which is not chosen, and ''o'' indicating an incorrect cup which is chosen). Thus a selection of any one correct cup and any three incorrect cups can occur in any of 4×4 = 16 ways. The frequencies of the other possible numbers of successes are calculated correspondingly. Thus the number of successes is distributed according to the
hypergeometric distribution
In probability theory and statistics, the hypergeometric distribution is a discrete probability distribution that describes the probability of k successes (random draws for which the object drawn has a specified feature) in n draws, ''without'' ...
. Specifically, for a random variable
equal to the number of successes, we may write
, where
is the population size or total number of cups of tea,
is the number of success states in the population or four cups of either type, and
is the number of draws, or four cups. The distribution of combinations for making ''k'' selections out of the ''2k'' available selections corresponds to the ''k''th row of Pascal's triangle, such that each integer in the row is squared. In this case,
because 4 teacups are selected from the 8 available teacups.
The critical region for rejection of the null of no ability to distinguish was the single case of 4 successes of 4 possible, based on the conventional probability criterion < 5%. This is the critical region because under the null of no ability to distinguish, 4 successes has 1 chance out of 70 (≈ 1.4% < 5%) of occurring, whereas at least 3 of 4 successes has a probability of (16+1)/70 (≈ 24.3% > 5%).
Thus,
if and only if
In logic and related fields such as mathematics and philosophy, "if and only if" (shortened as "iff") is a biconditional logical connective between statements, where either both statements are true or both are false.
The connective is bicondi ...
the lady properly categorized all 8 cups was Fisher willing to reject the null hypothesis – effectively acknowledging the lady's ability at a 1.4% significance level (but without quantifying her ability). Fisher later discussed the benefits of more trials and repeated tests.
David Salsburg
David S. Salsburg (born 1931) is an author. His 2002 book ''The Lady Tasting Tea'', subtitled ''How Statistics Revolutionized Science in the Twentieth Century'', provides a layman's overview of important developments in the field of statistics in t ...
reports that a colleague of Fisher,
H. Fairfield Smith, revealed that in the actual experiment the lady succeeded in identifying all eight cups correctly.
[
The chance of someone who just guesses of getting all correct, assuming she guesses that any four had the tea put in first and the other four the milk, would be only 1 in 70 (the combinations of 8 taken 4 at a time).
]
''The Lady Tasting Tea'' book
David Salsburg
David S. Salsburg (born 1931) is an author. His 2002 book ''The Lady Tasting Tea'', subtitled ''How Statistics Revolutionized Science in the Twentieth Century'', provides a layman's overview of important developments in the field of statistics in t ...
published a popular science
''Popular Science'' (also known as ''PopSci'') is an American digital magazine carrying popular science content, which refers to articles for the general reader on science and technology subjects. ''Popular Science'' has won over 58 awards, incl ...
book entitled ''The Lady Tasting Tea
''The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century'' () is a book by David Salsburg about the history of modern statistics and the role it played in the development of science and industry.
The title comes from ...
'',[Salsburg (2002)] which describes Fisher's experiment and ideas on randomization Randomization is the process of making something random. Randomization is not haphazard; instead, a random process is a sequence of random variables describing a process whose outcomes do not follow a deterministic pattern, but follow an evolution d ...
. Deb Basu wrote that "the famous case of the 'lady tasting tea'" was "one of the two supporting pillars ... of the randomization analysis of experimental data."[Basu (1980a, p. 575; 1980b)]
See also
* Hypergeometric distribution
In probability theory and statistics, the hypergeometric distribution is a discrete probability distribution that describes the probability of k successes (random draws for which the object drawn has a specified feature) in n draws, ''without'' ...
* Permutation test
A permutation test (also called re-randomization test) is an exact statistical hypothesis test making use of the proof by contradiction.
A permutation test involves two or more samples. The null hypothesis is that all samples come from the same di ...
* Random assignment
Random assignment or random placement is an experimental technique for assigning human participants or animal subjects to different groups in an experiment (e.g., a treatment group versus a control group) using randomization, such as by a chan ...
* Randomization test
References
*
*
* Basu, D. (1980b). "The Fisher Randomization Test", reprinted with a new preface in ''Statistical Information and Likelihood : A Collection of Critical Essays by Dr. D. Basu'' ; J. K. Ghosh, editor. Springer 1988.
*
* Salsburg, D. (2002) '' The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century'', W.H. Freeman / Owl Book.
{{refend
Design of experiments
Statistical hypothesis testing
Science experiments