HOME

TheInfoList



OR:

Frequentist probability or frequentism is an interpretation of probability; it defines an event's
probability Probability is a branch of mathematics and statistics concerning events and numerical descriptions of how likely they are to occur. The probability of an event is a number between 0 and 1; the larger the probability, the more likely an e ...
(the ''long-run probability'') as the limit of its relative frequency in infinitely many trials. Probabilities can be found (in principle) by a repeatable objective process, as in repeated sampling from the same
population Population is a set of humans or other organisms in a given region or area. Governments conduct a census to quantify the resident population size within a given jurisdiction. The term is also applied to non-human animals, microorganisms, and pl ...
, and are thus ideally devoid of subjectivity. The continued use of frequentist methods in scientific inference, however, has been called into question. The development of the frequentist account was motivated by the problems and paradoxes of the previously dominant viewpoint, the
classical interpretation The classical definition of probability or classical interpretation of probability is identified with the works of Jacob Bernoulli and Pierre-Simon Laplace: This definition is essentially a consequence of the principle of indifference. If element ...
. In the classical interpretation, probability was defined in terms of the
principle of indifference The principle of indifference (also called principle of insufficient reason) is a rule for assigning epistemic probabilities. The principle of indifference states that in the absence of any relevant evidence, agents should distribute their cre ...
, based on the natural symmetry of a problem, so, for example, the probabilities of dice games arise from the natural symmetric 6-sidedness of the cube. This classical interpretation stumbled at any statistical problem that has no natural symmetry for reasoning.


Definition

In the frequentist interpretation, probabilities are discussed only when dealing with well-defined random experiments. The set of all possible outcomes of a random experiment is called the sample space of the experiment. An event is defined as a particular subset of the sample space to be considered. For any given event, only one of two possibilities may hold: It occurs or it does not. The relative frequency of occurrence of an event, observed in a number of repetitions of the experiment, is a measure of the probability of that event. This is the core conception of probability in the frequentist interpretation. A claim of the frequentist approach is that, as the number of trials increases, the change in the relative frequency will diminish. Hence, one can view a probability as the ''limiting value'' of the corresponding relative frequencies.


Scope

The frequentist interpretation is a philosophical approach to the definition and use of probabilities; it is one of several such approaches. It does not claim to capture all connotations of the concept 'probable' in colloquial speech of natural languages. As an interpretation, it is not in conflict with the mathematical axiomatization of probability theory; rather, it provides guidance for how to apply mathematical probability theory to real-world situations. It offers distinct guidance in the construction and design of practical experiments, especially when contrasted with the Bayesian interpretation. As to whether this guidance is useful, or is apt to mis-interpretation, has been a source of controversy. Particularly when the frequency interpretation of probability is mistakenly assumed to be the only possible basis for
frequentist inference Frequentist inference is a type of statistical inference based in frequentist probability, which treats “probability” in equivalent terms to “frequency” and draws conclusions from sample-data by means of emphasizing the frequency or pr ...
. So, for example, a list of mis-interpretations of the meaning of
p-values In null-hypothesis significance testing, the ''p''-value is the probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is correct. A very small ''p''-value means ...
accompanies the article on -values; controversies are detailed in the article on
statistical hypothesis testing A statistical hypothesis test is a method of statistical inference used to decide whether the data provide sufficient evidence to reject a particular hypothesis. A statistical hypothesis test typically involves a calculation of a test statistic. T ...
. The Jeffreys–Lindley paradox shows how different interpretations, applied to the same data set, can lead to different conclusions about the 'statistical significance' of a result. As Feller notes:


History

The frequentist view may have been foreshadowed by
Aristotle Aristotle (; 384–322 BC) was an Ancient Greek philosophy, Ancient Greek philosopher and polymath. His writings cover a broad range of subjects spanning the natural sciences, philosophy, linguistics, economics, politics, psychology, a ...
, in ''
Rhetoric Rhetoric is the art of persuasion. It is one of the three ancient arts of discourse ( trivium) along with grammar and logic/ dialectic. As an academic discipline within the humanities, rhetoric aims to study the techniques that speakers or w ...
'', when he wrote: Poisson (1837) clearly distinguished between objective and subjective probabilities. Soon thereafter a flurry of nearly simultaneous publications by
Mill Mill may refer to: Science and technology * Factory * Mill (grinding) * Milling (machining) * Millwork * Paper mill * Steel mill, a factory for the manufacture of steel * Sugarcane mill * Textile mill * List of types of mill * Mill, the arithmetic ...
, Ellis (1843) and Ellis (1854), Cournot (1843), and
Fries French fries, or simply fries, also known as chips, and finger chips (Indian English), are ''List of culinary knife cuts#Batonnet, batonnet'' or ''Julienning, julienne''-cut deep frying, deep-fried potatoes of disputed origin. They are prepa ...
introduced the frequentist view. Venn (1866, 1876, 1888) provided a thorough exposition two decades later. These were further supported by the publications of
Boole George Boole ( ; 2 November 1815 – 8 December 1864) was a largely self-taught English mathematician, philosopher and logician, most of whose short career was spent as the first professor of mathematics at Queen's College, Cork in Ireland. ...
and Bertrand. By the end of the 19th century the frequentist interpretation was well established and perhaps dominant in the sciences. The following generation established the tools of classical inferential statistics (significance testing, hypothesis testing and confidence intervals) all based on frequentist probability. Alternatively, Bernoulli understood the concept of frequentist probability and published a critical proof (the weak law of large numbers) posthumously (Bernoulli, 1713). He is also credited with some appreciation for subjective probability (prior to and without
Bayes' theorem Bayes' theorem (alternatively Bayes' law or Bayes' rule, after Thomas Bayes) gives a mathematical rule for inverting Conditional probability, conditional probabilities, allowing one to find the probability of a cause given its effect. For exampl ...
).
Gauss Johann Carl Friedrich Gauss (; ; ; 30 April 177723 February 1855) was a German mathematician, astronomer, Geodesy, geodesist, and physicist, who contributed to many fields in mathematics and science. He was director of the Göttingen Observat ...
and
Laplace Pierre-Simon, Marquis de Laplace (; ; 23 March 1749 – 5 March 1827) was a French polymath, a scholar whose work has been instrumental in the fields of physics, astronomy, mathematics, engineering, statistics, and philosophy. He summariz ...
used frequentist (and other) probability in derivations of the least squares method a century later, a generation before Poisson.
Laplace Pierre-Simon, Marquis de Laplace (; ; 23 March 1749 – 5 March 1827) was a French polymath, a scholar whose work has been instrumental in the fields of physics, astronomy, mathematics, engineering, statistics, and philosophy. He summariz ...
considered the probabilities of testimonies, tables of mortality, judgments of tribunals, etc. which are unlikely candidates for classical probability. In this view, Poisson's contribution was his sharp criticism of the alternative "inverse" (subjective, Bayesian) probability interpretation. Any criticism by
Gauss Johann Carl Friedrich Gauss (; ; ; 30 April 177723 February 1855) was a German mathematician, astronomer, Geodesy, geodesist, and physicist, who contributed to many fields in mathematics and science. He was director of the Göttingen Observat ...
or
Laplace Pierre-Simon, Marquis de Laplace (; ; 23 March 1749 – 5 March 1827) was a French polymath, a scholar whose work has been instrumental in the fields of physics, astronomy, mathematics, engineering, statistics, and philosophy. He summariz ...
was muted and implicit. (However, note that their later derivations of
least squares The method of least squares is a mathematical optimization technique that aims to determine the best fit function by minimizing the sum of the squares of the differences between the observed values and the predicted values of the model. The me ...
did not use inverse probability.) Major contributors to "classical" statistics in the early 20th century included Fisher, Neyman, and Pearson. Fisher contributed to most of statistics and made significance testing the core of experimental science, although he was critical of the frequentist concept of ''"repeated sampling from the same population"''; Neyman formulated confidence intervals and contributed heavily to sampling theory; Neyman and Pearson paired in the creation of hypothesis testing. All valued objectivity, so the best interpretation of probability available to them was frequentist. All were suspicious of "inverse probability" (the available alternative) with prior probabilities chosen by using the principle of indifference. Fisher said, ''"... the theory of inverse probability is founded upon an error, eferring to Bayes' theoremand must be wholly rejected."'' While Neyman was a pure frequentist, Fisher's views of probability were unique: Both Fisher and Neyman had nuanced view of probability.
von Mises The Mises family or von Mises is the name of an Austrian noble family. Members of the family excelled especially in mathematics and economy. Notable members * Ludwig von Mises, an Austrian-American economist of the Austrian School, older bro ...
offered a combination of mathematical and philosophical support for frequentism in the era.


Etymology

According to the ''
Oxford English Dictionary The ''Oxford English Dictionary'' (''OED'') is the principal historical dictionary of the English language, published by Oxford University Press (OUP), a University of Oxford publishing house. The dictionary, which published its first editio ...
'', the term ''frequentist'' was first used by M.G. Kendall in 1949, to contrast with Bayesians, whom he called ''non-frequentists''. Kendall observed :3. ... we may broadly distinguish two main attitudes. One takes probability as 'a degree of rational belief', or some similar idea...the second defines probability in terms of frequencies of occurrence of events, or by relative proportions in 'populations' or 'collectives'; : ... :12. It might be thought that the differences between the frequentists and the non-frequentists (if I may call them such) are largely due to the differences of the domains which they purport to cover. : ... :''I assert that this is not so'' ... The essential distinction between the frequentists and the non-frequentists is, I think, that the former, in an effort to avoid anything savouring of matters of opinion, seek to define probability in terms of the objective properties of a population, real or hypothetical, whereas the latter do not. mphasis in original "The Frequency Theory of Probability" was used a generation earlier as a chapter title in Keynes (1921). The historical sequence: # Probability concepts were introduced and much of the mathematics of probability derived (prior to the 20th century) # classical statistical inference methods were developed # the mathematical foundations of probability were solidified and current terminology was introduced (all in the 20th century). The primary historical sources in probability and statistics did not use the current terminology of ''classical'', ''subjective'' (Bayesian), and ''frequentist'' probability.


Alternative views

Probability theory Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...
is a branch of mathematics. While its roots reach centuries into the past, it reached maturity with the axioms of
Andrey Kolmogorov Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Soviet ...
in 1933. The theory focuses on the valid operations on probability values rather than on the initial assignment of values; the mathematics is largely independent of any interpretation of probability. Applications and interpretations of
probability Probability is a branch of mathematics and statistics concerning events and numerical descriptions of how likely they are to occur. The probability of an event is a number between 0 and 1; the larger the probability, the more likely an e ...
are considered by philosophy, the sciences and statistics. All are interested in the extraction of knowledge from observations—
inductive reasoning Inductive reasoning refers to a variety of method of reasoning, methods of reasoning in which the conclusion of an argument is supported not with deductive certainty, but with some degree of probability. Unlike Deductive reasoning, ''deductive'' ...
. There are a variety of competing interpretations; All have problems. The frequentist interpretation does resolve difficulties with the classical interpretation, such as any problem where the natural symmetry of outcomes is not known. It does not address other issues, such as the
dutch book In decision theory, economics, and probability theory, the Dutch book arguments are a set of results showing that agents must satisfy the axioms of rational choice to avoid a kind of self-contradiction called a Dutch book. A Dutch book, somet ...
. * Classical probability assigns probabilities based on physical idealized symmetry (dice, coins, cards). The classical definition is at risk of circularity: Probabilities are defined by assuming equality of probabilities. In the absence of symmetry the utility of the definition is limited. * Subjective (Bayesian) probability (a family of competing interpretations) considers degrees of belief: All practical "subjective" probability interpretations are so constrained to rationality as to avoid most subjectivity. Real subjectivity is repellent to some definitions of science which strive for results independent of the observer and analyst. Other applications of Bayesianism in science (e.g. logical Bayesianism) embrace the inherent subjectivity of many scientific studies and objects and use Bayesian reasoning to place boundaries and context on the influence of subjectivities on all analysis. The historical roots of this concept extended to such non-numeric applications as legal evidence. *
Propensity probability The propensity theory of probability is a probability interpretation in which the probability is thought of as a physical propensity, disposition, or tendency of a given type of situation to yield an outcome of a certain kind, or to yield a lon ...
views probability as a causative phenomenon rather than a purely descriptive or subjective one.


Footnotes


Citations


References

* * * * * * * * * * {{DEFAULTSORT:Frequency Probability Probability interpretations