Frequentist Inference
   HOME

TheInfoList



OR:

Frequentist inference is a type of
statistical inference Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution, distribution of probability.Upton, G., Cook, I. (2008) ''Oxford Dictionary of Statistics'', OUP. . Inferential statistical ...
based in
frequentist probability Frequentist probability or frequentism is an interpretation of probability; it defines an event's probability as the limit of its relative frequency in many trials (the long-run probability). Probabilities can be found (in principle) by a repe ...
, which treats “probability” in equivalent terms to “frequency” and draws conclusions from sample-data by means of emphasizing the frequency or proportion of findings in the data. Frequentist-inference underlies frequentist statistics, in which the well-established methodologies of
statistical hypothesis testing A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis. Hypothesis testing allows us to make probabilistic statements about population parameters. ...
and
confidence interval In frequentist statistics, a confidence interval (CI) is a range of estimates for an unknown parameter. A confidence interval is computed at a designated ''confidence level''; the 95% confidence level is most common, but other levels, such as 9 ...
s are founded.


History of frequentist statistics

The history of frequentist statistics is more recent than its prevailing philosophical rival,
Bayesian statistics Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability where probability expresses a ''degree of belief'' in an event. The degree of belief may be based on prior knowledge about the event, ...
. Frequentist statistics were largely developed in the early 20th century and have recently developed to become the dominant paradigm in inferential statistics, while Bayesian statistics were invented in the 19th century. Despite this dominance, there is no agreement as to whether frequentism is better than Bayesian statistics, with a vocal minority of professionals studying statistical inference decrying frequentist inference for being internally-inconsistent. For the purposes of this article, frequentist methodology will be discussed as summarily as possible but it is worth noting that this subject remains controversial even into the modern day. The primary formulation of frequentism stems from the presumption that statistics could be perceived to have been a probabilistic frequency. This view was primarily developed by
Ronald Fisher Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who ...
and the team of
Jerzy Neyman Jerzy Neyman (April 16, 1894 – August 5, 1981; born Jerzy Spława-Neyman; ) was a Polish mathematician and statistician who spent the first part of his professional career at various institutions in Warsaw, Poland and then at University Colleg ...
and Egon Pearson. Ronald Fisher contributed to frequentist statistics by developing the frequentist concept of "significance testing", which is the study of the significance of a measure of a statistic when compared to the hypothesis. Neyman-Pearson extended Fisher's ideas to multiple hypotheses by conjecturing that the ratio of probabilities of hypotheses when maximizing the difference between the two hypotheses leads to a maximization of exceeding a given p-value, and also provides the basis of type I and type II errors. For more, see the
foundations of statistics The foundations of statistics concern the epistemological debate in statistics over how one should conduct inductive inference from data. Among the issues considered in statistical inference are the question of Bayesian inference versus frequentist ...
page.


Definition

For statistical inference, the relevant statistic about which we want to make inferences is y \in Y, where the random vector Y is a function of an unknown parameter, \theta. The parameter \theta is further partitioned into (\psi, \lambda), where \psi is the parameter of interest, and \lambda is the nuisance parameter. For concreteness, \psi in one area might be the population mean, \mu, and the nuisance parameter \lambda would then be the standard deviation of the population mean, \sigma. Thus, statistical inference is concerned with the expectation of random vector Y, namely E(Y)=E(Y;\theta)=\int y f_Y (y;\theta)dy . To construct areas of uncertainty in frequentist inference, a
pivot Pivot may refer to: *Pivot, the point of rotation in a lever system *More generally, the center point of any rotational system *Pivot joint, a kind of joint between bones in the body *Pivot turn, a dance move Companies *Incitec Pivot, an Austra ...
is used which defines the area around \psi that can be used to provide an interval to estimate uncertainty. The pivot is a probability such that for a pivot, p, which is a function, that p(t,\psi) is strictly increasing in \psi, where t \in T is a random vector. This allows that, for some 0 < c < 1, we can define P\, which is the probability that the pivot function is less than some well-defined value. This implies P\ = 1-c, where q(t,c) is a 1-c upper limit for \psi. Note that 1-c is a range of outcomes that define a one-sided limit for \psi, and that 1-2c is a two-sided limit for \psi, when we want to estimate a range of outcomes where \psi may occur. This rigorously defines the confidence interval, which is the range of outcomes about which we can make statistical inferences.


Fisherian reduction and Neyman-Pearson operational criteria

Two complementary concepts in frequentist inference are the Fisherian reduction and the Neyman-Pearson operational criteria. Together these concepts illustrate a way of constructing frequentist intervals that define the limits for \psi. The Fisherian reduction is a method of determining the interval within which the true value of \psi may lie, while the Neyman-Pearson operational criteria is a decision rule about making ''a priori'' probability assumptions. The Fisherian reduction is defined as follows: * Determine the likelihood function (this is usually just gathering the data); * Reduce to a
sufficient statistic In statistics, a statistic is ''sufficient'' with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the pa ...
S of the same dimension as \theta; * Find the function of S that has a distribution depending only on \psi; * Invert that distribution (this yields a cumulative distribution function or CDF) to obtain limits for \psi at an arbitrary set of probability levels ; * Use the conditional distribution of the data given S = s informally or formally as to assess the adequacy of the formulation. Essentially, the Fisherian reduction is design to find where the sufficient statistic can be used to determine the range of outcomes where \psi may occur on a probability distribution that defines all the potential values of \psi. This is necessary to formulating confidence intervals, where we can find a range of outcomes over which \psi is likely to occur in the long-run. The Neyman-Pearon operational criteria is an even more specific understanding of the range of outcomes where the relevant statistic, \psi, can be said to occur in the long run. The Neyman-Pearson operational criteria defines the likelihood of that range actually being adequate or of the range being inadequate. The Neyman-Pearson criteria defines the range of the probability distribution that, if \psi exists in this range, is still below the true population statistic. For example, if the distribution from the Fisherian reduction exceeds a threshold that we consider to be ''a priori'' implausible, then the Neyman-Pearson reduction's evaluation of that distribution can be used to infer where looking purely at the Fisherian reduction's distributions can give us inaccurate results. Thus, the Neyman-Pearson reduction is used to find the probability of type I and type II errors. As a point of reference, the complement to this in Bayesian statistics is the minimum Bayes risk criterion. Because of the reliance of the Neyman-Pearson criteria on our ability to find a range of outcomes where \psi is likely to occur, the Neyman-Pearson approach is only possible where a Fisherian reduction can be achieved.


Experimental design and methodology

Frequentist inferences are associated with the application
frequentist probability Frequentist probability or frequentism is an interpretation of probability; it defines an event's probability as the limit of its relative frequency in many trials (the long-run probability). Probabilities can be found (in principle) by a repe ...
to
experimental design The design of experiments (DOE, DOX, or experimental design) is the design of any task that aims to describe and explain the variation of information under conditions that are hypothesized to reflect the variation. The term is generally associ ...
and interpretation, and specifically with the view that any given experiment can be considered one of an infinite sequence of possible repetitions of the same experiment, each capable of producing
statistically independent Independence is a fundamental notion in probability theory, as in statistics and the theory of stochastic processes. Two events are independent, statistically independent, or stochastically independent if, informally speaking, the occurrence of o ...
results. In this view, the frequentist inference approach to drawing conclusions from data is effectively to require that the correct conclusion should be drawn with a given (high) probability, among this notional set of repetitions. However, exactly the same procedures can be developed under a subtly different formulation. This is one where a pre-experiment point of view is taken. It can be argued that the design of an experiment should include, before undertaking the experiment, decisions about exactly what steps will be taken to reach a conclusion from the data yet to be obtained. These steps can be specified by the scientist so that there is a high probability of reaching a correct decision where, in this case, the probability relates to a yet to occur set of random events and hence does not rely on the frequency interpretation of probability. This formulation has been discussed by Neyman, among others. This is especially pertinent because the significance of a frequentist test can vary under model selection, a violation of the likelihood principle.


The statistical philosophy of frequentism

Frequentism is the study of probability with the assumption that results occur with a given frequency over some period of time or with repeated sampling. As such, frequentist analysis must be formulated with consideration to the assumptions of the problem frequentism attempts to analyze. This requires looking into whether the question at hand is concerned with understanding variety of a statistic or locating the true value of a statistic. ''The difference between these assumptions is critical for interpreting a hypothesis test''. The next paragraph elaborates on this. There are broadly two camps of statistical inference, the ''epistemic approach'' and the ''epidemiological approach''. The epistemic approach is the study of ''variability''; namely, how often do we expect a statistic to deviate from some observed value. The epidemiological approach is concerned with the study of ''uncertainty''; in this approach, the value of the statistic is fixed but our understanding of that statistic is incomplete. For concreteness, imagine trying to measure the stock market quote versus evaluating an asset's price. The stock market fluctuates so greatly that trying to find exactly where a stock price is going to be is not useful: the stock market is better understood using the epistemic approach, where we can try to quantify its fickle movements. Conversely, the price of an asset might not change that much from day to day: it is better to locate the true value of the asset rather than find a range of prices and thus the epidemiological approach is better. The difference between these approaches is non-trivial for the purposes of inference. For the epistemic approach, we formulate the problem as if we want to attribute probability to a hypothesis. This can only be done with Bayesian statistics, where the interpretation of probability is straightforward because Bayesian statistics is conditional on the entire sample space, whereas frequentist testing is concerned with the whole experimental design. Frequentist statistics is conditioned not on solely the data but also on the ''experimental design''. In frequentist statistics, the cutoff for understanding the frequency occurrence is derived from the family distribution used in the experiment design. For example, a binomial distribution and a negative binomial distribution can be used to analyze exactly the same data, but because their tail ends are different the frequentist analysis will realize different levels of statistical significance for the same data that assumes different probability distributions. This difference does not occur in Bayesian inference. For more, see the
likelihood principle In statistics, the likelihood principle is the proposition that, given a statistical model, all the evidence in a sample relevant to model parameters is contained in the likelihood function. A likelihood function arises from a probability density f ...
, which frequentist statistics inherently violates. For the epidemiological approach, the central idea behind frequentist statistics must be discussed. Frequentist statistics is designed so that, in the ''long-run'', the frequency of a statistic may be understood, and in the ''long-run'' the range of the true mean of a statistic can be inferred. This leads to the Fisherian reduction and the Neyman-Pearson operational criteria, discussed above. When we define the Fisherian reduction and the Neyman-Pearson operational criteria for any statistic, we are assessing, according to these authors, the likelihood that the true value of the statistic will occur within a given range of outcomes assuming a number of repetitions of our sampling method. This allows for inference where, in the long-run, we can define that the combined results of multiple frequentist inferences to mean that a 95% confidence interval literally means the true mean lies in the confidence interval 95% of the time, but ''not'' that the mean is in a particular confidence interval with 95% certainty. This is a popular misconception. Very commonly the epistemic view and the epidemiological view are regarded as interconvertible. This is demonstrably false. First, the epistemic view is centered around Fisherian significance tests that are designed to provide inductive evidence against the null hypothesis, H_0, in a single experiment, and is defined by the Fisherian p-value. Conversely, the epidemiological view, conducted with Neyman-Pearson hypothesis testing, is designed to minimize the Type II false acceptance errors in the long-run by providing error minimizations that work in the long-run. The difference between the two is critical because the epistemic view stresses the conditions under which we might find one value to be statistically significant; meanwhile, the epidemiological view defines the conditions under which long-run results present valid results. These are extremely different inferences, because one-time, epistemic conclusions do not inform long-run errors, and long-run errors cannot be used to certify whether one-time experiments are sensical. The assumption of one-time experiments to long-run occurrences is a misattribution, and the assumption of long run trends to individuals experiments is an example of the ecological fallacy.


Relationship with other approaches

Frequentist inferences stand in contrast to other types of statistical inferences, such as
Bayesian inference Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian inference is an important technique in statistics, a ...
s and
fiducial inference Fiducial inference is one of a number of different types of statistical inference. These are rules, intended for general application, by which conclusions can be drawn from samples of data. In modern statistical practice, attempts to work with ...
s. While the "
Bayesian inference Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian inference is an important technique in statistics, a ...
" is sometimes held to include the approach to inferences leading to
optimal decision An optimal decision is a decision that leads to at least as good a known or expected outcome as all other available decision options. It is an important concept in decision theory. In order to compare the different decision outcomes, one commonly ...
s, a more restricted view is taken here for simplicity.


Bayesian inference

Bayesian inference is based in
Bayesian probability Bayesian probability is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation representing a state of knowledge or as quantification ...
, which treats “probability” as equivalent with “certainty”, and thus that the essential difference between the frequentist inference and the Bayesian inference is the same as the difference between the two interpretations of what a “probability” means. However, where appropriate, Bayesian inferences (meaning in this case an application of Bayes' theorem) are used by those employing
frequency probability Frequentist probability or frequentism is an interpretation of probability; it defines an event's probability as the limit of a sequence, limit of its relative Frequency_(statistics), frequency in many trials (the long-run probability). Probabi ...
. There are two major differences in the frequentist and Bayesian approaches to inference that are not included in the above consideration of the interpretation of probability: # In a frequentist approach to inference, unknown
parameter A parameter (), generally, is any characteristic that can help in defining or classifying a particular system (meaning an event, project, object, situation, etc.). That is, a parameter is an element of a system that is useful, or critical, when ...
s are often, but not always, treated as having fixed but unknown values that are not capable of being treated as
random variate In probability and statistics, a random variate or simply variate is a particular outcome of a ''random variable'': the random variates which are other outcomes of the same random variable might have different values ( random numbers). A random ...
s in any sense, and hence there is no way that probabilities can be associated with them. In contrast, a Bayesian approach to inference does allow probabilities to be associated with unknown parameters, where these probabilities can sometimes have a
frequency probability Frequentist probability or frequentism is an interpretation of probability; it defines an event's probability as the limit of a sequence, limit of its relative Frequency_(statistics), frequency in many trials (the long-run probability). Probabi ...
interpretation as well as a Bayesian one. The Bayesian approach allows these probabilities to have an interpretation as representing the scientist's belief that given values of the parameter are true (see Bayesian probability - Personal probabilities and objective methods for constructing priors). # While "probabilities" are involved in both approaches to inference, the probabilities are associated with different types of things. The result of a Bayesian approach can be a probability distribution for what is known about the parameters given the results of the experiment or study. The result of a frequentist approach is either a "true or false" conclusion from a
significance test A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis. Hypothesis testing allows us to make probabilistic statements about population parameters. ...
or a conclusion in the form that a given sample-derived
confidence interval In frequentist statistics, a confidence interval (CI) is a range of estimates for an unknown parameter. A confidence interval is computed at a designated ''confidence level''; the 95% confidence level is most common, but other levels, such as 9 ...
covers the true value: either of these conclusions has a given probability of being correct, where this probability has either a
frequency probability Frequentist probability or frequentism is an interpretation of probability; it defines an event's probability as the limit of a sequence, limit of its relative Frequency_(statistics), frequency in many trials (the long-run probability). Probabi ...
interpretation or a pre-experiment interpretation.


See also

*
Intuitive statistics Intuitive statistics, or folk statistics, refers to the cognitive phenomenon where organisms use data to make generalizations and predictions about the world. This can be a small amount of sample data or training instances, which in turn contribute ...
*
German tank problem In the statistical theory of estimation, the German tank problem consists of estimating the maximum of a discrete uniform distribution from sampling without replacement. In simple terms, suppose there exists an unknown number of items which are s ...


References


Bibliography

* * * * {{statistics, inference, collapsed Statistical inference