An ''F''-test is any
statistical test
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Hypothesis testing allows us to make probabilistic statements about population parameters.
...
in which the
test statistic
A test statistic is a statistic (a quantity derived from the sample) used in statistical hypothesis testing.Berger, R. L.; Casella, G. (2001). ''Statistical Inference'', Duxbury Press, Second Edition (p.374) A hypothesis test is typically specifi ...
has an
''F''-distribution under the
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
. It is most often used when
comparing statistical models that have been fitted to a
data
In the pursuit of knowledge, data (; ) is a collection of discrete Value_(semiotics), values that convey information, describing quantity, qualitative property, quality, fact, statistics, other basic units of meaning, or simply sequences of sy ...
set, in order to identify the model that best fits the
population
Population typically refers to the number of people in a single area, whether it be a city or town, region, country, continent, or the world. Governments typically quantify the size of the resident population within their jurisdiction using a ...
from which the data were sampled. Exact "''F''-tests" mainly arise when the models have been fitted to the data using
least squares. The name was coined by
George W. Snedecor
George Waddel Snedecor (October 20, 1881 – February 15, 1974) was an American mathematician and statistician. He contributed to the foundations of analysis of variance, data analysis, experimental design, and statistical methodology. Snedecor ...
, in honour of
Ronald Fisher
Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who ...
. Fisher initially developed the statistic as the variance ratio in the 1920s.
Common examples
Common examples of the use of ''F''-tests include the study of the following cases:
* The hypothesis that the
means
Means may refer to:
* Means LLC, an anti-capitalist media worker cooperative
* Means (band), a Christian hardcore band from Regina, Saskatchewan
* Means, Kentucky, a town in the US
* Means (surname)
* Means Johnston Jr. (1916–1989), US Navy adm ...
of a given set of
normally distributed populations, all having the same
standard deviation, are equal. This is perhaps the best-known ''F''-test, and plays an important role in the
analysis of variance
Analysis of variance (ANOVA) is a collection of statistical models and their associated estimation procedures (such as the "variation" among and between groups) used to analyze the differences among means. ANOVA was developed by the statistician ...
(ANOVA).
* The hypothesis that a proposed regression model fits the
data
In the pursuit of knowledge, data (; ) is a collection of discrete Value_(semiotics), values that convey information, describing quantity, qualitative property, quality, fact, statistics, other basic units of meaning, or simply sequences of sy ...
well. See
Lack-of-fit sum of squares In statistics, a sum of squares due to lack of fit, or more tersely a lack-of-fit sum of squares, is one of the components of a partition of the sum of squares of residuals in an analysis of variance, used in the numerator in an F-test of the nul ...
.
* The hypothesis that a data set in a
regression analysis
In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one ...
follows the simpler of two proposed linear models that are
nested
''Nested'' is the seventh studio album by Bronx-born singer, songwriter and pianist Laura Nyro, released in 1978 on Columbia Records.
Following on from her extensive tour to promote 1976's ''Smile'', which resulted in the 1977 live album '' Seas ...
within each other.
In addition, some statistical procedures, such as
Scheffé's method In statistics, Scheffé's method, named after the American statistician Henry Scheffé, is a method for adjusting significance levels in a linear regression analysis to account for multiple comparisons. It is particularly useful in analysis of ...
for multiple comparisons adjustment in linear models, also use ''F''-tests.
''F''-test of the equality of two variances
The ''F''-test is
sensitive to
non-normality. In the
analysis of variance
Analysis of variance (ANOVA) is a collection of statistical models and their associated estimation procedures (such as the "variation" among and between groups) used to analyze the differences among means. ANOVA was developed by the statistician ...
(ANOVA), alternative tests include
Levene's test In statistics, Levene's test is an inferential statistic used to assess the equality of variances for a variable calculated for two or more groups. Some common statistical procedures assume that variances of the populations from which different sam ...
,
Bartlett's test
In statistics, Bartlett's test, named after Maurice Stevenson Bartlett, is used to test homoscedasticity, that is, if multiple samples are from populations with equal variances. Some statistical tests, such as the analysis of variance, assume tha ...
, and the
Brown–Forsythe test
The Brown–Forsythe test is a statistical test for the equality of group variances based on performing an Analysis of Variance (ANOVA) on a transformation of the response variable. When a one-way ANOVA is performed, samples are assumed to have ...
. However, when any of these tests are conducted to test the underlying assumption of
homoscedasticity
In statistics, a sequence (or a vector) of random variables is homoscedastic () if all its random variables have the same finite variance. This is also known as homogeneity of variance. The complementary notion is called heteroscedasticity. Th ...
(''i.e.'' homogeneity of variance), as a preliminary step to testing for mean effects, there is an increase in the experiment-wise
Type I error
In statistical hypothesis testing, a type I error is the mistaken rejection of an actually true null hypothesis (also known as a "false positive" finding or conclusion; example: "an innocent person is convicted"), while a type II error is the fa ...
rate.
Formula and calculation
Most ''F''-tests arise by considering a decomposition of the
variability in a collection of data in terms of
sums of squares. The
test statistic
A test statistic is a statistic (a quantity derived from the sample) used in statistical hypothesis testing.Berger, R. L.; Casella, G. (2001). ''Statistical Inference'', Duxbury Press, Second Edition (p.374) A hypothesis test is typically specifi ...
in an ''F''-test is the ratio of two scaled sums of squares reflecting different sources of variability. These sums of squares are constructed so that the statistic tends to be greater when the null hypothesis is not true. In order for the statistic to follow the
''F''-distribution under the null hypothesis, the sums of squares should be
statistically independent
Independence is a fundamental notion in probability theory, as in statistics and the theory of stochastic processes. Two events are independent, statistically independent, or stochastically independent if, informally speaking, the occurrence of o ...
, and each should follow a scaled
χ²-distribution. The latter condition is guaranteed if the data values are independent and
normally distributed with a common
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbe ...
.
Multiple-comparison ANOVA problems
The ''F''-test in one-way analysis of variance (
ANOVA
Analysis of variance (ANOVA) is a collection of statistical models and their associated estimation procedures (such as the "variation" among and between groups) used to analyze the differences among means. ANOVA was developed by the statistician ...
) is used to assess whether the
expected values of a quantitative variable within several pre-defined groups differ from each other. For example, suppose that a medical trial compares four treatments. The ANOVA ''F''-test can be used to assess whether any of the treatments is on average superior, or inferior, to the others versus the null hypothesis that all four treatments yield the same mean response. This is an example of an "omnibus" test, meaning that a single test is performed to detect any of several possible differences. Alternatively, we could carry out pairwise tests among the treatments (for instance, in the medical trial example with four treatments we could carry out six tests among pairs of treatments). The advantage of the ANOVA ''F''-test is that we do not need to pre-specify which treatments are to be compared, and we do not need to adjust for making
multiple comparisons
In statistics, the multiple comparisons, multiplicity or multiple testing problem occurs when one considers a set of statistical inferences simultaneously or infers a subset of parameters selected based on the observed values.
The more inferences ...
. The disadvantage of the ANOVA ''F''-test is that if we reject the
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
, we do not know which treatments can be said to be significantly different from the others, nor, if the ''F''-test is performed at level α, can we state that the treatment pair with the greatest mean difference is significantly different at level α.
The formula for the one-way ANOVA ''F''-test
statistic is
:
or
:
The "explained variance", or "between-group variability" is
:
where
denotes the
sample mean
The sample mean (or "empirical mean") and the sample covariance are statistics computed from a sample of data on one or more random variables.
The sample mean is the average value (or mean value) of a sample of numbers taken from a larger popu ...
in the ''i''-th group,
is the number of observations in the ''i''-th group,
denotes the overall mean of the data, and
denotes the number of groups.
The "unexplained variance", or "within-group variability" is
:
where
is the ''j''
th observation in the ''i''
th out of
groups and
is the overall sample size. This ''F''-statistic follows the
''F''-distribution with degrees of freedom
and
under the null hypothesis. The statistic will be large if the between-group variability is large relative to the within-group variability, which is unlikely to happen if the
population means of the groups all have the same value.
Note that when there are only two groups for the one-way ANOVA ''F''-test,
where ''t'' is the
Student's statistic.
Regression problems
Consider two models, 1 and 2, where model 1 is 'nested' within model 2. Model 1 is the restricted model, and model 2 is the unrestricted one. That is, model 1 has ''p''
1 parameters, and model 2 has ''p''
2 parameters, where ''p''
1 < ''p''
2, and for any choice of parameters in model 1, the same regression curve can be achieved by some choice of the parameters of model 2.
One common context in this regard is that of deciding whether a model fits the data significantly better than does a naive model, in which the only explanatory term is the intercept term, so that all predicted values for the dependent variable are set equal to that variable's sample mean. The naive model is the restricted model, since the coefficients of all potential explanatory variables are restricted to equal zero.
Another common context is deciding whether there is a structural break in the data: here the restricted model uses all data in one regression, while the unrestricted model uses separate regressions for two different subsets of the data. This use of the F-test is known as the
Chow test
The Chow test (), proposed by econometrician Gregory Chow in 1960, is a test of whether the true coefficients in two linear regressions on different data sets are equal. In econometrics, it is most commonly used in time series analysis to test fo ...
.
The model with more parameters will always be able to fit the data at least as well as the model with fewer parameters. Thus typically model 2 will give a better (i.e. lower error) fit to the data than model 1. But one often wants to determine whether model 2 gives a ''significantly'' better fit to the data. One approach to this problem is to use an ''F''-test.
If there are ''n'' data points to estimate parameters of both models from, then one can calculate the ''F'' statistic, given by
:
where RSS
''i'' is the
residual sum of squares of model ''i''. If the regression model has been calculated with weights, then replace RSS
''i'' with χ
2, the weighted sum of squared residuals. Under the null hypothesis that model 2 does not provide a significantly better fit than model 1, ''F'' will have an ''F'' distribution, with (''p''
2−''p''
1, ''n''−''p''
2)
degrees of freedom. The null hypothesis is rejected if the ''F'' calculated from the data is greater than the critical value of the
''F''-distribution for some desired false-rejection probability (e.g. 0.05). Since ''F'' is a monotone function of the likelihood ratio statistic, the ''F''-test is a
likelihood ratio test
In statistics, the likelihood-ratio test assesses the goodness of fit of two competing statistical models based on the ratio of their likelihoods, specifically one found by maximization over the entire parameter space and another found after ...
.
See also
*
Goodness of fit
The goodness of fit of a statistical model describes how well it fits a set of observations. Measures of goodness of fit typically summarize the discrepancy between observed values and the values expected under the model in question. Such measure ...
References
Further reading
*
*
*
*
External links
Table of ''F''-test critical values* by
Mark Thoma
Mark Allen Thoma (born December 15, 1956) is a macroeconomist and econometrician and a professor of economics at the Department of Economics of the University of Oregon. Thoma is best known as a regular columnist for ''The Fiscal Times'' throug ...
{{Statistics, inference
Analysis of variance
Statistical ratios
Statistical tests