HOME

TheInfoList



OR:

In
probability theory Probability theory or probability calculus is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...
and
statistics Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
, the \chi^2-distribution with k
degrees of freedom In many scientific fields, the degrees of freedom of a system is the number of parameters of the system that may vary independently. For example, a point in the plane has two degrees of freedom for translation: its two coordinates; a non-infinite ...
is the distribution of a sum of the squares of k
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in Pennsylvania, United States * Independentes (English: Independents), a Portuguese artist ...
standard normal random variables. The chi-squared distribution \chi^2_k is a special case of the
gamma distribution In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
and the univariate Wishart distribution. Specifically if X \sim \chi^2_k then X \sim \text(\alpha=\frac, \theta=2) (where \alpha is the shape parameter and \theta the scale parameter of the gamma distribution) and X \sim \text_1(1,k) . The scaled chi-squared distribution s^2 \chi^2_k is a reparametrization of the
gamma distribution In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
and the univariate Wishart distribution. Specifically if X \sim s^2 \chi^2_k then X \sim \text(\alpha=\frac, \theta=2 s^2) and X \sim \text_1(s^2,k) . The chi-squared distribution is one of the most widely used
probability distribution In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
s in
inferential statistics Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution.Upton, G., Cook, I. (2008) ''Oxford Dictionary of Statistics'', OUP. . Inferential statistical analysis infers properties of ...
, notably in
hypothesis testing A statistical hypothesis test is a method of statistical inference used to decide whether the data provide sufficient evidence to reject a particular hypothesis. A statistical hypothesis test typically involves a calculation of a test statistic. T ...
and in construction of confidence intervals. This distribution is sometimes called the central chi-squared distribution, a special case of the more general noncentral chi-squared distribution. The chi-squared distribution is used in the common
chi-squared test A chi-squared test (also chi-square or test) is a Statistical hypothesis testing, statistical hypothesis test used in the analysis of contingency tables when the sample sizes are large. In simpler terms, this test is primarily used to examine w ...
s for
goodness of fit The goodness of fit of a statistical model describes how well it fits a set of observations. Measures of goodness of fit typically summarize the discrepancy between observed values and the values expected under the model in question. Such measur ...
of an observed distribution to a theoretical one, the
independence Independence is a condition of a nation, country, or state, in which residents and population, or some portion thereof, exercise self-government, and usually sovereignty, over its territory. The opposite of independence is the status of ...
of two criteria of classification of
qualitative data Qualitative properties are properties that are observed and can generally not be measured with a numerical result, unlike Quantitative property, quantitative properties, which have numerical characteristics. Description Qualitative properties a ...
, and in finding the confidence interval for estimating the population
standard deviation In statistics, the standard deviation is a measure of the amount of variation of the values of a variable about its Expected value, mean. A low standard Deviation (statistics), deviation indicates that the values tend to be close to the mean ( ...
of a normal distribution from a sample standard deviation. Many other statistical tests also use this distribution, such as Friedman's analysis of variance by ranks.


Definitions

If are
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in Pennsylvania, United States * Independentes (English: Independents), a Portuguese artist ...
, standard normal random variables, then the sum of their squares, : X\ = \sum_^k Z_i^2, is distributed according to the chi-squared distribution with degrees of freedom. This is usually denoted as : X\ \sim\ \chi^2(k)\ \ \text\ \ X\ \sim\ \chi^2_k. The chi-squared distribution has one parameter: a positive integer that specifies the number of
degrees of freedom In many scientific fields, the degrees of freedom of a system is the number of parameters of the system that may vary independently. For example, a point in the plane has two degrees of freedom for translation: its two coordinates; a non-infinite ...
(the number of random variables being summed, ''Z''''i'' s).


Introduction

The chi-squared distribution is used primarily in hypothesis testing, and to a lesser extent for confidence intervals for population variance when the underlying distribution is normal. Unlike more widely known distributions such as the
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
and the
exponential distribution In probability theory and statistics, the exponential distribution or negative exponential distribution is the probability distribution of the distance between events in a Poisson point process, i.e., a process in which events occur continuousl ...
, the chi-squared distribution is not as often applied in the direct modeling of natural phenomena. It arises in the following hypothesis tests, among others: *
Chi-squared test A chi-squared test (also chi-square or test) is a Statistical hypothesis testing, statistical hypothesis test used in the analysis of contingency tables when the sample sizes are large. In simpler terms, this test is primarily used to examine w ...
of independence in
contingency tables In statistics, a contingency table (also known as a cross tabulation or crosstab) is a type of table in a matrix format that displays the multivariate frequency distribution of the variables. They are heavily used in survey research, business int ...
*
Chi-squared test A chi-squared test (also chi-square or test) is a Statistical hypothesis testing, statistical hypothesis test used in the analysis of contingency tables when the sample sizes are large. In simpler terms, this test is primarily used to examine w ...
of goodness of fit of observed data to hypothetical distributions *
Likelihood-ratio test In statistics, the likelihood-ratio test is a hypothesis test that involves comparing the goodness of fit of two competing statistical models, typically one found by maximization over the entire parameter space and another found after imposing ...
for nested models * Log-rank test in survival analysis * Cochran–Mantel–Haenszel test for stratified contingency tables * Wald test * Score test It is also a component of the definition of the ''t''-distribution and the ''F''-distribution used in ''t''-tests, analysis of variance, and regression analysis. The primary reason for which the chi-squared distribution is extensively used in hypothesis testing is its relationship to the normal distribution. Many hypothesis tests use a test statistic, such as the ''t''-statistic in a ''t''-test. For these hypothesis tests, as the sample size, , increases, the sampling distribution of the test statistic approaches the normal distribution (
central limit theorem In probability theory, the central limit theorem (CLT) states that, under appropriate conditions, the Probability distribution, distribution of a normalized version of the sample mean converges to a Normal distribution#Standard normal distributi ...
). Because the test statistic (such as ) is asymptotically normally distributed, provided the sample size is sufficiently large, the distribution used for hypothesis testing may be approximated by a normal distribution. Testing hypotheses using a normal distribution is well understood and relatively easy. The simplest chi-squared distribution is the square of a standard normal distribution. So wherever a normal distribution could be used for a hypothesis test, a chi-squared distribution could be used. Suppose that Z is a random variable sampled from the standard normal distribution, where the mean is 0 and the variance is 1: Z \sim N(0,1). Now, consider the random variable X = Z^2. The distribution of the random variable X is an example of a chi-squared distribution: \ X\ \sim\ \chi^2_1. The subscript 1 indicates that this particular chi-squared distribution is constructed from only 1 standard normal distribution. A chi-squared distribution constructed by squaring a single standard normal distribution is said to have 1 degree of freedom. Thus, as the sample size for a hypothesis test increases, the distribution of the test statistic approaches a normal distribution. Just as extreme values of the normal distribution have low probability (and give small p-values), extreme values of the chi-squared distribution have low probability. An additional reason that the chi-squared distribution is widely used is that it turns up as the large sample distribution of generalized likelihood ratio tests (LRT). LRTs have several desirable properties; in particular, simple LRTs commonly provide the highest power to reject the null hypothesis ( Neyman–Pearson lemma) and this leads also to optimality properties of generalised LRTs. However, the normal and chi-squared approximations are only valid asymptotically. For this reason, it is preferable to use the ''t'' distribution rather than the normal approximation or the chi-squared approximation for a small sample size. Similarly, in analyses of contingency tables, the chi-squared approximation will be poor for a small sample size, and it is preferable to use Fisher's exact test. Ramsey shows that the exact
binomial test Binomial test is an exact test of the statistical significance of deviations from a theoretically expected distribution of observations into two categories using sample data. Usage A binomial test is a statistical hypothesis test used to deter ...
is always more powerful than the normal approximation. Lancaster shows the connections among the binomial, normal, and chi-squared distributions, as follows. De Moivre and Laplace established that a binomial distribution could be approximated by a normal distribution. Specifically they showed the asymptotic normality of the random variable : \chi = where m is the observed number of successes in N trials, where the probability of success is p, and q = 1 - p. Squaring both sides of the equation gives : \chi^2 = Using N = Np + N(1 - p), N = m + (N - m), and q = 1 - p, this equation can be rewritten as : \chi^2 = + The expression on the right is of the form that
Karl Pearson Karl Pearson (; born Carl Pearson; 27 March 1857 – 27 April 1936) was an English biostatistician and mathematician. He has been credited with establishing the discipline of mathematical statistics. He founded the world's first university ...
would generalize to the form : \chi^2 = \sum_^n \frac where \chi^2 = Pearson's cumulative test statistic, which asymptotically approaches a \chi^2 distribution; O_i = the number of observations of type i; E_i = N p_i = the expected (theoretical) frequency of type i, asserted by the null hypothesis that the fraction of type i in the population is p_i; and n = the number of cells in the table. In the case of a binomial outcome (flipping a coin), the binomial distribution may be approximated by a normal distribution (for sufficiently large n). Because the square of a standard normal distribution is the chi-squared distribution with one degree of freedom, the probability of a result such as 1 heads in 10 trials can be approximated either by using the normal distribution directly, or the chi-squared distribution for the normalised, squared difference between observed and expected value. However, many problems involve more than the two possible outcomes of a binomial, and instead require 3 or more categories, which leads to the multinomial distribution. Just as de Moivre and Laplace sought for and found the normal approximation to the binomial, Pearson sought for and found a degenerate multivariate normal approximation to the multinomial distribution (the numbers in each category add up to the total sample size, which is considered fixed). Pearson showed that the chi-squared distribution arose from such a multivariate normal approximation to the multinomial distribution, taking careful account of the statistical dependence (negative correlations) between numbers of observations in different categories.


Probability density function

The
probability density function In probability theory, a probability density function (PDF), density function, or density of an absolutely continuous random variable, is a Function (mathematics), function whose value at any given sample (or point) in the sample space (the s ...
(pdf) of the chi-squared distribution is : f(x;\,k) = \begin \dfrac, & x > 0; \\ 0, & \text. \end where \Gamma(k/2) denotes the
gamma function In mathematics, the gamma function (represented by Γ, capital Greek alphabet, Greek letter gamma) is the most common extension of the factorial function to complex numbers. Derived by Daniel Bernoulli, the gamma function \Gamma(z) is defined ...
, which has closed-form values for integer k. For derivations of the pdf in the cases of one, two and k degrees of freedom, see Proofs related to chi-squared distribution.


Cumulative distribution function

Its
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ever ...
is: : F(x;\,k) = \frac = P\left(\frac,\,\frac\right), where \gamma(s,t) is the lower incomplete gamma function and P(s,t) is the regularized gamma function. In a special case of k = 2 this function has the simple form: : F(x;\,2) = 1 - e^ which can be easily derived by integrating f(x;\,2)=\frace^ directly. The integer recurrence of the gamma function makes it easy to compute F(x;\,k) for other small, even k. Tables of the chi-squared cumulative distribution function are widely available and the function is included in many
spreadsheet A spreadsheet is a computer application for computation, organization, analysis and storage of data in tabular form. Spreadsheets were developed as computerized analogs of paper accounting worksheets. The program operates on data entered in c ...
s and all statistical packages. Letting z \equiv x/k, Chernoff bounds on the lower and upper tails of the CDF may be obtained. For the cases when 0 < z < 1 (which include all of the cases when this CDF is less than half): F(z k;\,k) \leq (z e^)^. The tail bound for the cases when z > 1, similarly, is : 1-F(z k;\,k) \leq (z e^)^. For another
approximation An approximation is anything that is intentionally similar but not exactly equal to something else. Etymology and usage The word ''approximation'' is derived from Latin ''approximatus'', from ''proximus'' meaning ''very near'' and the prefix ...
for the CDF modeled after the cube of a Gaussian, see under Noncentral chi-squared distribution.


Properties


Cochran's theorem

The following is a special case of Cochran's theorem. Theorem. If Z_1,...,Z_n are
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in Pennsylvania, United States * Independentes (English: Independents), a Portuguese artist ...
identically distributed (i.i.d.), standard normal random variables, then \sum_^n(Z_t - \bar Z)^2 \sim \chi^2_ where \bar Z = \frac \sum_^n Z_t. Proof. Let Z\sim\mathcal(\bar 0,1\!\!1) be a vector of n independent normally distributed random variables, and \bar Z their average. Then \sum_^n(Z_t-\bar Z)^2 ~=~ \sum_^n Z_t^2 -n\bar Z^2 ~=~ Z^\top \!\!1 -\bar 1\bar 1^\top ~=:~ Z^\top\!M Z where 1\!\!1 is the identity matrix and \bar 1 the all ones vector. M has one eigenvector b_1:= \bar 1 with eigenvalue 0, and n-1 eigenvectors b_2,...,b_n (all orthogonal to b_1) with eigenvalue 1, which can be chosen so that Q:=(b_1,...,b_n) is an orthogonal matrix. Since also X:=Q^\top\!Z\sim\mathcal(\bar 0,Q^\top\!1\!\!1 Q) =\mathcal(\bar 0,1\!\!1), we have \sum_^n(Z_t-\bar Z)^2 ~=~ Z^\top\!M Z ~=~ X^\top\!Q^\top\!M Q X ~=~ X_2^2+...+X_n^2 ~\sim~ \chi^2_, which proves the claim.


Additivity

It follows from the definition of the chi-squared distribution that the sum of independent chi-squared variables is also chi-squared distributed. Specifically, if X_i,i=\overline are independent chi-squared variables with k_i, i=\overline degrees of freedom, respectively, then Y = X_1 + \cdots + X_n is chi-squared distributed with k_1 + \cdots + k_n degrees of freedom.


Sample mean

The sample mean of n i.i.d. chi-squared variables of degree k is distributed according to a gamma distribution with shape \alpha and scale \theta parameters: : \overline X = \frac \sum_^n X_i \sim \operatorname\left(\alpha=n\, k /2, \theta= 2/n \right) \qquad \text X_i \sim \chi^2(k) Asymptotically, given that for a shape parameter \alpha going to infinity, a Gamma distribution converges towards a normal distribution with expectation \mu = \alpha\cdot \theta and variance \sigma^2 = \alpha\, \theta^2 , the sample mean converges towards: \overline X \xrightarrow N(\mu = k, \sigma^2 = 2\, k /n ) Note that we would have obtained the same result invoking instead the
central limit theorem In probability theory, the central limit theorem (CLT) states that, under appropriate conditions, the Probability distribution, distribution of a normalized version of the sample mean converges to a Normal distribution#Standard normal distributi ...
, noting that for each chi-squared variable of degree k the expectation is k , and its variance 2\,k (and hence the variance of the sample mean \overline being \sigma^2 = \frac ).


Entropy

The
differential entropy Differential entropy (also referred to as continuous entropy) is a concept in information theory that began as an attempt by Claude Shannon to extend the idea of (Shannon) entropy (a measure of average surprisal) of a random variable, to continu ...
is given by : h = \int_^\infty f(x;\,k)\ln f(x;\,k) \, dx = \frac k 2 + \ln \left \,\Gamma \left(\frac k 2 \right)\right+ \left(1-\frac k 2 \right)\, \psi\!\left(\frac k 2 \right), where \psi(x) is the
Digamma function In mathematics, the digamma function is defined as the logarithmic derivative of the gamma function: :\psi(z) = \frac\ln\Gamma(z) = \frac. It is the first of the polygamma functions. This function is Monotonic function, strictly increasing a ...
. The chi-squared distribution is the
maximum entropy probability distribution In statistics and information theory, a maximum entropy probability distribution has entropy that is at least as great as that of all other members of a specified class of probability distributions. According to the principle of maximum entropy, ...
for a random variate X for which \operatorname(X)=k and \operatorname(\ln(X))=\psi(k/2)+\ln(2) are fixed. Since the chi-squared is in the family of gamma distributions, this can be derived by substituting appropriate values in the Expectation of the log moment of gamma. For derivation from more basic principles, see the derivation in moment-generating function of the sufficient statistic.


Noncentral moments

The noncentral moments (raw moments) of a chi-squared distribution with k degrees of freedom are given by : \operatorname(X^m) = k (k+2) (k+4) \cdots (k+2m-2) = 2^m \frac.


Cumulants

The
cumulant In probability theory and statistics, the cumulants of a probability distribution are a set of quantities that provide an alternative to the '' moments'' of the distribution. Any two probability distributions whose moments are identical will have ...
s are readily obtained by a
power series In mathematics, a power series (in one variable) is an infinite series of the form \sum_^\infty a_n \left(x - c\right)^n = a_0 + a_1 (x - c) + a_2 (x - c)^2 + \dots where ''a_n'' represents the coefficient of the ''n''th term and ''c'' is a co ...
expansion of the logarithm of the characteristic function: : \kappa_n = 2^(n-1)!\,k with
cumulant generating function In probability theory and statistics, the cumulants of a probability distribution are a set of quantities that provide an alternative to the '' moments'' of the distribution. Any two probability distributions whose moments are identical will have ...
\ln E ^= - \frac k2 \ln(1-2t) .


Concentration

The chi-squared distribution exhibits strong concentration around its mean. The standard Laurent-Massart bounds are: : \operatorname(X - k \ge 2 \sqrt + 2x) \le \exp(-x) : \operatorname(k - X \ge 2 \sqrt) \le \exp(-x) One consequence is that, if Z \sim N(0, 1)^k is a gaussian random vector in \R^k, then as the dimension k grows, the squared length of the vector is concentrated tightly around k with a width k^:Pr(\, Z\, ^2 \in - 2k^, k + 2k^ + 2k^ \geq 1-e^where the exponent \alpha can be chosen as any value in \R. Since the cumulant generating function for \chi^2(k) is K(t) = -\frac k2 \ln(1-2t) , and its convex dual is K^*(q) = \frac 12 (q-k + k\ln\frac kq) , the standard Chernoff bound yields\begin \ln Pr(X \geq (1 + \epsilon) k) &\leq -\frac k2 ( \epsilon - \ln(1+\epsilon)) \\ \ln Pr(X \leq (1 - \epsilon) k) &\leq -\frac k2 ( -\epsilon - \ln(1-\epsilon)) \endwhere 0< \epsilon < 1. By the union bound,Pr(X \in (1\pm \epsilon ) k ) \geq 1 - 2e^ This result is used in proving the Johnson–Lindenstrauss lemma.


Asymptotic properties

By the
central limit theorem In probability theory, the central limit theorem (CLT) states that, under appropriate conditions, the Probability distribution, distribution of a normalized version of the sample mean converges to a Normal distribution#Standard normal distributi ...
, because the chi-squared distribution is the sum of k independent random variables with finite mean and variance, it converges to a normal distribution for large k. For many practical purposes, for k>50 the distribution is sufficiently close to a
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
, so the difference is ignorable. Specifically, if X \sim \chi^2(k), then as k tends to infinity, the distribution of (X-k)/\sqrt tends to a standard normal distribution. However, convergence is slow as the
skewness In probability theory and statistics, skewness is a measure of the asymmetry of the probability distribution of a real-valued random variable about its mean. The skewness value can be positive, zero, negative, or undefined. For a unimodal ...
is \sqrt and the
excess kurtosis In probability theory and statistics, kurtosis (from , ''kyrtos'' or ''kurtos'', meaning "curved, arching") refers to the degree of “tailedness” in the probability distribution of a real-valued random variable. Similar to skewness, kurtosi ...
is 12/k. The sampling distribution of \ln(\chi^2) converges to normality much faster than the sampling distribution of \chi^2, as the logarithmic transform removes much of the asymmetry. Other functions of the chi-squared distribution converge more rapidly to a normal distribution. Some examples are: * If X \sim \chi^2(k) then \sqrt is approximately normally distributed with mean \sqrt and unit variance (1922, by R. A. Fisher, see (18.23), p. 426 of Johnson. * If X \sim \chi^2(k) then \sqrt /math> is approximately normally distributed with mean 1-\frac and variance \frac . This is known as the Wilson–Hilferty transformation, see (18.24), p. 426 of Johnson. ** This normalizing transformation leads directly to the commonly used median approximation k\bigg(1-\frac\bigg)^3\; by back-transforming from the mean, which is also the median, of the normal distribution.


Related distributions

* As k\to\infty, (\chi^2_k-k)/\sqrt ~ \xrightarrow\ N(0,1) \, (
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
) * \chi_k^2 \sim ^2_k(0) ( noncentral chi-squared distribution with non-centrality parameter \lambda = 0 ) * If Y \sim \mathrm(\nu_1, \nu_2) then X = \lim_ \nu_1 Y has the chi-squared distribution \chi^2_ :*As a special case, if Y \sim \mathrm(1, \nu_2)\, then X = \lim_ Y\, has the chi-squared distribution \chi^2_ * \, \boldsymbol_ (0,1) \, ^2 \sim \chi^2_k (The squared norm of ''k'' standard normally distributed variables is a chi-squared distribution with ''k''
degrees of freedom In many scientific fields, the degrees of freedom of a system is the number of parameters of the system that may vary independently. For example, a point in the plane has two degrees of freedom for translation: its two coordinates; a non-infinite ...
) * If X \sim \chi^2_\nu\, and c>0 \,, then cX \sim \Gamma(k=\nu/2, \theta=2c)\,. (
gamma distribution In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
) * If X \sim \chi^2_k then \sqrt \sim \chi_k (
chi distribution In probability theory and statistics, the chi distribution is a continuous probability distribution over the non-negative real line. It is the distribution of the positive square root of a sum of squared independent Gaussian random variables. E ...
) * If X \sim \chi^2_2, then X \sim \operatorname(1/2) is an
exponential distribution In probability theory and statistics, the exponential distribution or negative exponential distribution is the probability distribution of the distance between events in a Poisson point process, i.e., a process in which events occur continuousl ...
. (See
gamma distribution In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
for more.) * If X \sim \chi^2_, then X \sim \operatorname(k, 1/2) is an
Erlang distribution The Erlang distribution is a two-parameter family of continuous probability distributions with Support (mathematics), support x \in [0, \infty). The two parameters are: * a positive integer k, the "shape", and * a positive real number \lambda, ...
. * If X \sim \operatorname(k,\lambda), then 2\lambda X\sim \chi^2_ * If X \sim \operatorname(1)\, (Rayleigh distribution) then X^2 \sim \chi^2_2\, * If X \sim \operatorname(1)\, (Maxwell distribution) then X^2 \sim \chi^2_3\, * If X \sim \chi^2_\nu then \tfrac \sim \operatorname\chi^2_\nu\, ( Inverse-chi-squared distribution) * The chi-squared distribution is a special case of type III
Pearson distribution The Pearson distribution is a family of continuous probability distributions. It was first published by Karl Pearson in 1895 and subsequently extended by him in 1901 and 1916 in a series of articles on biostatistics. History The Pearson syste ...
* If X \sim \chi^2_\, and Y \sim \chi^2_\, are independent then \tfrac \sim \operatorname(\tfrac, \tfrac)\, (
beta distribution In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval
, 1 The comma is a punctuation mark that appears in several variants in different languages. Some typefaces render it as a small line, slightly curved or straight, but inclined from the vertical; others give it the appearance of a miniature fille ...
or (0, 1) in terms of two positive Statistical parameter, parameters, denoted by ''alpha'' (''α'') an ...
) * If X \sim \operatorname(0,1)\, ( uniform distribution) then -2\log(X) \sim \chi^2_2\, * If X_i \sim \operatorname(\mu,\beta)\, then \sum_^n \frac \sim \chi^2_\, * If X_i follows the
generalized normal distribution The generalized normal distribution (GND) or generalized Gaussian distribution (GGD) is either of two families of parametric continuous probability distributions on the real line. Both families add a shape parameter to the normal distribution. ...
(version 1) with parameters \mu,\alpha,\beta then \sum_^n \frac \sim \chi^2_\, * The chi-squared distribution is a transformation of
Pareto distribution The Pareto distribution, named after the Italian civil engineer, economist, and sociologist Vilfredo Pareto, is a power-law probability distribution that is used in description of social, quality control, scientific, geophysical, actuarial scien ...
*
Student's t-distribution In probability theory and statistics, Student's  distribution (or simply the  distribution) t_\nu is a continuous probability distribution that generalizes the Normal distribution#Standard normal distribution, standard normal distribu ...
is a transformation of chi-squared distribution *
Student's t-distribution In probability theory and statistics, Student's  distribution (or simply the  distribution) t_\nu is a continuous probability distribution that generalizes the Normal distribution#Standard normal distribution, standard normal distribu ...
can be obtained from chi-squared distribution and
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
* The noncentral beta distribution can be obtained as a transformation of chi-squared distribution and noncentral chi-squared distribution * The noncentral t-distribution can be obtained from normal distribution and chi-squared distribution A chi-squared variable with k degrees of freedom is defined as the sum of the squares of k independent standard normal random variables. If Y is a k-dimensional Gaussian random vector with mean vector \mu and rank k covariance matrix C, then X = (Y-\mu )^C^(Y-\mu) is chi-squared distributed with k degrees of freedom. The sum of squares of
statistically independent Independence is a fundamental notion in probability theory, as in statistics and the theory of stochastic processes. Two event (probability theory), events are independent, statistically independent, or stochastically independent if, informally s ...
unit-variance Gaussian variables which do ''not'' have mean zero yields a generalization of the chi-squared distribution called the noncentral chi-squared distribution. If Y is a vector of k i.i.d. standard normal random variables and A is a k\times k
symmetric Symmetry () in everyday life refers to a sense of harmonious and beautiful proportion and balance. In mathematics, the term has a more precise definition and is usually used to refer to an object that is invariant under some transformations ...
,
idempotent matrix In linear algebra, an idempotent matrix is a matrix which, when multiplied by itself, yields itself. That is, the matrix A is idempotent if and only if A^2 = A. For this product A^2 to be defined, A must necessarily be a square matrix. Viewed thi ...
with rank k-n, then the
quadratic form In mathematics, a quadratic form is a polynomial with terms all of degree two (" form" is another name for a homogeneous polynomial). For example, 4x^2 + 2xy - 3y^2 is a quadratic form in the variables and . The coefficients usually belong t ...
Y^TAY is chi-square distributed with k-n degrees of freedom. If \Sigma is a p\times p positive-semidefinite covariance matrix with strictly positive diagonal entries, then for X\sim N(0,\Sigma) and w a random p-vector independent of X such that w_1+\cdots+w_p=1 and w_i\geq 0, i=1,\ldots,p, then : \frac \sim \chi_1^2. The chi-squared distribution is also naturally related to other distributions arising from the Gaussian. In particular, * Y is F-distributed, Y \sim F(k_1, k_2) if Y = \frac, where X_1 \sim \chi^2_ and X_2 \sim \chi^2_ are statistically independent. * If X_1 \sim \chi^2_ and X_2 \sim \chi^2_ are statistically independent, then X_1 + X_2\sim \chi^2_. If X_1 and X_2 are not independent, then X_1+X_2 is not chi-square distributed.


Generalizations

The chi-squared distribution is obtained as the sum of the squares of independent, zero-mean, unit-variance Gaussian random variables. Generalizations of this distribution can be obtained by summing the squares of other types of Gaussian random variables. Several such distributions are described below.


Linear combination

If X_1,\ldots,X_n are chi square random variables and a_1,\ldots,a_n\in\mathbb_, then the distribution of X=\sum_^n a_i X_i is a special case of a Generalized Chi-squared Distribution. A closed expression for this distribution is not known. It may be, however, approximated efficiently using the property of characteristic functions of chi-square random variables.


Chi-squared distributions


Noncentral chi-squared distribution

The noncentral chi-squared distribution is obtained from the sum of the squares of independent Gaussian random variables having unit variance and ''nonzero'' means.


Generalized chi-squared distribution

The generalized chi-squared distribution is obtained from the quadratic form where is a zero-mean Gaussian vector having an arbitrary covariance matrix, and is an arbitrary matrix.


Gamma, exponential, and related distributions

The chi-squared distribution X \sim \chi_k^2 is a special case of the
gamma distribution In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
, in that X \sim \Gamma \left(\frac2,\frac2\right) using the rate parameterization of the gamma distribution (or X \sim \Gamma \left(\frac2,2 \right) using the scale parameterization of the gamma distribution) where is an integer. Because the
exponential distribution In probability theory and statistics, the exponential distribution or negative exponential distribution is the probability distribution of the distance between events in a Poisson point process, i.e., a process in which events occur continuousl ...
is also a special case of the gamma distribution, we also have that if X \sim \chi_2^2, then X\sim \operatorname\left(\frac 1 2\right) is an
exponential distribution In probability theory and statistics, the exponential distribution or negative exponential distribution is the probability distribution of the distance between events in a Poisson point process, i.e., a process in which events occur continuousl ...
. The
Erlang distribution The Erlang distribution is a two-parameter family of continuous probability distributions with Support (mathematics), support x \in [0, \infty). The two parameters are: * a positive integer k, the "shape", and * a positive real number \lambda, ...
is also a special case of the gamma distribution and thus we also have that if X \sim\chi_k^2 with even k, then X is Erlang distributed with shape parameter k/2 and scale parameter 1/2.


Occurrence and applications

The chi-squared distribution has numerous applications in inferential
statistics Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
, for instance in
chi-squared test A chi-squared test (also chi-square or test) is a Statistical hypothesis testing, statistical hypothesis test used in the analysis of contingency tables when the sample sizes are large. In simpler terms, this test is primarily used to examine w ...
s and in estimating variances. It enters the problem of estimating the mean of a normally distributed population and the problem of estimating the slope of a linear regression, regression line via its role in
Student's t-distribution In probability theory and statistics, Student's  distribution (or simply the  distribution) t_\nu is a continuous probability distribution that generalizes the Normal distribution#Standard normal distribution, standard normal distribu ...
. It enters all
analysis of variance Analysis of variance (ANOVA) is a family of statistical methods used to compare the Mean, means of two or more groups by analyzing variance. Specifically, ANOVA compares the amount of variation ''between'' the group means to the amount of variati ...
problems via its role in the
F-distribution In probability theory and statistics, the ''F''-distribution or ''F''-ratio, also known as Snedecor's ''F'' distribution or the Fisher–Snedecor distribution (after Ronald Fisher and George W. Snedecor), is a continuous probability distribut ...
, which is the distribution of the ratio of two independent chi-squared
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...
s, each divided by their respective degrees of freedom. Following are some of the most common situations in which the chi-squared distribution arises from a Gaussian-distributed sample. * if X_1, ..., X_n are i.i.d. N(\mu, \sigma^2)
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a Mathematics, mathematical formalization of a quantity or object which depends on randomness, random events. The term 'random variable' in its mathema ...
s, then \sum_^n(X_i - \overline)^2 \sim \sigma^2 \chi^2_ where \overline = \frac \sum_^n X_i. * The box below shows some
statistics Statistics (from German language, German: ', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a s ...
based on X_i \sim N(\mu_i, \sigma^2_i), i= 1, \ldots, k independent random variables that have probability distributions related to the chi-squared distribution: The chi-squared distribution is also often encountered in
magnetic resonance imaging Magnetic resonance imaging (MRI) is a medical imaging technique used in radiology to generate pictures of the anatomy and the physiological processes inside the body. MRI scanners use strong magnetic fields, magnetic field gradients, and ...
.


Computational methods


Table of values vs -values

The p-value is the probability of observing a test statistic ''at least'' as extreme in a chi-squared distribution. Accordingly, since the
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ever ...
(CDF) for the appropriate degrees of freedom ''(df)'' gives the probability of having obtained a value ''less extreme'' than this point, subtracting the CDF value from 1 gives the ''p''-value. A low ''p''-value, below the chosen significance level, indicates
statistical significance In statistical hypothesis testing, a result has statistical significance when a result at least as "extreme" would be very infrequent if the null hypothesis were true. More precisely, a study's defined significance level, denoted by \alpha, is the ...
, i.e., sufficient evidence to reject the null hypothesis. A significance level of 0.05 is often used as the cutoff between significant and non-significant results. The table below gives a number of ''p''-values matching to \chi^2 for the first 10 degrees of freedom. These values can be calculated evaluating the quantile function (also known as "inverse CDF" or "ICDF") of the chi-squared distribution; e. g., the ICDF for and yields as in the table above, noticing that is the ''p''-value from the table.


History

This distribution was first described by the German geodesist and statistician Friedrich Robert Helmert in papers of 1875–6, where he computed the sampling distribution of the sample variance of a normal population. Thus in German this was traditionally known as the ''Helmert'sche'' ("Helmertian") or "Helmert distribution". The distribution was independently rediscovered by the English mathematician
Karl Pearson Karl Pearson (; born Carl Pearson; 27 March 1857 – 27 April 1936) was an English biostatistician and mathematician. He has been credited with establishing the discipline of mathematical statistics. He founded the world's first university ...
in the context of
goodness of fit The goodness of fit of a statistical model describes how well it fits a set of observations. Measures of goodness of fit typically summarize the discrepancy between observed values and the values expected under the model in question. Such measur ...
, for which he developed his
Pearson's chi-squared test Pearson's chi-squared test or Pearson's \chi^2 test is a statistical test applied to sets of categorical data to evaluate how likely it is that any observed difference between the sets arose by chance. It is the most widely used of many chi-squa ...
, published in 1900, with computed table of values published in , collected in . The name "chi-square" ultimately derives from Pearson's shorthand for the exponent in a
multivariate normal distribution In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional ( univariate) normal distribution to higher dimensions. One d ...
with the Greek letter Chi, writing for what would appear in modern notation as (Σ being the
covariance matrix In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of ...
).R. L. Plackett, ''Karl Pearson and the Chi-Squared Test'', International Statistical Review, 1983
61f.
See also Jeff Miller

The idea of a family of "chi-squared distributions", however, is not due to Pearson but arose as a further development due to Fisher in the 1920s.


See also

*
Chi distribution In probability theory and statistics, the chi distribution is a continuous probability distribution over the non-negative real line. It is the distribution of the positive square root of a sum of squared independent Gaussian random variables. E ...
* Scaled inverse chi-squared distribution *
Gamma distribution In probability theory and statistics, the gamma distribution is a versatile two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-squared distribution are special cases of the g ...
* Generalized chi-squared distribution * Noncentral chi-squared distribution *
Pearson's chi-squared test Pearson's chi-squared test or Pearson's \chi^2 test is a statistical test applied to sets of categorical data to evaluate how likely it is that any observed difference between the sets arose by chance. It is the most widely used of many chi-squa ...
* Reduced chi-squared statistic * Wilks's lambda distribution *
Modified half-normal distribution In probability theory and statistics, the modified half-normal distribution (MHN) is a three-parameter family of continuous probability distributions supported on the positive part of the real line. It can be viewed as a generalization of multiple ...
with the pdf on (0, \infty) is given as f(x)= \frac, where \Psi(\alpha,z)=_1\Psi_1\left(\begin\left(\alpha,\frac\right)\\(1,0)\end;z \right) denotes the Fox–Wright Psi function.


References


Sources

* * *


Further reading

* *


External links


Earliest Uses of Some of the Words of Mathematics: entry on Chi squared has a brief history


from Yale University Stats 101 class.
''Mathematica'' demonstration showing the chi-squared sampling distribution of various statistics, e. g. Σ''x''², for a normal population

Simple algorithm for approximating cdf and inverse cdf for the chi-squared distribution with a pocket calculator

Values of the Chi-squared distribution
{{Authority control Normal distribution Infinitely divisible probability distributions