In
probability theory, the
central limit theorem states that, under certain circumstances, the
probability distribution
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
of the scaled
mean of a random sample converges to a
normal distribution
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu ...
as the sample size increases to infinity. Under stronger assumptions, the Berry–Esseen theorem, or Berry–Esseen inequality, gives a more quantitative result, because it also specifies the rate at which this convergence takes place by giving a bound on the maximal error of
approximation
An approximation is anything that is intentionally similar but not exactly equality (mathematics), equal to something else.
Etymology and usage
The word ''approximation'' is derived from Latin ''approximatus'', from ''proximus'' meaning ''very ...
between the normal distribution and the true distribution of the scaled sample mean. The approximation is measured by the
Kolmogorov–Smirnov distance. In the case of
independent samples, the convergence rate is , where is the sample size, and the constant is estimated in terms of the
third
Third or 3rd may refer to:
Numbers
* 3rd, the ordinal form of the cardinal number 3
* , a fraction of one third
* Second#Sexagesimal divisions of calendar time and day, 1⁄60 of a ''second'', or 1⁄3600 of a ''minute''
Places
* 3rd Street (d ...
absolute
normalized moment.
Statement of the theorem
Statements of the theorem vary, as it was independently discovered by two
mathematicians,
Andrew C. Berry (in 1941) and
Carl-Gustav Esseen (1942), who then, along with other authors, refined it repeatedly over subsequent decades.
Identically distributed summands
One version, sacrificing generality somewhat for the sake of clarity, is the following:
:There exists a positive
constant ''C'' such that if ''X''
1, ''X''
2, ..., are
i.i.d. random variables
In probability theory and statistics, a collection of random variables is independent and identically distributed if each random variable has the same probability distribution as the others and all are mutually independent. This property is us ...
with
E(''X''
1) = 0, E(''X''
12) = σ
2 > 0, and E(, ''X''
1,
3) = ρ < ∞,
[Since the random variables are identically distributed, ''X''2, ''X''3, ... all have the same moments as ''X''1.] and if we define
::
:the
sample mean, with ''F''
''n'' the
cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ev ...
of
::
:and Φ the cumulative distribution function of the
standard normal distribution, then for all ''x'' and ''n'',
::
That is: given a sequence of
independent and identically distributed random variables, each having
mean zero and positive
variance, if additionally the third absolute
moment
Moment or Moments may refer to:
* Present time
Music
* The Moments, American R&B vocal group Albums
* ''Moment'' (Dark Tranquillity album), 2020
* ''Moment'' (Speed album), 1998
* ''Moments'' (Darude album)
* ''Moments'' (Christine Guldbrand ...
is finite, then the
cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ev ...
s of the
standardized sample mean and the standard normal distribution differ (vertically, on a graph) by no more than the specified amount. Note that the approximation error for all ''n'' (and hence the limiting rate of convergence for indefinite ''n'' sufficiently large) is bounded by the
order
Order, ORDER or Orders may refer to:
* Categorization, the process in which ideas and objects are recognized, differentiated, and understood
* Heterarchy, a system of organization wherein the elements have the potential to be ranked a number of d ...
of ''n''
−1/2.
Calculated values of the constant ''C'' have decreased markedly over the years, from the original value of 7.59 by , to 0.7882 by , then 0.7655 by , then 0.7056 by , then 0.7005 by , then 0.5894 by , then 0.5129 by , then 0.4785 by . The detailed review can be found in the papers and . The best estimate , ''C'' < 0.4748, follows from the inequality
:
due to , since σ
3 ≤ ρ and 0.33554 · 1.415 < 0.4748. However, if ρ ≥ 1.286σ
3, then the estimate
:
which is also proved in , gives an even tighter upper estimate.
proved that the constant also satisfies the lower bound
:
Non-identically distributed summands
:Let ''X''
1, ''X''
2, ..., be independent random variables with
E(''X''
''i'') = 0, E(''X''
''i''2) = σ
''i''2 > 0, and E(, ''X''
''i'',
3) = ρ
''i'' < ∞. Also, let
::
:be the normalized ''n''-th partial sum. Denote ''F''
''n'' the
cdf of ''S''
''n'', and Φ the cdf of the
standard normal distribution. For the sake of convenience denote
::
:In 1941,
Andrew C. Berry proved that for all ''n'' there exists an absolute constant ''C''
1 such that
::
:where
::
:Independently, in 1942,
Carl-Gustav Esseen proved that for all ''n'' there exists an absolute constant ''C''
0 such that
::
:where
::
It is easy to make sure that ψ
0≤ψ
1. Due to this circumstance inequality (3) is conventionally called the Berry–Esseen inequality, and the quantity ψ
0 is called the Lyapunov fraction of the third order. Moreover, in the case where the summands ''X''
1, ..., ''X''
''n'' have identical distributions
::
and thus the bounds stated by inequalities (1), (2) and (3) coincide apart from the constant.
Regarding ''C''
0, obviously, the lower bound established by remains valid:
:
The upper bounds for ''C''
0 were subsequently lowered from the original estimate 7.59 due to to (considering recent results only) 0.9051 due to , 0.7975 due to , 0.7915 due to , 0.6379 and 0.5606 due to and . the best estimate is 0.5600 obtained by .
Multidimensional version
As with the
multidimensional central limit theorem, there is a multidimensional version of the Berry–Esseen theorem.
[Bentkus, Vidmantas. "A Lyapunov-type bound in Rd." Theory of Probability & Its Applications 49.2 (2005): 311–323.]
Let
be independent
-valued random vectors each having mean zero. Write
and assume