In
statistics, a consistent estimator or asymptotically consistent estimator is an
estimator
In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the ...
—a rule for computing estimates of a parameter ''θ''
0—having the property that as the number of data points used increases indefinitely, the resulting sequence of estimates
converges in probability to ''θ''
0. This means that the distributions of the estimates become more and more concentrated near the true value of the parameter being estimated, so that the probability of the estimator being arbitrarily close to ''θ''
0 converges to one.
In practice one constructs an estimator as a function of an available sample of
size
Size in general is the Magnitude (mathematics), magnitude or dimensions of a thing. More specifically, ''geometrical size'' (or ''spatial size'') can refer to linear dimensions (length, width, height, diameter, perimeter), area, or volume ...
''n'', and then imagines being able to keep collecting data and expanding the sample ''ad infinitum''. In this way one would obtain a sequence of estimates indexed by ''n'', and consistency is a property of what occurs as the sample size “grows to infinity”. If the sequence of estimates can be mathematically shown to converge in probability to the true value ''θ''
0, it is called a consistent estimator; otherwise the estimator is said to be inconsistent.
Consistency as defined here is sometimes referred to as ''weak consistency''. When we replace convergence in probability with
almost sure convergence, then the estimator is said to be ''strongly consistent''. Consistency is related to
bias
Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is closed-minded, prejudicial, or unfair. Biases can be innate or learned. People may develop biases for or against an individual, a group ...
; see
bias versus consistency.
Definition
Formally speaking, an
estimator
In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the ...
''T
n'' of parameter ''θ'' is said to be consistent, if it
converges in probability to the true value of the parameter:
:
i.e. if, for all ''ε'' > 0
:
A more rigorous definition takes into account the fact that ''θ'' is actually unknown, and thus the convergence in probability must take place for every possible value of this parameter. Suppose is a family of distributions (the
parametric model
In statistics, a parametric model or parametric family or finite-dimensional model is a particular class of statistical models. Specifically, a parametric model is a family of probability distributions that has a finite number of parameters.
Def ...
), and is an infinite
sample from the distribution ''p
θ''. Let be a sequence of estimators for some parameter ''g''(''θ''). Usually ''T
n'' will be based on the first ''n'' observations of a sample. Then this sequence is said to be (weakly) consistent if
:
This definition uses ''g''(''θ'') instead of simply ''θ'', because often one is interested in estimating a certain function or a sub-vector of the underlying parameter. In the next example we estimate the location parameter of the model, but not the scale:
Examples
Sample mean of a normal random variable
Suppose one has a sequence of
statistically independent observations from a
normal ''N''(''μ'', ''σ''2) distribution. To estimate ''μ'' based on the first ''n'' observations, one can use the
sample mean
The sample mean (or "empirical mean") and the sample covariance are statistics computed from a sample of data on one or more random variables.
The sample mean is the average value (or mean value) of a sample of numbers taken from a larger po ...
: ''T
n'' = (''X''
1 + ... + ''X
n'')/''n''. This defines a sequence of estimators, indexed by the sample size ''n''.
From the properties of the normal distribution, we know the
sampling distribution of this statistic: ''T''
''n'' is itself normally distributed, with mean ''μ'' and variance ''σ''
2/''n''. Equivalently,
has a standard normal distribution:
:
as ''n'' tends to infinity, for any fixed . Therefore, the sequence ''T
n'' of sample means is consistent for the population mean ''μ'' (recalling that
is the
cumulative distribution of the normal distribution).
Establishing consistency
The notion of asymptotic consistency is very close, almost synonymous to the notion of convergence in probability. As such, any theorem, lemma, or property which establishes convergence in probability may be used to prove the consistency. Many such tools exist:
* In order to demonstrate consistency directly from the definition one can use the inequality
::
the most common choice for function ''h'' being either the absolute value (in which case it is known as
Markov inequality), or the quadratic function (respectively
Chebyshev's inequality).
* Another useful result is the
continuous mapping theorem: if ''T
n'' is consistent for ''θ'' and ''g''(·) is a real-valued function continuous at point ''θ'', then ''g''(''T
n'') will be consistent for ''g''(''θ''):
::
*
Slutsky’s theorem can be used to combine several different estimators, or an estimator with a non-random convergent sequence. If ''T
n'' →
''d''''α'', and ''S
n'' →
''p''''β'', then
::
* If estimator ''T
n'' is given by an explicit formula, then most likely the formula will employ sums of random variables, and then the
law of large numbers
In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials sho ...
can be used: for a sequence of random variables and under suitable conditions,
::