In
statistics
Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an
unbiased estimator that has lower variance than any other unbiased estimator for all possible values of the parameter.
For practical statistics problems, it is important to determine the MVUE if one exists, since less-than-optimal procedures would naturally be avoided, other things being equal. This has led to substantial development of statistical theory related to the problem of optimal estimation.
While combining the constraint of
unbiasedness with the desirability metric of least
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbe ...
leads to good results in most practical settings—making MVUE a natural starting point for a broad range of analyses—a targeted specification may perform better for a given problem; thus, MVUE is not always the best stopping point.
Definition
Consider estimation of
based on data
i.i.d. from some member of a family of densities
, where
is the parameter space. An unbiased estimator
of
is ''UMVUE'' if
,
:
for any other unbiased estimator
If an unbiased estimator of
exists, then one can prove there is an essentially unique MVUE.
Using the
Rao–Blackwell theorem
In statistics, the Rao–Blackwell theorem, sometimes referred to as the Rao–Blackwell–Kolmogorov theorem, is a result which characterizes the transformation of an arbitrarily crude estimator into an estimator that is optimal by the mean-squ ...
one can also prove that determining the MVUE is simply a matter of finding a
complete
Complete may refer to:
Logic
* Completeness (logic)
* Completeness of a theory, the property of a theory that every formula in the theory's language or its negation is provable
Mathematics
* The completeness of the real numbers, which implies t ...
sufficient statistic for the family
and conditioning ''any'' unbiased estimator on it.
Further, by the
Lehmann–Scheffé theorem, an unbiased estimator that is a function of a complete, sufficient statistic is the UMVUE estimator.
Put formally, suppose
is unbiased for
, and that
is a complete sufficient statistic for the family of densities. Then
:
is the MVUE for
A
Bayesian analog is a
Bayes estimator
In estimation theory and decision theory, a Bayes estimator or a Bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function (i.e., the posterior expected loss). Equivalently, it maximizes the ...
, particularly with
minimum mean square error
In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE).
Estimator selection
An
efficient estimator need not exist, but if it does and if it is unbiased,
it is the MVUE. Since the
mean squared error
In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference between ...
(MSE) of an estimator ''δ'' is
:
the MVUE minimizes MSE ''among unbiased estimators''. In some cases biased estimators have lower MSE because they have a smaller variance than does any unbiased estimator; see
estimator bias.
Example
Consider the data to be a single observation from an
absolutely continuous distribution on
with density
:
and we wish to find the UMVU estimator of
:
First we recognize that the density can be written as
:
Which is an exponential family with
sufficient statistic
In statistics, a statistic is ''sufficient'' with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the para ...
. In fact this is a full rank exponential family, and therefore
is complete sufficient. See
exponential family
In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, including the enabling of the user to calculate ...
for a derivation which shows
:
Therefore,
:
Here we use Lehmann–Scheffé theorem to get the MVUE
Clearly
is unbiased and
is complete sufficient, thus the UMVU estimator is
:
This example illustrates that an unbiased function of the complete sufficient statistic will be UMVU, as
Lehmann–Scheffé theorem states.
Other examples
* For a normal distribution with unknown mean and variance, the
sample mean
The sample mean (or "empirical mean") and the sample covariance are statistics computed from a sample of data on one or more random variables.
The sample mean is the average value (or mean value) of a sample of numbers taken from a larger popu ...
and (unbiased)
sample variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbe ...
are the MVUEs for the population mean and population variance.
*:However, the
sample standard deviation is not unbiased for the population standard deviation – see
unbiased estimation of standard deviation.
*:Further, for other distributions the sample mean and sample variance are not in general MVUEs – for a
uniform distribution
Uniform distribution may refer to:
* Continuous uniform distribution
* Discrete uniform distribution
* Uniform distribution (ecology)
* Equidistributed sequence
See also
*
* Homogeneous distribution
In mathematics, a homogeneous distribution ...
with unknown upper and lower bounds, the
mid-range is the MVUE for the population mean.
* If ''k'' exemplars are chosen (without replacement) from a
discrete uniform distribution
In probability theory and statistics, the discrete uniform distribution is a symmetric probability distribution wherein a finite number of values are equally likely to be observed; every one of ''n'' values has equal probability 1/''n''. Anoth ...
over the set with unknown upper bound ''N'', the MVUE for ''N'' is
::
:where ''m'' is the
sample maximum
In statistics, the sample maximum and sample minimum, also called the largest observation and smallest observation, are the values of the greatest and least elements of a sample. They are basic summary statistics, used in descriptive statistics ...
. This is a scaled and shifted (so unbiased) transform of the sample maximum, which is a sufficient and complete statistic. See
German tank problem for details.
See also
*
Cramér–Rao bound
In estimation theory and statistics, the Cramér–Rao bound (CRB) expresses a lower bound on the variance of unbiased estimators of a deterministic (fixed, though unknown) parameter, the variance of any such estimator is at least as high as the ...
*
Best linear unbiased estimator (BLUE)
*
Bias–variance tradeoff
*
Lehmann–Scheffé theorem
*
U-statistic
Bayesian analogs
*
Bayes estimator
In estimation theory and decision theory, a Bayes estimator or a Bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function (i.e., the posterior expected loss). Equivalently, it maximizes the ...
*
Minimum mean square error
In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE)
References
*
*
{{Statistics, inference, collapsed
Estimator