In
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an
unbiased estimator
In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
that has lower variance than any other unbiased estimator for all possible values of the parameter.
For practical statistics problems, it is important to determine the MVUE if one exists, since less-than-optimal procedures would naturally be avoided, other things being equal. This has led to substantial development of statistical theory related to the problem of optimal estimation.
While combining the constraint of
unbiasedness
In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
with the desirability metric of least
variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers ...
leads to good results in most practical settings—making MVUE a natural starting point for a broad range of analyses—a targeted specification may perform better for a given problem; thus, MVUE is not always the best stopping point.
Definition
Consider estimation of
based on data
i.i.d. from some member of a family of densities
, where
is the parameter space. An unbiased estimator
of
is ''UMVUE'' if
,
:
for any other unbiased estimator
If an unbiased estimator of
exists, then one can prove there is an essentially unique MVUE.
Using the
Rao–Blackwell theorem
In statistics, the Rao–Blackwell theorem, sometimes referred to as the Rao–Blackwell–Kolmogorov theorem, is a result which characterizes the transformation of an arbitrarily crude estimator into an estimator that is optimal by the mean-squa ...
one can also prove that determining the MVUE is simply a matter of finding a
complete
Complete may refer to:
Logic
* Completeness (logic)
* Completeness of a theory, the property of a theory that every formula in the theory's language or its negation is provable
Mathematics
* The completeness of the real numbers, which implies t ...
sufficient
In logic and mathematics, necessity and sufficiency are terms used to describe a conditional or implicational relationship between two statements. For example, in the conditional statement: "If then ", is necessary for , because the truth of ...
statistic for the family
and conditioning ''any'' unbiased estimator on it.
Further, by the
Lehmann–Scheffé theorem
In statistics, the Lehmann–Scheffé theorem is a prominent statement, tying together the ideas of completeness, sufficiency, uniqueness, and best unbiased estimation. The theorem states that any estimator which is unbiased for a given unknown qu ...
, an unbiased estimator that is a function of a complete, sufficient statistic is the UMVUE estimator.
Put formally, suppose
is unbiased for
, and that
is a complete sufficient statistic for the family of densities. Then
:
is the MVUE for
A
Bayesian
Thomas Bayes (/beɪz/; c. 1701 – 1761) was an English statistician, philosopher, and Presbyterian minister.
Bayesian () refers either to a range of concepts and approaches that relate to statistical methods based on Bayes' theorem, or a followe ...
analog is a
Bayes estimator
In estimation theory and decision theory, a Bayes estimator or a Bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function (i.e., the posterior expected loss). Equivalently, it maximizes the po ...
, particularly with
minimum mean square error
In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE).
Estimator selection
An
efficient estimator
In statistics, efficiency is a measure of quality of an estimator, of an experimental design, or of a hypothesis testing procedure. Essentially, a more efficient estimator, needs fewer input data or observations than a less efficient one to achie ...
need not exist, but if it does and if it is unbiased,
it is the MVUE. Since the
mean squared error
In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference between ...
(MSE) of an estimator ''δ'' is
:
the MVUE minimizes MSE ''among unbiased estimators''. In some cases biased estimators have lower MSE because they have a smaller variance than does any unbiased estimator; see
estimator bias
In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In stat ...
.
Example
Consider the data to be a single observation from an
absolutely continuous distribution on
with density
:
and we wish to find the UMVU estimator of
:
First we recognize that the density can be written as
:
Which is an exponential family with
sufficient statistic
In statistics, a statistic is ''sufficient'' with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the pa ...
. In fact this is a full rank exponential family, and therefore
is complete sufficient. See
exponential family
In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, including the enabling of the user to calculate ...
for a derivation which shows
:
Therefore,
:
Here we use Lehmann–Scheffé theorem to get the MVUE
Clearly
is unbiased and
is complete sufficient, thus the UMVU estimator is
:
This example illustrates that an unbiased function of the complete sufficient statistic will be UMVU, as
Lehmann–Scheffé theorem
In statistics, the Lehmann–Scheffé theorem is a prominent statement, tying together the ideas of completeness, sufficiency, uniqueness, and best unbiased estimation. The theorem states that any estimator which is unbiased for a given unknown qu ...
states.
Other examples
* For a normal distribution with unknown mean and variance, the
sample mean
The sample mean (or "empirical mean") and the sample covariance are statistics computed from a Sample (statistics), sample of data on one or more random variables.
The sample mean is the average value (or mean, mean value) of a sample (statistic ...
and (unbiased)
sample variance
In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers ...
are the MVUEs for the population mean and population variance.
*:However, the
sample standard deviation
In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while ...
is not unbiased for the population standard deviation – see
unbiased estimation of standard deviation In statistics and in particular statistical theory, unbiased estimation of a standard deviation is the calculation from a statistical sample of an estimated value of the standard deviation (a measure of statistical dispersion) of a population of va ...
.
*:Further, for other distributions the sample mean and sample variance are not in general MVUEs – for a
uniform distribution with unknown upper and lower bounds, the
mid-range
In statistics, the mid-range or mid-extreme is a measure of central tendency of a sample defined as the arithmetic mean of the maximum and minimum values of the data set:
:M=\frac.
The mid-range is closely related to the range, a measure of ...
is the MVUE for the population mean.
* If ''k'' exemplars are chosen (without replacement) from a
discrete uniform distribution
In probability theory and statistics, the discrete uniform distribution is a symmetric probability distribution wherein a finite number of values are equally likely to be observed; every one of ''n'' values has equal probability 1/''n''. Anothe ...
over the set with unknown upper bound ''N'', the MVUE for ''N'' is
::
:where ''m'' is the
sample maximum
In statistics, the sample maximum and sample minimum, also called the largest observation and smallest observation, are the values of the greatest and least elements of a sample. They are basic summary statistics, used in descriptive statistics ...
. This is a scaled and shifted (so unbiased) transform of the sample maximum, which is a sufficient and complete statistic. See
German tank problem
In the statistical theory of estimation theory, estimation, the German tank problem consists of estimating the maximum of a discrete uniform distribution from sampling without replacement. In simple terms, suppose there exists an unknown number o ...
for details.
See also
*
Cramér–Rao bound
In estimation theory and statistics, the Cramér–Rao bound (CRB) expresses a lower bound on the variance of unbiased estimators of a deterministic (fixed, though unknown) parameter, the variance of any such estimator is at least as high as the in ...
*
Best linear unbiased estimator
Best or The Best may refer to:
People
* Best (surname), people with the surname Best
* Best (footballer, born 1968), retired Portuguese footballer
Companies and organizations
* Best & Co., an 1879–1971 clothing chain
* Best Lock Corporation, ...
(BLUE)
*
Bias–variance tradeoff
In statistics and machine learning, the bias–variance tradeoff is the property of a model that the variance of the parameter estimated across samples can be reduced by increasing the bias in the estimated parameters.
The bias–variance dil ...
*
Lehmann–Scheffé theorem
In statistics, the Lehmann–Scheffé theorem is a prominent statement, tying together the ideas of completeness, sufficiency, uniqueness, and best unbiased estimation. The theorem states that any estimator which is unbiased for a given unknown qu ...
*
U-statistic
In statistical theory, a U-statistic is a class of statistics that is especially important in estimation theory; the letter "U" stands for unbiased. In elementary statistics, U-statistics arise naturally in producing minimum-variance unbiased est ...
Bayesian analogs
*
Bayes estimator
In estimation theory and decision theory, a Bayes estimator or a Bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function (i.e., the posterior expected loss). Equivalently, it maximizes the po ...
*
Minimum mean square error
In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
(MMSE)
References
*
*
{{Statistics, inference, collapsed
Estimator