In
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, Fisher consistency, named after
Ronald Fisher
Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who a ...
, is a desirable property of an
estimator
In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the ...
asserting that if the estimator were calculated using the entire
population
Population typically refers to the number of people in a single area, whether it be a city or town, region, country, continent, or the world. Governments typically quantify the size of the resident population within their jurisdiction using a ...
rather than a
sample
Sample or samples may refer to:
Base meaning
* Sample (statistics), a subset of a population – complete data set
* Sample (signal), a digital discrete sample of a continuous analog signal
* Sample (material), a specimen or small quantity of s ...
, the true value of the estimated parameter would be obtained.
Definition
Suppose we have a
statistical sample ''X''
1, ..., ''X''
''n'' where each ''X''
''i'' follows a
cumulative distribution
In statistics, the frequency (or absolute frequency) of an event i is the number n_i of times the observation has occurred/recorded in an experiment or study. These frequencies are often depicted graphically or in tabular form.
Types
The cumul ...
''F''
''θ'' which depends on an unknown
parameter
A parameter (), generally, is any characteristic that can help in defining or classifying a particular system (meaning an event, project, object, situation, etc.). That is, a parameter is an element of a system that is useful, or critical, when ...
''θ''. If an estimator of ''θ'' based on the sample can be represented as a
functional
Functional may refer to:
* Movements in architecture:
** Functionalism (architecture)
** Form follows function
* Functional group, combination of atoms within molecules
* Medical conditions without currently visible organic basis:
** Functional sy ...
of the
empirical distribution function
In statistics, an empirical distribution function (commonly also called an empirical Cumulative Distribution Function, eCDF) is the distribution function associated with the empirical measure of a sample. This cumulative distribution function ...
''F̂
n'':
:
the estimator is said to be ''Fisher consistent'' if:
:
As long as the ''X''
''i'' are
exchangeable, an estimator ''T'' defined in terms of the ''X''
''i'' can be converted into an estimator ''T′'' that can be defined in terms of ''F̂
n'' by averaging ''T'' over all permutations of the data. The resulting estimator will have the same expected value as ''T'' and its variance will be no larger than that of ''T''.
If the
strong law of large numbers
In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials shou ...
can be applied, the empirical distribution functions ''F̂
n'' converge pointwise to ''F
θ'', allowing us to express Fisher consistency as a limit — the estimator is ''Fisher consistent'' if
:
Finite population example
Suppose our sample is obtained from a finite population ''Z''
1, ..., ''Z''
m. We can represent our sample of size ''n'' in terms of the proportion of the sample ''n''
i / ''n'' taking on each value in the population. Writing our estimator of θ as ''T''(''n''
1 / ''n'', ..., ''n''
m / ''n''), the population analogue of the estimator is ''T''(''p''
1, ..., ''p''
m), where p
i = ''P''(''X'' = ''Z''
i). Thus we have ''Fisher consistency'' if ''T''(''p''
1, ..., ''p''
m) = θ.
Suppose the parameter of interest is the
expected value
In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a l ...
μ and the estimator is the
sample mean
The sample mean (or "empirical mean") and the sample covariance are statistics computed from a Sample (statistics), sample of data on one or more random variables.
The sample mean is the average value (or mean, mean value) of a sample (statistic ...
, which can be written
:
where ''I'' is the
indicator function
In mathematics, an indicator function or a characteristic function of a subset of a set is a function that maps elements of the subset to one, and all other elements to zero. That is, if is a subset of some set , one has \mathbf_(x)=1 if x\i ...
. The population analogue of this expression is
:
so we have Fisher consistency.
Role in maximum likelihood estimation
Maximising the likelihood function ''L'' gives an estimate that is Fisher consistent for a parameter ''b'' if
:
where ''b''
0 represents the true value of ''b''.
Relationship to asymptotic consistency and unbiasedness
The term ''consistency'' in statistics usually refers to an estimator that is
asymptotically consistent. Fisher consistency and asymptotic consistency are distinct concepts, although both aim to define a desirable property of an estimator. While many estimators are consistent in both senses, neither definition encompasses the other. For example, suppose we take an estimator ''T''
n that is both Fisher consistent and asymptotically consistent, and then form ''T''
n + ''E''
n, where ''E''
n is a deterministic sequence of nonzero numbers converging to zero. This estimator is asymptotically consistent, but not Fisher consistent for any ''n''.
The sample mean is a Fisher consistent and
unbiased
Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is closed-minded, prejudicial, or unfair. Biases can be innate or learned. People may develop biases for or against an individual, a group, ...
estimate of the population mean, but not all Fisher consistent estimates are unbiased. Suppose we observe a sample from a
uniform distribution on (0,θ) and we wish to estimate θ. The sample maximum is Fisher consistent, but downwardly biased. Conversely, the sample variance is an unbiased estimate of the population variance, but is not Fisher consistent.
Role in decision theory
A loss function is Fisher consistent if the population minimizer of the risk leads to the Bayes optimal decision rule.
[http://www.stat.osu.edu/~yklee/881/consistency.pdf ]
References
{{reflist
Estimation theory