In
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the ''k''th order statistic of a
statistical sample
In statistics, quality assurance, and survey methodology, sampling is the selection of a subset (a statistical sample) of individuals from within a statistical population to estimate characteristics of the whole population. Statisticians attempt ...
is equal to its ''k''th-smallest value.
Together with
rank statistics, order statistics are among the most fundamental tools in
non-parametric statistics and
inference
Inferences are steps in reasoning, moving from premises to logical consequences; etymologically, the word '' infer'' means to "carry forward". Inference is theoretically traditionally divided into deduction and induction, a distinction that in ...
.
Important special cases of the order statistics are the
minimum and
maximum value of a sample, and (with some qualifications discussed below) the
sample median
Sample or samples may refer to:
Base meaning
* Sample (statistics), a subset of a population – complete data set
* Sample (signal), a digital discrete sample of a continuous analog signal
* Sample (material), a specimen or small quantity of so ...
and other
sample quantiles.
When using
probability theory to analyze order statistics of
random samples from a
continuous distribution, the
cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ev ...
is used to reduce the analysis to the case of order statistics of the
uniform distribution
Uniform distribution may refer to:
* Continuous uniform distribution
* Discrete uniform distribution
* Uniform distribution (ecology)
* Equidistributed sequence In mathematics, a sequence (''s''1, ''s''2, ''s''3, ...) of real numbers is said to be ...
.
Notation and examples
For example, suppose that four numbers are observed or recorded, resulting in a sample of size 4. If the sample values are
:6, 9, 3, 8,
the order statistics would be denoted
:
where the subscript enclosed in parentheses indicates the th order statistic of the sample.
The first order statistic (or smallest order statistic) is always the
minimum of the sample, that is,
:
where, following a common convention, we use upper-case letters to refer to random variables, and lower-case letters (as above) to refer to their actual observed values.
Similarly, for a sample of size , the th order statistic (or largest order statistic) is the
maximum, that is,
:
The
sample range
In statistics, the range of a set of data is the difference between the largest and smallest values,
the result of subtracting the sample maximum and minimum. It is expressed in the same units as the data.
In descriptive statistics, range is t ...
is the difference between the maximum and minimum. It is a function of the order statistics:
:
A similar important statistic in
exploratory data analysis that is simply related to the order statistics is the sample
interquartile range.
The sample median may or may not be an order statistic, since there is a single middle value only when the number of observations is
odd. More precisely, if for some integer , then the sample median is
and so is an order statistic. On the other hand, when is
even, and there are two middle values,
and
, and the sample median is some function of the two (usually the average) and hence not an order statistic. Similar remarks apply to all sample quantiles.
Probabilistic analysis
Given any random variables ''X''
1, ''X''
2..., ''X''
''n'', the order statistics X
(1), X
(2), ..., X
(''n'') are also random variables, defined by sorting the values (
realizations) of ''X''
1, ..., ''X''
''n'' in increasing order.
When the random variables ''X''
1, ''X''
2..., ''X''
''n'' form a
sample
Sample or samples may refer to:
Base meaning
* Sample (statistics), a subset of a population – complete data set
* Sample (signal), a digital discrete sample of a continuous analog signal
* Sample (material), a specimen or small quantity of s ...
they are
independent and identically distributed. This is the case treated below. In general, the random variables ''X''
1, ..., ''X''
''n'' can arise by sampling from more than one population. Then they are
independent, but not necessarily identically distributed, and their
joint probability distribution
Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered ...
is given by the
Bapat–Beg theorem In probability theory, the Bapat–Beg theorem gives the joint probability distribution of order statistics of independent but not necessarily identically distributed random variables in terms of the cumulative distribution functions of the rando ...
.
From now on, we will assume that the random variables under consideration are
continuous and, where convenient, we will also assume that they have a
probability density function (PDF), that is, they are
absolutely continuous. The peculiarities of the analysis of distributions assigning mass to points (in particular,
discrete distributions) are discussed at the end.
Cumulative distribution function of order statistics
For a random sample as above, with cumulative distribution
, the order statistics for that sample have cumulative distributions as follows
(where ''r'' specifies which order statistic):
:
the corresponding probability density function may be derived from this result, and is found to be
:
Moreover, there are two special cases, which have CDFs that are easy to compute.
:
:
Which can be derived by careful consideration of probabilities.
Probability distributions of order statistics
Order statistics sampled from a uniform distribution
In this section we show that the order statistics of the
uniform distribution
Uniform distribution may refer to:
* Continuous uniform distribution
* Discrete uniform distribution
* Uniform distribution (ecology)
* Equidistributed sequence In mathematics, a sequence (''s''1, ''s''2, ''s''3, ...) of real numbers is said to be ...
on the
unit interval have
marginal distributions belonging to the
beta distribution family. We also give a simple method to derive the joint distribution of any number of order statistics, and finally translate these results to arbitrary continuous distributions using the
cdf.
We assume throughout this section that
is a
random sample drawn from a continuous distribution with cdf
. Denoting
we obtain the corresponding random sample
from the standard
uniform distribution
Uniform distribution may refer to:
* Continuous uniform distribution
* Discrete uniform distribution
* Uniform distribution (ecology)
* Equidistributed sequence In mathematics, a sequence (''s''1, ''s''2, ''s''3, ...) of real numbers is said to be ...
. Note that the order statistics also satisfy
.
The probability density function of the order statistic
is equal to
[.]
:
that is, the ''k''th order statistic of the uniform distribution is a
beta-distributed random variable.
:
The proof of these statements is as follows. For
to be between ''u'' and ''u'' + ''du'', it is necessary that exactly ''k'' − 1 elements of the sample are smaller than ''u'', and that at least one is between ''u'' and ''u'' + d''u''. The probability that more than one is in this latter interval is already
, so we have to calculate the probability that exactly ''k'' − 1, 1 and ''n'' − ''k'' observations fall in the intervals
,
and
respectively. This equals (refer to
multinomial distribution for details)
:
and the result follows.
The mean of this distribution is ''k'' / (''n'' + 1).
The joint distribution of the order statistics of the uniform distribution
Similarly, for ''i'' < ''j'', the
joint probability density function of the two order statistics ''U''
(''i'') < ''U''
(''j'') can be shown to be
:
which is (up to terms of higher order than
) the probability that ''i'' − 1, 1, ''j'' − 1 − ''i'', 1 and ''n'' − ''j'' sample elements fall in the intervals
,
,
,
,
respectively.
One reasons in an entirely analogous way to derive the higher-order joint distributions. Perhaps surprisingly, the joint density of the ''n'' order statistics turns out to be ''constant'':
:
One way to understand this is that the unordered sample does have constant density equal to 1, and that there are ''n''! different permutations of the sample corresponding to the same sequence of order statistics. This is related to the fact that 1/''n''! is the volume of the region