Statistical Distribution
   HOME

TheInfoList



OR:

In statistics, an empirical distribution function (commonly also called an empirical Cumulative Distribution Function, eCDF) is the distribution function associated with the empirical measure of a sample. This
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ev ...
is a
step function In mathematics, a function on the real numbers is called a step function if it can be written as a finite linear combination of indicator functions of intervals. Informally speaking, a step function is a piecewise constant function having on ...
that jumps up by at each of the data points. Its value at any specified value of the measured variable is the fraction of observations of the measured variable that are less than or equal to the specified value. The empirical distribution function is an estimate of the cumulative distribution function that generated the points in the sample. It converges with probability 1 to that underlying distribution, according to the Glivenko–Cantelli theorem. A number of results exist to quantify the rate of convergence of the empirical distribution function to the underlying cumulative distribution function.


Definition

Let be independent, identically distributed real random variables with the common
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ev ...
. Then the empirical distribution function is defined as :\widehat F_n(t) = \frac = \frac \sum_^n \mathbf_, where \mathbf_ is the indicator of event . For a fixed , the indicator \mathbf_ is a
Bernoulli random variable In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabil ...
with parameter ; hence n \widehat F_n(t) is a binomial random variable with
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set. For a data set, the '' ari ...
and
variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of number ...
. This implies that \widehat F_n(t) is an unbiased estimator for . However, in some textbooks, the definition is given as \widehat F_n(t) = \frac \sum_^n \mathbf_Madsen, H.O., Krenk, S., Lind, S.C. (2006) ''Methods of Structural Safety''. Dover Publications. p. 148-149.


Mean

The
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value ( magnitude and sign) of a given data set. For a data set, the '' ari ...
of the empirical distribution is an
unbiased estimator In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called ''unbiased''. In st ...
of the mean of the population distribution. E_n(X) = \frac\left (\sum_^n\right ) which is more commonly denoted \bar


Variance

The
variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of number ...
of the empirical distribution times \tfrac is an unbiased estimator of the variance of the population distribution, for any distribution of X that has a finite variance. \begin \operatorname(X) &= \operatorname\left X - \operatorname[X^2\right">.html" ;"title="X - \operatorname[X">X - \operatorname[X^2\right\\ pt&= \operatorname\left[(X - \bar)^2\right] \\ pt&= \frac\left (\sum_^n\right ) \end


Mean squared error

The
mean squared error In statistics, the mean squared error (MSE) or mean squared deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors—that is, the average squared difference betwe ...
for the empirical distribution is as follows. \begin \operatorname&=\frac\sum_^n(Y_i-\hat)^2\\ pt&=\operatorname_(\hat)+ \operatorname(\hat,\theta)^2 \end Where \hat is an estimator and \theta an unknown parameter


Quantiles

For any real number a the notation \lceil\rceil (read “ceiling of a”) denotes the least integer greater than or equal to a. For any real number a, the notation \lfloor\rfloor (read “floor of a”) denotes the greatest integer less than or equal to a. If nq is not an integer, then the q-th quantile is unique and is equal to x_ If nq is an integer, then the q-th quantile is not unique and is any real number x such that x_


Empirical median

If n is odd, then the empirical median is the number \tilde = x_ If n is even, then the empirical median is the number \tilde =\frac


Asymptotic properties

Since the ratio approaches 1 as goes to infinity, the asymptotic properties of the two definitions that are given above are the same. By the strong law of large numbers, the estimator \scriptstyle\widehat_n(t) converges to as
almost surely In probability theory, an event is said to happen almost surely (sometimes abbreviated as a.s.) if it happens with probability 1 (or Lebesgue measure 1). In other words, the set of possible exceptions may be non-empty, but it has probability 0 ...
, for every value of : : \widehat F_n(t)\ \xrightarrow\ F(t); thus the estimator \scriptstyle\widehat_n(t) is
consistent In classical deductive logic, a consistent theory is one that does not lead to a logical contradiction. The lack of contradiction can be defined in either semantic or syntactic terms. The semantic definition states that a theory is consisten ...
. This expression asserts the pointwise convergence of the empirical distribution function to the true cumulative distribution function. There is a stronger result, called the Glivenko–Cantelli theorem, which states that the convergence in fact happens uniformly over : : \, \widehat F_n-F\, _\infty \equiv \sup_ \big, \widehat F_n(t)-F(t)\big, \ \xrightarrow\ 0. The sup-norm in this expression is called the Kolmogorov–Smirnov statistic for testing the goodness-of-fit between the empirical distribution \scriptstyle\widehat_n(t) and the assumed true cumulative distribution function . Other norm functions may be reasonably used here instead of the sup-norm. For example, the L2-norm gives rise to the Cramér–von Mises statistic. The asymptotic distribution can be further characterized in several different ways. First, the
central limit theorem In probability theory, the central limit theorem (CLT) establishes that, in many situations, when independent random variables are summed up, their properly normalized sum tends toward a normal distribution even if the original variables thems ...
states that ''pointwise'', \scriptstyle\widehat_n(t) has asymptotically normal distribution with the standard \sqrt rate of convergence: : \sqrt\big(\widehat F_n(t) - F(t)\big)\ \ \xrightarrow\ \ \mathcal\Big( 0, F(t)\big(1-F(t)\big) \Big). This result is extended by the
Donsker’s theorem In probability theory, Donsker's theorem (also known as Donsker's invariance principle, or the functional central limit theorem), named after Monroe D. Donsker, is a functional extension of the central limit theorem. Let X_1, X_2, X_3, \ldots be ...
, which asserts that the '' empirical process'' \scriptstyle\sqrt(\widehat_n - F), viewed as a function indexed by \scriptstyle t\in\mathbb, converges in distribution in the Skorokhod space \scriptstyle D \infty, +\infty/math> to the mean-zero
Gaussian process In probability theory and statistics, a Gaussian process is a stochastic process (a collection of random variables indexed by time or space), such that every finite collection of those random variables has a multivariate normal distribution, i.e. ...
\scriptstyle G_F = B \circ F, where is the standard
Brownian bridge A Brownian bridge is a continuous-time stochastic process ''B''(''t'') whose probability distribution is the conditional probability distribution of a standard Wiener process ''W''(''t'') (a mathematical model of Brownian motion) subject to the co ...
. The covariance structure of this Gaussian process is : \operatorname ,G_F(t_1)G_F(t_2)\,= F(t_1\wedge t_2) - F(t_1)F(t_2). The uniform rate of convergence in Donsker’s theorem can be quantified by the result known as the Hungarian embedding: : \limsup_ \frac \big\, \sqrt(\widehat F_n-F) - G_\big\, _\infty < \infty, \quad \text Alternatively, the rate of convergence of \scriptstyle\sqrt(\widehat_n-F) can also be quantified in terms of the asymptotic behavior of the sup-norm of this expression. Number of results exist in this venue, for example the
Dvoretzky–Kiefer–Wolfowitz inequality In the theory of probability Probability is the branch of mathematics concerning numerical descriptions of how likely an Event (probability theory), event is to occur, or how likely it is that a proposition is true. The probability of an ...
provides bound on the tail probabilities of \scriptstyle\sqrt\, \widehat_n-F\, _\infty: : \Pr\!\Big( \sqrt\, \widehat_n-F\, _\infty > z \Big) \leq 2e^. In fact, Kolmogorov has shown that if the cumulative distribution function is continuous, then the expression \scriptstyle\sqrt\, \widehat_n-F\, _\infty converges in distribution to \scriptstyle\, B\, _\infty, which has the Kolmogorov distribution that does not depend on the form of . Another result, which follows from the
law of the iterated logarithm In probability theory, the law of the iterated logarithm describes the magnitude of the fluctuations of a random walk. The original statement of the law of the iterated logarithm is due to A. Ya. Khinchin (1924). Another statement was given by A ...
, is that : \limsup_ \frac \leq \frac12, \quad \text and : \liminf_ \sqrt \, \widehat_n-F\, _\infty = \frac, \quad \text


Confidence intervals

As per
Dvoretzky–Kiefer–Wolfowitz inequality In the theory of probability Probability is the branch of mathematics concerning numerical descriptions of how likely an Event (probability theory), event is to occur, or how likely it is that a proposition is true. The probability of an ...
the interval that contains the true CDF, F(x), with probability 1-\alpha is specified as : F_n(x) - \varepsilon \le F(x) \le F_n(x) + \varepsilon \; \text \varepsilon = \sqrt. As per the above bounds, we can plot the Empirical CDF, CDF and Confidence intervals for different distributions by using any one of the Statistical implementations. Following is the syntax fro
Statsmodel
for plotting empirical distribution.


Statistical implementation

A non-exhaustive list of software implementations of Empirical Distribution function includes: * I
R software
we compute an empirical cumulative distribution function, with several methods for plotting, printing and computing with such an “ecdf” object. * I

we can use Empirical cumulative distribution function (cdf) plot
jmp from SAS
the CDF plot creates a plot of the empirical cumulative distribution function.
Minitab
create an Empirical CDF

we can fit probability distribution to our data

we can plot Empirical CDF plot

using scipy.stats we can plot the distribution

we can use statsmodels.distributions.empirical_distribution.ECDF

we can use histograms to plot a cumulative distribution

using the seaborn.ecdfplot function
Plotly
using the plotly.express.ecdf function
Excel
we can plot Empirical CDF plot


See also

* Càdlàg functions * Count data * Distribution fitting *
Dvoretzky–Kiefer–Wolfowitz inequality In the theory of probability Probability is the branch of mathematics concerning numerical descriptions of how likely an Event (probability theory), event is to occur, or how likely it is that a proposition is true. The probability of an ...
* Empirical probability * Empirical process * Estimating quantiles from a sample * Frequency (statistics) * Kaplan–Meier estimator for censored processes * Survival function *
Q–Q plot In statistics, a Q–Q plot (quantile-quantile plot) is a probability plot, a graphical method for comparing two probability distributions by plotting their '' quantiles'' against each other. A point on the plot corresponds to one of the q ...


References


Further reading

*


External links

* {{DEFAULTSORT:Empirical Distribution Function Nonparametric statistics Empirical process