Density estimation
   HOME

TheInfoList



OR:

In statistics, probability density estimation or simply density estimation is the construction of an
estimate Estimation (or estimating) is the process of finding an estimate or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is de ...
, based on observed
data In the pursuit of knowledge, data (; ) is a collection of discrete Value_(semiotics), values that convey information, describing quantity, qualitative property, quality, fact, statistics, other basic units of meaning, or simply sequences of sy ...
, of an unobservable underlying
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) ca ...
. The unobservable density function is thought of as the density according to which a large population is distributed; the data are usually thought of as a random sample from that population. A variety of approaches to density estimation are used, including Parzen windows and a range of
data clustering Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). It is a main task of ...
techniques, including
vector quantization Vector quantization (VQ) is a classical quantization technique from signal processing that allows the modeling of probability density functions by the distribution of prototype vectors. It was originally used for data compression. It works by di ...
. The most basic form of density estimation is a rescaled histogram.


Example

We will consider records of the incidence of
diabetes Diabetes, also known as diabetes mellitus, is a group of metabolic disorders characterized by a high blood sugar level ( hyperglycemia) over a prolonged period of time. Symptoms often include frequent urination, increased thirst and increased ...
. The following is quoted verbatim from the
data set A data set (or dataset) is a collection of data. In the case of tabular data, a data set corresponds to one or more database tables, where every column of a table represents a particular variable, and each row corresponds to a given record of the ...
description: :''A population of women who were at least 21 years old, of
Pima Pima or PIMA may refer to: People * Pima people, the Akimel O'odham, Indigenous peoples in Arizona (U.S.) and Sonora (Mexico) Places * Pima, Arizona, a town in Graham County * Pima County, Arizona * Pima Canyon, in the Santa Catalina Mountains ...
Indian heritage and living near Phoenix, Arizona, was tested for
diabetes mellitus Diabetes, also known as diabetes mellitus, is a group of metabolic disorders characterized by a high blood sugar level ( hyperglycemia) over a prolonged period of time. Symptoms often include frequent urination, increased thirst and increased ...
according to
World Health Organization The World Health Organization (WHO) is a specialized agency of the United Nations responsible for international public health. The WHO Constitution states its main objective as "the attainment by all peoples of the highest possible level of ...
criteria. The data were collected by the US National Institute of Diabetes and Digestive and Kidney Diseases. We used the 532 complete records.'' In this example, we construct three density estimates for "glu" ( plasma
glucose Glucose is a simple sugar with the molecular formula . Glucose is overall the most abundant monosaccharide, a subcategory of carbohydrates. Glucose is mainly made by plants and most algae during photosynthesis from water and carbon dioxide, u ...
concentration), one conditional on the presence of diabetes, the second conditional on the absence of diabetes, and the third not conditional on diabetes. The conditional density estimates are then used to construct the probability of diabetes conditional on "glu". The "glu" data were obtained from the MASS package of the
R programming language R is a programming language for statistical computing and graphics supported by the R Core Team and the R Foundation for Statistical Computing. Created by statisticians Ross Ihaka and Robert Gentleman, R is used among data miners, bioinforma ...
. Within R, ?Pima.tr and ?Pima.te give a fuller account of the data. The
mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set. For a data set, the '' ari ...
of "glu" in the diabetes cases is 143.1 and the standard deviation is 31.26. The mean of "glu" in the non-diabetes cases is 110.0 and the standard deviation is 24.29. From this we see that, in this data set, diabetes cases are associated with greater levels of "glu". This will be made clearer by plots of the estimated density functions. The first figure shows density estimates of ''p''(glu , diabetes=1), ''p''(glu , diabetes=0), and ''p''(glu). The density estimates are kernel density estimates using a Gaussian kernel. That is, a Gaussian density function is placed at each data point, and the sum of the density functions is computed over the range of the data. From the density of "glu" conditional on diabetes, we can obtain the probability of diabetes conditional on "glu" via
Bayes' rule In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule), named after Thomas Bayes, describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For examp ...
. For brevity, "diabetes" is abbreviated "db." in this formula. : p(\mbox=1, \mbox) = \frac The second figure shows the estimated posterior probability ''p''(diabetes=1 , glu). From these data, it appears that an increased level of "glu" is associated with diabetes.


Application and Purpose

A very natural use of density estimates is in the informal investigation of the properties of a given set of data. Density estimates can give valuable indication of such features as skewness and multimodality in the data. In some cases they will yield conclusions that may then be regarded as self-evidently true, while in others all they will do is to point the way to further analysis and/or data collection. An important aspect of statistics is often the presentation of data back to the client in order to provide explanation and illustration of conclusions that may possibly have been obtained by other means. Density estimates are ideal for this purpose, for the simple reason that they are fairly easily comprehensible to non-mathematicians. More examples illustrating the use of density estimates for exploratory and presentational purposes, including the important case of bivariate data. Density estimation is also frequently used in
anomaly detection In data analysis, anomaly detection (also referred to as outlier detection and sometimes as novelty detection) is generally understood to be the identification of rare items, events or observations which deviate significantly from the majority o ...
or
novelty detection Novelty detection is the mechanism by which an intelligent organism is able to identify an incoming sensory pattern as being hitherto unknown. If the pattern is sufficiently salient or associated with a high positive or strong negative utility, i ...
: if an observation lies in a very low-density region, it is likely to be an anomaly or a novelty. * In
hydrology Hydrology () is the scientific study of the movement, distribution, and management of water on Earth and other planets, including the water cycle, water resources, and environmental watershed sustainability. A practitioner of hydrology is call ...
the histogram and estimated density function of rainfall and river discharge data, analysed with a probability distribution, are used to gain insight in their behaviour and frequency of occurrence.An illustration of histograms and probability density functions
/ref> An example is shown in the blue figure.


Kernel density estimation


See also

*
Kernel density estimation In statistics, kernel density estimation (KDE) is the application of kernel smoothing for probability density estimation, i.e., a non-parametric method to estimate the probability density function of a random variable based on '' kernels'' as ...
*
Mean integrated squared error In statistics, the mean integrated squared error (MISE) is used in density estimation. The MISE of an estimate of an unknown probability density is given by :\operatorname\, f_n-f\, _2^2=\operatorname\int (f_n(x)-f(x))^2 \, dx where ''ƒ'' is ...
* Histogram * Multivariate kernel density estimation * Spectral density estimation *
Kernel embedding of distributions In machine learning, the kernel embedding of distributions (also called the kernel mean or mean map) comprises a class of nonparametric methods in which a probability distribution is represented as an element of a reproducing kernel Hilbert space ...
*
Generative model In statistical classification, two main approaches are called the generative approach and the discriminative approach. These compute classifiers by different approaches, differing in the degree of statistical modelling. Terminology is inconsi ...
* Application of Order Statistics: Non-parametric Density Estimation *
Probability distribution fitting Probability distribution fitting or simply distribution fitting is the fitting of a probability distribution to a series of data concerning the repeated measurement of a variable phenomenon. The aim of distribution fitting is to predict the proba ...


References

Sources * *
Trevor Hastie Trevor John Hastie (born 27 June 1953) is an American statistician and computer scientist. He is currently serving as the John A. Overdeck Professor of Mathematical Sciences and Professor of Statistics at Stanford University. Hastie is known for ...
,
Robert Tibshirani Robert Tibshirani (born July 10, 1956) is a professor in the Departments of Statistics and Biomedical Data Science at Stanford University. He was a professor at the University of Toronto from 1985 to 1998. In his work, he develops statistical to ...
, and Jerome Friedman. ''The Elements of Statistical Learning''. New York: Springer, 2001. . ''(See Chapter 6.)'' * Qi Li and Jeffrey S. Racine. ''Nonparametric Econometrics: Theory and Practice''. Princeton University Press, 2007, . ''(See Chapter 1.)'' * D.W. Scott. ''Multivariate Density Estimation. Theory, Practice and Visualization''. New York: Wiley, 1992. * B.W. Silverman. ''Density Estimation''. London: Chapman and Hall, 1986.


External links


CREEM: Centre for Research Into Ecological and Environmental Modelling
Downloads for free density estimation software package
''Distance 4''
(from Research Unit for Wildlife Population Assessment "RUWPA") an
''WiSP''


''(See "Pima Indians Diabetes Database" for the original data set of 732 records, and additional notes.)'' * MATLAB code fo
one dimensional
and
two dimensional
density estimation
libAGF
C++ software for
variable kernel density estimation In statistics, adaptive or "variable-bandwidth" kernel density estimation is a form of kernel density estimation in which the size of the kernels used in the estimate are varied depending upon either the location of the samples or the location of th ...
. {{Statistics, inference * Nonparametric statistics