In
probability
Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speakin ...
and
statistics, a mixture distribution is the
probability distribution of a
random variable that is derived from a collection of other random variables as follows: first, a random variable is selected by chance from the collection according to given probabilities of selection, and then the value of the selected random variable is realized. The underlying random variables may be random real numbers, or they may be
random vector
In probability, and statistics, a multivariate random variable or random vector is a list of mathematical variables each of whose value is unknown, either because the value has not yet occurred or because there is imperfect knowledge of its value ...
s (each having the same dimension), in which case the mixture distribution is a
multivariate distribution
Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered ...
.
In cases where each of the underlying random variables is
continuous
Continuity or continuous may refer to:
Mathematics
* Continuity (mathematics), the opposing concept to discreteness; common examples include
** Continuous probability distribution or random variable in probability and statistics
** Continuous ...
, the outcome variable will also be continuous and its
probability density function
In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) ca ...
is sometimes referred to as a mixture density. The
cumulative distribution function (and the
probability density function
In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) ca ...
if it exists) can be expressed as a
convex combination
In convex geometry and vector algebra, a convex combination is a linear combination of points (which can be vectors, scalars, or more generally points in an affine space) where all coefficients are non-negative and sum to 1. In other w ...
(i.e. a weighted sum, with non-negative weights that sum to 1) of other distribution functions and density functions. The individual distributions that are combined to form the mixture distribution are called the mixture components, and the probabilities (or weights) associated with each component are called the mixture weights. The number of components in a mixture distribution is often restricted to being finite, although in some cases the components may be
countably infinite in number. More general cases (i.e. an
uncountable
In mathematics, an uncountable set (or uncountably infinite set) is an infinite set that contains too many elements to be countable. The uncountability of a set is closely related to its cardinal number: a set is uncountable if its cardinal num ...
set of component distributions), as well as the countable case, are treated under the title of
compound distributions.
A distinction needs to be made between a
random variable whose distribution function or density is the sum of a set of components (i.e. a mixture distribution) and a random variable whose value is the sum of the values of two or more underlying random variables, in which case the distribution is given by the
convolution
In mathematics (in particular, functional analysis), convolution is a mathematical operation on two functions ( and ) that produces a third function (f*g) that expresses how the shape of one is modified by the other. The term ''convolution'' ...
operator. As an example, the sum of two
jointly normally distributed random variables, each with different means, will still have a normal distribution. On the other hand, a mixture density created as a mixture of two normal distributions with different means will have two peaks provided that the two means are far enough apart, showing that this distribution is radically different from a normal distribution.
Mixture distributions arise in many contexts in the literature and arise naturally where a
statistical population
In statistics, a population is a set of similar items or events which is of interest for some question or experiment. A statistical population can be a group of existing objects (e.g. the set of all stars within the Milky Way galaxy) or a hypoth ...
contains two or more
subpopulation
In statistics, a population is a Set (mathematics), set of similar items or events which is of interest for some question or experiment. A statistical population can be a group of existing objects (e.g. the set of all stars within the Milky Way g ...
s. They are also sometimes used as a means of representing non-normal distributions. Data analysis concerning
statistical models involving mixture distributions is discussed under the title of
mixture model
In statistics, a mixture model is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed data set should identify the sub-population to which an individual observatio ...
s, while the present article concentrates on simple probabilistic and statistical properties of mixture distributions and how these relate to properties of the underlying distributions.
Finite and countable mixtures
Given a finite set of probability density functions ''p''
1(''x''), ..., ''p
n''(''x''), or corresponding cumulative distribution functions ''P''
1(''x''), ..., ''P
n''(''x'') and weights ''w''
1, ..., ''w
n'' such that and the mixture distribution can be represented by writing either the density, ''f'', or the distribution function, ''F'', as a sum (which in both cases is a convex combination):
:
:
This type of mixture, being a finite sum, is called a finite mixture, and in applications, an unqualified reference to a "mixture density" usually means a finite mixture. The case of a countably infinite set of components is covered formally by allowing
.
Uncountable mixtures
Where the set of component distributions is
uncountable
In mathematics, an uncountable set (or uncountably infinite set) is an infinite set that contains too many elements to be countable. The uncountability of a set is closely related to its cardinal number: a set is uncountable if its cardinal num ...
, the result is often called a
compound probability distribution In probability and statistics, a compound probability distribution (also known as a mixture distribution or contagious distribution) is the probability distribution that results from assuming that a random variable is distributed according to som ...
. The construction of such distributions has a formal similarity to that of mixture distributions, with either infinite summations or integrals replacing the finite summations used for finite mixtures.
Consider a probability density function ''p''(''x'';''a'') for a variable ''x'', parameterized by ''a''. That is, for each value of ''a'' in some set ''A'', ''p''(''x'';''a'') is a probability density function with respect to ''x''. Given a probability density function ''w'' (meaning that ''w'' is nonnegative and integrates to 1), the function
:
is again a probability density function for ''x''. A similar integral can be written for the cumulative distribution function. Note that the formulae here reduce to the case of a finite or infinite mixture if the density ''w'' is allowed to be a
generalized function
In mathematics, generalized functions are objects extending the notion of functions. There is more than one recognized theory, for example the theory of distributions. Generalized functions are especially useful in making discontinuous functions ...
representing the "derivative" of the cumulative distribution function of a
discrete distribution
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon ...
.
Mixtures within a parametric family
The mixture components are often not arbitrary probability distributions, but instead are members of a
parametric family
In mathematics and its applications, a parametric family or a parameterized family is a family of objects (a set of related objects) whose differences depend only on the chosen values for a set of parameters.
Common examples are parametrized (fam ...
(such as normal distributions), with different values for a parameter or parameters. In such cases, assuming that it exists, the density can be written in the form of a sum as:
:
for one parameter, or
:
for two parameters, and so forth.
Properties
Convexity
A general
linear combination of probability density functions is not necessarily a probability density, since it may be negative or it may integrate to something other than 1. However, a
convex combination
In convex geometry and vector algebra, a convex combination is a linear combination of points (which can be vectors, scalars, or more generally points in an affine space) where all coefficients are non-negative and sum to 1. In other w ...
of probability density functions preserves both of these properties (non-negativity and integrating to 1), and thus mixture densities are themselves probability density functions.
Moments
Let ''X''
1, ..., ''X''
''n'' denote random variables from the ''n'' component distributions, and let ''X'' denote a random variable from the mixture distribution. Then, for any function ''H''(·) for which