![ProbOnto2](https://upload.wikimedia.org/wikipedia/commons/a/af/ProbOnto2.5.jpg)
In
probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
and
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, there are several relationships among
probability distributions. These relations can be categorized in the following groups:
*One distribution is a special case of another with a broader parameter space
*Transforms (function of a random variable);
*Combinations (function of several variables);
*Approximation (limit) relationships;
*Compound relationships (useful for Bayesian inference);
*
Duality
Duality may refer to:
Mathematics
* Duality (mathematics), a mathematical concept
** Dual (category theory), a formalization of mathematical duality
** Duality (optimization)
** Duality (order theory), a concept regarding binary relations
** Dual ...
;
*
Conjugate prior
In Bayesian probability theory, if the posterior distribution p(\theta \mid x) is in the same probability distribution family as the prior probability distribution p(\theta), the prior and posterior are then called conjugate distributions, and th ...
s.
Special case of distribution parametrization
* A
binomial distribution
In probability theory and statistics, the binomial distribution with parameters ''n'' and ''p'' is the discrete probability distribution of the number of successes in a sequence of ''n'' independent experiments, each asking a yes–no quest ...
with parameters ''n'' = 1 and ''p'' is a
Bernoulli distribution
In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabil ...
with parameter ''p''.
* A
negative binomial distribution
In probability theory and statistics, the negative binomial distribution is a discrete probability distribution that models the number of failures in a sequence of independent and identically distributed Bernoulli trials before a specified (non-r ...
with parameters ''n'' = 1 and ''p'' is a
geometric distribution with parameter ''p''.
* A
gamma distribution
In probability theory and statistics, the gamma distribution is a two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-square distribution are special cases of the gamma distri ...
with shape parameter ''α'' = 1 and rate parameter ''β'' is an
exponential distribution
In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average ...
with rate parameter ''β''.
* A
gamma distribution
In probability theory and statistics, the gamma distribution is a two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-square distribution are special cases of the gamma distri ...
with shape parameter ''α'' = ''v''/2 and rate parameter ''β'' = 1/2 is a
chi-squared distribution
In probability theory and statistics, the chi-squared distribution (also chi-square or \chi^2-distribution) with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables. The chi-squa ...
with ''ν''
degrees of freedom.
* A
chi-squared distribution
In probability theory and statistics, the chi-squared distribution (also chi-square or \chi^2-distribution) with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables. The chi-squa ...
with 2 degrees of freedom (''k'' = 2) is an
exponential distribution
In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average ...
with a mean value of 2 (rate ''λ'' = 1/2 .)
* A
Weibull distribution with shape parameter ''k'' = 1 and rate parameter ''β'' is an
exponential distribution
In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average ...
with rate parameter ''β''.
* A
beta distribution
In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval , 1in terms of two positive parameters, denoted by ''alpha'' (''α'') and ''beta'' (''β''), that appear as ...
with shape parameters ''α'' = ''β'' = 1 is a
continuous uniform distribution
In probability theory and statistics, the continuous uniform distribution or rectangular distribution is a family of symmetric probability distributions. The distribution describes an experiment where there is an arbitrary outcome that lies betw ...
over the real numbers 0 to 1.
* A
beta-binomial distribution
In probability theory and statistics, the beta-binomial distribution is a family of discrete probability distributions on a finite support of non-negative integers arising when the probability of success in each of a fixed or known number of Bern ...
with parameter ''n'' and shape parameters ''α'' = ''β'' = 1 is a
discrete uniform distribution
In probability theory and statistics, the discrete uniform distribution is a symmetric probability distribution wherein a finite number of values are equally likely to be observed; every one of ''n'' values has equal probability 1/''n''. Anothe ...
over the integers 0 to ''n''.
* A
Student's t-distribution
In probability and statistics, Student's ''t''-distribution (or simply the ''t''-distribution) is any member of a family of continuous probability distributions that arise when estimating the mean of a normally distributed population in sit ...
with one degree of freedom (''v'' = 1) is a
Cauchy distribution with location parameter ''x'' = 0 and scale parameter ''γ'' = 1.
* A
Burr distribution with parameters ''c'' = 1 and ''k'' (and scale ''λ'') is a
Lomax distribution with shape ''k'' (and scale ''λ''.)
Transform of a variable
Multiple of a random variable
Multiplying the variable by any positive real constant yields a scaling of the original distribution.
Some are self-replicating, meaning that the scaling yields the same family of distributions, albeit with a different parameter:
normal distribution
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu ...
,
gamma distribution
In probability theory and statistics, the gamma distribution is a two-parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-square distribution are special cases of the gamma distri ...
,
Cauchy distribution,
exponential distribution
In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average ...
,
Erlang distribution
The Erlang distribution is a two-parameter family of continuous probability distributions with support x \in independent exponential distribution">exponential variables with mean 1/\lambda each. Equivalently, it is the distribution of the time ...
,
Weibull distribution,
logistic distribution,
error distribution,
power-law distribution
In statistics, a power law is a functional relationship between two quantities, where a relative change in one quantity results in a proportional relative change in the other quantity, independent of the initial size of those quantities: one q ...
,
Rayleigh distribution.
Example:
* If ''X'' is a gamma random variable with shape and rate parameters (''α'', ''β''), then ''Y'' = ''aX'' is a gamma random variable with parameters (''α'',''β''/''a'').
* If ''X'' is a gamma random variable with shape and scale parameters (''k'', ''θ''), then ''Y'' = ''aX'' is a gamma random variable with parameters (''k'',''aθ'').
Linear function of a random variable
The affine transform ''ax'' + ''b'' yields a relocation and scaling of the original distribution. The following are self-replicating:
Normal distribution
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu ...
,
Cauchy distribution,
Logistic distribution,
Error distribution,
Power distribution
Electric power distribution is the final stage in the delivery of electric power; it carries electricity from the transmission system to individual consumers. Distribution substations connect to the transmission system and lower the transmissio ...
,
Rayleigh distribution.
Example:
* If ''Z'' is a normal random variable with parameters (''μ'' = ''m'', ''σ''
2 = ''s''
2), then ''X'' = ''aZ'' + ''b'' is a normal random variable with parameters (''μ'' = ''am'' + ''b'', ''σ''
2 = ''a''
2''s''
2).
Reciprocal of a random variable
The reciprocal 1/''X'' of a random variable ''X'', is a member of the same family of distribution as ''X'', in the following cases:
Cauchy distribution,
F distribution
In probability theory and statistics, the ''F''-distribution or F-ratio, also known as Snedecor's ''F'' distribution or the Fisher–Snedecor distribution (after Ronald Fisher and George W. Snedecor) is a continuous probability distribution t ...
,
log logistic distribution
In probability and statistics, the log-logistic distribution (known as the Fisk distribution in economics) is a continuous probability distribution for a non-negative random variable. It is used in survival analysis as a parametric model for events ...
.
Examples:
* If X is a Cauchy (''μ'', ''σ'') random variable, then 1/''X'' is a Cauchy (''μ''/''C'', ''σ''/''C'') random variable where ''C'' = ''μ''
2 + ''σ''
2.
* If ''X'' is an ''F''(''ν''
1, ''ν''
2) random variable then 1/''X'' is an ''F''(''ν''
2, ''ν''
1) random variable.
Other cases
Some distributions are invariant under a specific transformation.
Example:
* If ''X'' is a beta (''α'', ''β'') random variable then (1 − ''X'') is a beta (''β'', ''α'') random variable.
* If ''X'' is a binomial (''n'', ''p'') random variable then (''n'' − ''X'') is a binomial (''n'', 1 − ''p'') random variable.
* If ''X'' has
cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ev ...
''F''
''X'', then the inverse of the cumulative distribution ''F''(''X'') is a standard uniform (0,1) random variable
* If ''X'' is a normal (''μ'', ''σ''
2) random variable then ''e''
''X'' is a lognormal (''μ'', ''σ''
2) random variable.
:Conversely, if ''X'' is a lognormal (''μ'', ''σ''
2) random variable then log ''X'' is a normal (''μ'', ''σ''
2) random variable.
* If ''X'' is an exponential random variable with mean ''β'', then ''X''
1/''γ'' is a Weibull (''γ'', ''β'') random variable.
* The square of a standard normal random variable has a chi-squared distribution with one degree of freedom.
* If ''X'' is a
Student’s t random variable with ''ν'' degree of freedom, then ''X''
2 is an ''F'' (1,''ν'') random variable.
* If ''X'' is a double exponential random variable with mean 0 and scale ''λ'', then , ''X'', is an exponential random variable with mean ''λ''.
* A geometric random variable is the
floor
A floor is the bottom surface of a room or vehicle. Floors vary from simple dirt in a cave to many layered surfaces made with modern technology. Floors may be stone, wood, bamboo, metal or any other material that can support the expected load ...
of an exponential random variable.
* A
rectangular random variable is the floor of a uniform random variable.
* A
reciprocal
Reciprocal may refer to:
In mathematics
* Multiplicative inverse, in mathematics, the number 1/''x'', which multiplied by ''x'' gives the product 1, also known as a ''reciprocal''
* Reciprocal polynomial, a polynomial obtained from another pol ...
random variable is the exponential of a uniform random variable.
Functions of several variables
Sum of variables
The distribution of the sum of
independent random variables is the
convolution
In mathematics (in particular, functional analysis), convolution is a operation (mathematics), mathematical operation on two function (mathematics), functions ( and ) that produces a third function (f*g) that expresses how the shape of one is ...
of their distributions. Suppose
is the sum of
independent random variables
each with
probability mass functions . Then
has
If it has a distribution from the same family of distributions as the original variables, that family of distributions is said to be ''closed under convolution''.
Examples of such
univariate distributions are:
normal distributions
In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
:
f(x) = \frac e^
The parameter \mu is t ...
,
Poisson distributions,
binomial distributions (with common success probability),
negative binomial distributions (with common success probability),
gamma distributions (with common
rate parameter
In probability theory and statistics, a scale parameter is a special kind of numerical parameter of a parametric family of probability distributions. The larger the scale parameter, the more spread out the distribution.
Definition
If a family o ...
),
chi-squared distributions,
Cauchy distributions,
hyperexponential distributions.
Examples:
**If ''X''
1 and ''X''
2 are Poisson random variables with means ''μ''
1 and ''μ''
2 respectively, then ''X''
1 + ''X''
2 is a Poisson random variable with mean ''μ''
1 + ''μ''
2.
** The sum of gamma (''α''
''i'', ''β'') random variables has a gamma (Σ''α''
''i'', ''β'') distribution.
**If ''X''
1 is a Cauchy (''μ''
1, ''σ''
1) random variable and ''X''
2 is a Cauchy (''μ''
2, ''σ''
2), then ''X''
1 + ''X''
2 is a Cauchy (''μ''
1 + ''μ''
2, ''σ''
1 + ''σ''
2) random variable.
**If X
1 and X
2 are chi-squared random variables with ν
1 and ν
2 degrees of freedom respectively, then X
1 + X
2 is a chi-squared random variable with ν
1 + ν
2 degrees of freedom.
**If ''X''
1 is a normal (''μ''
1, ''σ'') random variable and ''X''
2 is a normal (''μ''
2, ''σ'') random variable, then X
1 + ''X''
2 is a normal (''μ''
1 + ''μ''
2, ''σ'' + ''σ'') random variable.
**The sum of ''N'' chi-squared (1) random variables has a chi-squared distribution with ''N'' degrees of freedom.
Other distributions are not closed under convolution, but their sum has a known distribution:
* The sum of ''n'' Bernoulli (p) random variables is a binomial (''n'', ''p'') random variable.
* The sum of ''n'' geometric random variables with probability of success ''p'' is a negative binomial random variable with parameters ''n'' and ''p''.
* The sum of ''n'' exponential (''β'') random variables is a gamma (''n'', ''β'') random variable. Since n is an integer, the gamma distribution is also a
Erlang distribution
The Erlang distribution is a two-parameter family of continuous probability distributions with support x \in independent exponential distribution">exponential variables with mean 1/\lambda each. Equivalently, it is the distribution of the time ...
.
*The sum of the squares of ''N'' standard normal random variables has a chi-squared distribution with N degrees of freedom.
Product of variables
The product of independent random variables ''X'' and ''Y'' may belong to the same family of distribution as ''X'' and ''Y'':
Bernoulli distribution
In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabil ...
and
log-normal distribution
In probability theory, a log-normal (or lognormal) distribution is a continuous probability distribution of a random variable whose logarithm is normally distributed. Thus, if the random variable is log-normally distributed, then has a normal ...
.
Example:
* If ''X''
1 and ''X''
2 are independent log-normal random variables with parameters (''μ''
1, ''σ'') and (''μ''
2, ''σ'') respectively, then ''X''
1 ''X''
2 is a log-normal random variable with parameters (''μ''
1 + ''μ''
2, ''σ'' + ''σ'').
Minimum and maximum of independent random variables
For some distributions, the minimum value of several independent random variables is a member of the same family, with different parameters:
Bernoulli distribution
In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabil ...
,
Geometric distribution,
Exponential distribution
In probability theory and statistics, the exponential distribution is the probability distribution of the time between events in a Poisson point process, i.e., a process in which events occur continuously and independently at a constant average ...
,
Extreme value distribution
In probability theory and statistics, the generalized extreme value (GEV) distribution is a family of continuous probability distributions developed within extreme value theory to combine the Gumbel, Fréchet and Weibull families also known as ...
,
Pareto distribution
The Pareto distribution, named after the Italian civil engineer, economist, and sociologist Vilfredo Pareto ( ), is a power-law probability distribution that is used in description of social, quality control, scientific, geophysical, actua ...
,
Rayleigh distribution,
Weibull distribution.
Examples:
* If ''X''
1 and ''X''
2 are independent geometric random variables with probability of success ''p''
1 and ''p''
2 respectively, then min(''X''
1, ''X''
2) is a geometric random variable with probability of success ''p'' = ''p''
1 + ''p''
2 − ''p''
1 ''p''
2. The relationship is simpler if expressed in terms probability of failure: ''q'' = ''q''
1 ''q''
2.
* If ''X''
1 and ''X''
2 are independent exponential random variables with rate ''μ''
1 and ''μ''
2 respectively, then min(''X''
1, ''X''
2) is an exponential random variable with rate ''μ'' = ''μ''
1 + ''μ''
2.
Similarly, distributions for which the maximum value of several independent random variables is a member of the same family of distribution include:
Bernoulli distribution
In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabil ...
,
Power law
In statistics, a power law is a Function (mathematics), functional relationship between two quantities, where a Relative change and difference, relative change in one quantity results in a proportional relative change in the other quantity, inde ...
distribution.
Other
* If ''X'' and ''Y'' are independent standard normal random variables, ''X''/''Y'' is a Cauchy (0,1) random variable.
* If ''X''
1 and ''X''
2 are independent chi-squared random variables with ''ν''
1 and ''ν''
2 degrees of freedom respectively, then (''X''
1/''ν''
1)/(''X''
2/''ν''
2) is an ''F''(''ν''
1, ''ν''
2) random variable.
* If ''X'' is a standard normal random variable and U is an independent chi-squared random variable with ''ν'' degrees of freedom, then
is a Student's ''t''(''ν'') random variable.
* If ''X''
1 is a gamma (''α''
1, 1) random variable and ''X''
2 is an independent gamma (α
2, 1) random variable then ''X''
1/(''X''
1 + ''X''
2) is a beta(''α''
1, ''α''
2) random variable. More generally, if ''X''
1 is a gamma(''α''
1, ''β''
1) random variable and ''X''
2 is an independent gamma(''α''
2, ''β''
2) random variable then β
2 X
1/(''β''
2 ''X''
1 + ''β''
1 ''X''
2) is a beta(''α''
1, ''α''
2) random variable.
* If ''X'' and ''Y'' are independent exponential random variables with mean μ, then ''X'' − ''Y'' is a
double exponential random variable with mean 0 and scale μ.
*If X
i are independent Bernoulli random variables then their
parity
Parity may refer to:
* Parity (computing)
** Parity bit in computing, sets the parity of data for the purpose of error detection
** Parity flag in computing, indicates if the number of set bits is odd or even in the binary representation of the r ...
(XOR) is a Bernoulli variable described by the
piling-up lemma
In cryptanalysis, the piling-up lemma is a principle used in linear cryptanalysis to construct linear approximation, linear approximations to the action of block ciphers. It was introduced by Mitsuru Matsui (1993) as an analytical tool for linear c ...
.
Approximate (limit) relationships
Approximate or limit relationship means
*either that the combination of an infinite number of ''iid'' random variables tends to some distribution,
*or that the limit when a parameter tends to some value approaches to a different distribution.
Combination of ''iid'' random variables:
* Given certain conditions, the sum (hence the average) of a sufficiently large number of iid random variables, each with finite mean and variance, will be approximately normally distributed. This is the
central limit theorem (CLT).
Special case of distribution parametrization:
* ''X'' is a hypergeometric (''m'', ''N'', ''n'') random variable. If ''n'' and ''m'' are large compared to ''N'', and ''p'' = ''m''/''N'' is not close to 0 or 1, then ''X'' approximately has a Binomial(''n'', ''p'') distribution.
* ''X'' is a beta-binomial random variable with parameters (''n'', ''α'', ''β''). Let ''p'' = ''α''/(''α'' + ''β'') and suppose ''α'' + ''β'' is large, then ''X'' approximately has a binomial(''n'', ''p'') distribution.
* If ''X'' is a binomial (''n'', ''p'') random variable and if ''n'' is large and ''np'' is small then ''X'' approximately has a Poisson(''np'') distribution.
* If ''X'' is a negative binomial random variable with ''r'' large, ''P'' near 1, and ''r''(1 − ''P'') = ''λ'', then ''X'' approximately has a Poisson distribution with mean ''λ''.
Consequences of the CLT:
* If ''X'' is a Poisson random variable with large mean, then for integers ''j'' and ''k'', P(''j'' ≤ ''X'' ≤ ''k'') approximately equals to ''P''(''j'' − 1/2 ≤ ''Y'' ≤ ''k'' + 1/2) where ''Y'' is a normal distribution with the same mean and variance as ''X''.
* If ''X'' is a binomial(''n'', ''p'') random variable with large ''np'' and ''n''(1 − ''p''), then for integers ''j'' and ''k'', P(''j'' ≤ ''X'' ≤ ''k'') approximately equals to P(''j'' − 1/2 ≤ ''Y'' ≤ ''k'' + 1/2) where ''Y'' is a normal random variable with the same mean and variance as ''X'', i.e. ''np'' and ''np''(1 − ''p'').
* If ''X'' is a beta random variable with parameters ''α'' and ''β'' equal and large, then ''X'' approximately has a normal distribution with the same mean and variance, i. e. mean ''α''/(''α'' + ''β'') and variance ''αβ''/((''α'' + ''β'')
2(''α'' + ''β'' + 1)).
* If ''X'' is a gamma(''α'', ''β'') random variable and the shape parameter ''α'' is large relative to the scale parameter ''β'', then ''X'' approximately has a normal random variable with the same mean and variance.
* If ''X'' is a Student's ''t'' random variable with a large number of degrees of freedom ''ν'' then ''X'' approximately has a standard normal distribution.
* If ''X'' is an ''F''(''ν'', ''ω'') random variable with ''ω'' large, then ''νX'' is approximately distributed as a chi-squared random variable with ''ν'' degrees of freedom.
Compound (or Bayesian) relationships
When one or more parameter(s) of a distribution are random variables, the
compound distribution is the marginal distribution of the variable.
Examples:
* If ''X'' , ''N'' is a binomial (''N'',''p'') random variable, where parameter ''N'' is a random variable with negative-binomial (''m'', ''r'') distribution, then ''X'' is distributed as a negative-binomial (''m'', ''r''/(''p'' + ''qr'')).
* If ''X'' , ''N'' is a binomial (''N'',''p'') random variable, where parameter ''N'' is a random variable with Poisson(''μ'') distribution, then ''X'' is distributed as a Poisson (''μp'').
* If ''X'' , ''μ'' is a Poisson(''μ'') random variable and parameter ''μ'' is random variable with gamma(''m'', ''θ'') distribution (where ''θ'' is the scale parameter), then ''X'' is distributed as a negative-binomial (''m'', ''θ''/(1 + ''θ'')), sometimes called
gamma-Poisson distribution
In probability theory and statistics, the negative binomial distribution is a discrete probability distribution that models the number of failures in a sequence of independent and identically distributed Bernoulli trials before a specified (non-r ...
.
Some distributions have been specially named as compounds:
beta-binomial distribution
In probability theory and statistics, the beta-binomial distribution is a family of discrete probability distributions on a finite support of non-negative integers arising when the probability of success in each of a fixed or known number of Bern ...
,
Beta negative binomial distribution,
gamma-normal distribution.
Examples:
* If ''X'' is a Binomial(''n'',''p'') random variable, and parameter p is a random variable with beta(''α'', ''β'') distribution, then ''X'' is distributed as a Beta-Binomial(''α'',''β'',''n'').
* If ''X'' is a negative-binomial(''r'',''p'') random variable, and parameter ''p'' is a random variable with beta(''α'',''β'') distribution, then ''X'' is distributed as a
Beta negative binomial distribution(''r'',''α'',''β'').
See also
*
Central limit theorem
*
Compound probability distribution In probability and statistics, a compound probability distribution (also known as a mixture distribution or contagious distribution) is the probability distribution that results from assuming that a random variable is distributed according to some p ...
*
List of convolutions of probability distributions
References
{{Reflist
External links
* Interactive graphic
Univariate Distribution Relationships*
ProbOnto
ProbOnto is a knowledge base and ontology of probability distributions.Main project website, URL: http://probonto.org ProbOnto 2.5 (released on January 16, 2017) contains over 150 uni- and multivariate distributions and alternative parameterizat ...
- Ontology and knowledge base of probability distributions
ProbOnto Probability Distributome project includes calculators, simulators, experiments, and navigators for inter-distributional refashions and distribution meta-data
Theory of probability distributions