Moment generating function
   HOME

TheInfoList



OR:

In
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
and
statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the moment-generating function of a real-valued
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
is an alternative specification of its
probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
. Thus, it provides the basis of an alternative route to analytical results compared with working directly with
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
s or
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ev ...
s. There are particularly simple results for the moment-generating functions of distributions defined by the weighted sums of random variables. However, not all random variables have moment-generating functions. As its name implies, the moment-
generating function In mathematics, a generating function is a way of encoding an infinite sequence of numbers () by treating them as the coefficients of a formal power series. This series is called the generating function of the sequence. Unlike an ordinary seri ...
can be used to compute a distribution’s moments: the ''n''th moment about 0 is the ''n''th derivative of the moment-generating function, evaluated at 0. In addition to real-valued distributions (univariate distributions), moment-generating functions can be defined for vector- or matrix-valued random variables, and can even be extended to more general cases. The moment-generating function of a real-valued distribution does not always exist, unlike the
characteristic function In mathematics, the term "characteristic function" can refer to any of several distinct concepts: * The indicator function of a subset, that is the function ::\mathbf_A\colon X \to \, :which for a given subset ''A'' of ''X'', has value 1 at points ...
. There are relations between the behavior of the moment-generating function of a distribution and properties of the distribution, such as the existence of moments.


Definition

Let X be a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
with CDF F_X. The moment generating function (mgf) of X (or F_X), denoted by M_X(t), is : M_X(t) = \operatorname E \left ^\right provided this expectation exists for t in some
neighborhood A neighbourhood (British English, Irish English, Australian English and Canadian English) or neighborhood (American English; see spelling differences) is a geographically localised community within a larger city, town, suburb or rural area, ...
of 0. That is, there is an h>0 such that for all t in -h, \operatorname E \left ^\right exists. If the expectation does not exist in a neighborhood of 0, we say that the moment generating function does not exist. In other words, the moment-generating function of ''X'' is the expectation of the random variable e^. More generally, when \mathbf X = ( X_1, \ldots, X_n)^, an n-dimensional
random vector In probability, and statistics, a multivariate random variable or random vector is a list of mathematical variables each of whose value is unknown, either because the value has not yet occurred or because there is imperfect knowledge of its value. ...
, and \mathbf t is a fixed vector, one uses \mathbf t \cdot \mathbf X = \mathbf t^\mathrm T\mathbf X instead of tX: : M_(\mathbf t) := \operatorname E \left(e^\right). M_X(0) always exists and is equal to 1. However, a key problem with moment-generating functions is that moments and the moment-generating function may not exist, as the integrals need not converge absolutely. By contrast, the
characteristic function In mathematics, the term "characteristic function" can refer to any of several distinct concepts: * The indicator function of a subset, that is the function ::\mathbf_A\colon X \to \, :which for a given subset ''A'' of ''X'', has value 1 at points ...
or Fourier transform always exists (because it is the integral of a bounded function on a space of finite
measure Measure may refer to: * Measurement, the assignment of a number to a characteristic of an object or event Law * Ballot measure, proposed legislation in the United States * Church of England Measure, legislation of the Church of England * Mea ...
), and for some purposes may be used instead. The moment-generating function is so named because it can be used to find the moments of the distribution. The series expansion of e^ is : e^ = 1 + t\,X + \frac + \frac + \cdots +\frac + \cdots. Hence : \begin M_X(t) = \operatorname E (e^) &= 1 + t \operatorname E (X) + \frac + \frac+\cdots + \frac+\cdots \\ & = 1 + tm_1 + \frac + \frac+\cdots + \frac + \cdots, \end where m_n is the nth
moment Moment or Moments may refer to: * Present time Music * The Moments, American R&B vocal group Albums * ''Moment'' (Dark Tranquillity album), 2020 * ''Moment'' (Speed album), 1998 * ''Moments'' (Darude album) * ''Moments'' (Christine Guldbrand ...
. Differentiating M_X(t) i times with respect to t and setting t = 0, we obtain the ith moment about the origin, m_i; see Calculations of moments below. If X is a continuous random variable, the following relation between its moment-generating function M_X(t) and the
two-sided Laplace transform In mathematics, the two-sided Laplace transform or bilateral Laplace transform is an integral transform equivalent to probability's moment generating function. Two-sided Laplace transforms are closely related to the Fourier transform, the Mellin t ...
of its probability density function f_X(x) holds: : M_X(t) = \mathcal\(-t), since the PDF's two-sided Laplace transform is given as : \mathcal\(s) = \int_^\infty e^ f_X(x)\, dx, and the moment-generating function's definition expands (by the
law of the unconscious statistician In probability theory and statistics, the law of the unconscious statistician, or LOTUS, is a theorem used to calculate the expected value of a function ''g''(''X'') of a random variable ''X'' when one knows the probability distribution of ''X'' but ...
) to : M_X(t) = \operatorname E \left ^\right= \int_^\infty e^ f_X(x)\, dx. This is consistent with the characteristic function of X being a
Wick rotation In physics, Wick rotation, named after Italian physicist Gian Carlo Wick, is a method of finding a solution to a mathematical problem in Minkowski space from a solution to a related problem in Euclidean space by means of a transformation that sub ...
of M_X(t) when the moment generating function exists, as the characteristic function of a continuous random variable X is the
Fourier transform A Fourier transform (FT) is a mathematical transform that decomposes functions into frequency components, which are represented by the output of the transform as a function of frequency. Most commonly functions of time or space are transformed, ...
of its probability density function f_X(x), and in general when a function f(x) is of exponential order, the Fourier transform of f is a Wick rotation of its two-sided Laplace transform in the region of convergence. See the relation of the Fourier and Laplace transforms for further information.


Examples

Here are some examples of the moment-generating function and the characteristic function for comparison. It can be seen that the characteristic function is a
Wick rotation In physics, Wick rotation, named after Italian physicist Gian Carlo Wick, is a method of finding a solution to a mathematical problem in Minkowski space from a solution to a related problem in Euclidean space by means of a transformation that sub ...
of the moment-generating function M_X(t) when the latter exists. :


Calculation

The moment-generating function is the expectation of a function of the random variable, it can be written as: * For a discrete
probability mass function In probability and statistics, a probability mass function is a function that gives the probability that a discrete random variable is exactly equal to some value. Sometimes it is also known as the discrete density function. The probability mass ...
, M_X(t)=\sum_^\infty e^\, p_i * For a continuous
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
, M_X(t) = \int_^\infty e^ f(x)\,dx * In the general case: M_X(t) = \int_^\infty e^\,dF(x), using the
Riemann–Stieltjes integral In mathematics, the Riemann–Stieltjes integral is a generalization of the Riemann integral, named after Bernhard Riemann and Thomas Joannes Stieltjes. The definition of this integral was first published in 1894 by Stieltjes. It serves as an in ...
, and where F is the
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ev ...
. This is simply the Laplace-Stieltjes transform of F, but with the sign of the argument reversed. Note that for the case where X has a continuous
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
f(x), M_X(-t) is the
two-sided Laplace transform In mathematics, the two-sided Laplace transform or bilateral Laplace transform is an integral transform equivalent to probability's moment generating function. Two-sided Laplace transforms are closely related to the Fourier transform, the Mellin t ...
of f(x). : \begin M_X(t) & = \int_^\infty e^ f(x)\,dx \\ & = \int_^\infty \left( 1+ tx + \frac + \cdots + \frac + \cdots\right) f(x)\,dx \\ & = 1 + tm_1 + \frac +\cdots + \frac +\cdots, \end where m_n is the nth
moment Moment or Moments may refer to: * Present time Music * The Moments, American R&B vocal group Albums * ''Moment'' (Dark Tranquillity album), 2020 * ''Moment'' (Speed album), 1998 * ''Moments'' (Darude album) * ''Moments'' (Christine Guldbrand ...
.


Linear transformations of random variables

If random variable X has moment generating function M_X(t), then \alpha X + \beta has moment generating function M_(t) = e^M_X(\alpha t) : M_(t) = E ^= e^E ^= e^M_X(\alpha t)


Linear combination of independent random variables

If S_n = \sum_^ a_i X_i, where the ''X''''i'' are independent random variables and the ''a''''i'' are constants, then the probability density function for ''S''''n'' is the
convolution In mathematics (in particular, functional analysis), convolution is a operation (mathematics), mathematical operation on two function (mathematics), functions ( and ) that produces a third function (f*g) that expresses how the shape of one is ...
of the probability density functions of each of the ''X''''i'', and the moment-generating function for ''S''''n'' is given by : M_(t)=M_(a_1t)M_(a_2t)\cdots M_(a_nt) \, .


Vector-valued random variables

For vector-valued random variables \mathbf X with
real Real may refer to: Currencies * Brazilian real (R$) * Central American Republic real * Mexican real * Portuguese real * Spanish real * Spanish colonial real Music Albums * ''Real'' (L'Arc-en-Ciel album) (2000) * ''Real'' (Bright album) (2010) ...
components, the moment-generating function is given by : M_X(\mathbf t) = E\left(e^\right) where \mathbf t is a vector and \langle \cdot, \cdot \rangle is the
dot product In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidean space. is an algebra ...
.


Important properties

Moment generating functions are positive and
log-convex In mathematics, a function ''f'' is logarithmically convex or superconvex if \circ f, the composition of the logarithm with ''f'', is itself a convex function. Definition Let be a convex subset of a real vector space, and let be a function taking ...
, with ''M''(0) = 1. An important property of the moment-generating function is that it uniquely determines the distribution. In other words, if X and Y are two random variables and for all values of ''t'', :M_X(t) = M_Y(t),\, then :F_X(x) = F_Y(x) \, for all values of ''x'' (or equivalently ''X'' and ''Y'' have the same distribution). This statement is not equivalent to the statement "if two distributions have the same moments, then they are identical at all points." This is because in some cases, the moments exist and yet the moment-generating function does not, because the limit :\lim_ \sum_^n \frac may not exist. The
log-normal distribution In probability theory, a log-normal (or lognormal) distribution is a continuous probability distribution of a random variable whose logarithm is normally distributed. Thus, if the random variable is log-normally distributed, then has a normal ...
is an example of when this occurs.


Calculations of moments

The moment-generating function is so called because if it exists on an open interval around ''t'' = 0, then it is the
exponential generating function In mathematics, a generating function is a way of encoding an infinite sequence of numbers () by treating them as the coefficients of a formal power series. This series is called the generating function of the sequence. Unlike an ordinary ser ...
of the moments of the
probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
: :m_n = E \left( X^n \right) = M_X^(0) = \left. \frac\_. That is, with ''n'' being a nonnegative integer, the ''n''th moment about 0 is the ''n''th derivative of the moment generating function, evaluated at ''t'' = 0.


Other properties

Jensen's inequality In mathematics, Jensen's inequality, named after the Danish mathematician Johan Jensen, relates the value of a convex function of an integral to the integral of the convex function. It was proved by Jensen in 1906, building on an earlier pr ...
provides a simple lower bound on the moment-generating function: : M_X(t) \geq e^, where \mu is the mean of ''X''. The moment-generating function can be used in conjunction with
Markov's inequality In probability theory, Markov's inequality gives an upper bound for the probability that a non-negative function (mathematics), function of a random variable is greater than or equal to some positive Constant (mathematics), constant. It is named a ...
to bound the upper tail of a real random variable ''X''. This statement is also called the
Chernoff bound In probability theory, the Chernoff bound gives exponentially decreasing bounds on tail distributions of sums of independent random variables. Despite being named after Herman Chernoff, the author of the paper it first appeared in, the result is d ...
. Since x\mapsto e^ is monotonically increasing for t>0, we have : P(X\ge a) = P(e^\ge e^) \le e^E ^= e^M_X(t) for any t>0 and any ''a'', provided M_X(t) exists. For example, when ''X'' is a standard normal distribution and a>0, we can choose t=a and recall that M_X(t)=e^. This gives P(X\ge a)\le e^, which is within a factor of 1+''a'' of the exact value. Various lemmas, such as
Hoeffding's lemma In probability theory, Hoeffding's lemma is an inequality that bounds the moment-generating function of any bounded random variable. It is named after the Finnish–American mathematical statistician Wassily Hoeffding. The proof of Hoeffding ...
or Bennett's inequality provide bounds on the moment-generating function in the case of a zero-mean, bounded random variable. When X is non-negative, the moment generating function gives a simple, useful bound on the moments: :E ^m\le \left(\frac\right)^m M_X(t), For any X,m\ge 0 and t>0. This follows from the inequality 1+x\le e^x into which we can substitute x'=tx/m-1 implies tx/m\le e^ for any x,t,m\in\mathbb R. Now, if t>0 and x,m\ge 0, this can be rearranged to x^m \le (m/(te))^m e^. Taking the expectation on both sides gives the bound on E ^m/math> in terms of E ^/math>. As an example, consider X\sim\text with k degrees of freedom. Then from the
examples Example may refer to: * '' exempli gratia'' (e.g.), usually read out in English as "for example" * .example, reserved as a domain name that may not be installed as a top-level domain of the Internet ** example.com, example.net, example.org, ex ...
M_X(t)=(1-2t)^. Picking t=m/(2m+k) and substituting into the bound: :E ^m\le (1+2m/k)^ e^ (k+2m)^m. We know that in this case the correct bound is E ^mle 2^m \Gamma(m+k/2)/\Gamma(k/2). To compare the bounds, we can consider the asymptotics for large k. Here the moment-generating function bound is k^m(1+m^2/k + O(1/k^2)), where the real bound is k^m(1+(m^2-m)/k + O(1/k^2)). The moment-generating function bound is thus very strong in this case.


Relation to other functions

Related to the moment-generating function are a number of other
transforms Transform may refer to: Arts and entertainment *Transform (scratch), a type of scratch used by turntablists * ''Transform'' (Alva Noto album), 2001 * ''Transform'' (Howard Jones album) or the title song, 2019 * ''Transform'' (Powerman 5000 album) ...
that are common in probability theory: ;
Characteristic function In mathematics, the term "characteristic function" can refer to any of several distinct concepts: * The indicator function of a subset, that is the function ::\mathbf_A\colon X \to \, :which for a given subset ''A'' of ''X'', has value 1 at points ...
: The
characteristic function In mathematics, the term "characteristic function" can refer to any of several distinct concepts: * The indicator function of a subset, that is the function ::\mathbf_A\colon X \to \, :which for a given subset ''A'' of ''X'', has value 1 at points ...
\varphi_X(t) is related to the moment-generating function via \varphi_X(t) = M_(t) = M_X(it): the characteristic function is the moment-generating function of ''iX'' or the moment generating function of ''X'' evaluated on the imaginary axis. This function can also be viewed as the
Fourier transform A Fourier transform (FT) is a mathematical transform that decomposes functions into frequency components, which are represented by the output of the transform as a function of frequency. Most commonly functions of time or space are transformed, ...
of the
probability density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
, which can therefore be deduced from it by inverse Fourier transform. ; Cumulant-generating function: The cumulant-generating function is defined as the logarithm of the moment-generating function; some instead define the cumulant-generating function as the logarithm of the
characteristic function In mathematics, the term "characteristic function" can refer to any of several distinct concepts: * The indicator function of a subset, that is the function ::\mathbf_A\colon X \to \, :which for a given subset ''A'' of ''X'', has value 1 at points ...
, while others call this latter the ''second'' cumulant-generating function. ;
Probability-generating function In probability theory, the probability generating function of a discrete random variable is a power series representation (the generating function) of the probability mass function of the random variable. Probability generating functions are oft ...
: The
probability-generating function In probability theory, the probability generating function of a discrete random variable is a power series representation (the generating function) of the probability mass function of the random variable. Probability generating functions are oft ...
is defined as G(z) = E\left ^X\right\, This immediately implies that G(e^t) = E\left ^\right= M_X(t).\,


See also

*
Characteristic function (probability theory) In probability theory and statistics, the characteristic function of any real-valued random variable completely defines its probability distribution. If a random variable admits a probability density function, then the characteristic function is ...
*
Entropic value at risk In financial mathematics and stochastic optimization, the concept of risk measure is used to quantify the risk involved in a random outcome or risk position. Many risk measures have hitherto been proposed, each having certain characteristics. The en ...
*
Factorial moment generating function In probability theory and statistics, the factorial moment generating function (FMGF) of the probability distribution of a real-valued random variable ''X'' is defined as :M_X(t)=\operatorname\bigl ^\bigr/math> for all complex numbers ''t'' for w ...
*
Rate function In mathematics — specifically, in large deviations theory — a rate function is a function used to quantify the probabilities of rare events. It is required to have several properties which assist in the formulation of the large deviati ...
*
Hamburger moment problem In mathematics, the Hamburger moment problem, named after Hans Ludwig Hamburger, is formulated as follows: given a sequence (''m''0, ''m''1, ''m''2, ...), does there exist a positive Borel measure ''μ'' (for instance, the measure determined by the ...


References


Citations


Sources

* {{DEFAULTSORT:Moment-Generating Function Moment (mathematics) Generating functions