HOME

TheInfoList



OR:

In
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set ...
and
statistics Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
, the characteristic function of any real-valued random variable completely defines its probability distribution. If a random variable admits a probability density function, then the characteristic function is the Fourier transform of the probability density function. Thus it provides an alternative route to analytical results compared with working directly with probability density functions or cumulative distribution functions. There are particularly simple results for the characteristic functions of distributions defined by the weighted sums of random variables. In addition to univariate distributions, characteristic functions can be defined for vector- or matrix-valued random variables, and can also be extended to more generic cases. The characteristic function always exists when treated as a function of a real-valued argument, unlike the moment-generating function. There are relations between the behavior of the characteristic function of a distribution and properties of the distribution, such as the existence of moments and the existence of a density function.


Introduction

The characteristic function provides an alternative way for describing a random variable. Similar to the cumulative distribution function, :F_X(x) = \operatorname \left mathbf_ \right/math> (where 1 is the indicator function — it is equal to 1 when , and zero otherwise), which completely determines the behavior and properties of the probability distribution of the random variable ''X''. The characteristic function, : \varphi_X(t) = \operatorname \left e^ \right also completely determines the behavior and properties of the probability distribution of the random variable ''X''. The two approaches are equivalent in the sense that knowing one of the functions it is always possible to find the other, yet they provide different insights for understanding the features of the random variable. Moreover, in particular cases, there can be differences in whether these functions can be represented as expressions involving simple standard functions. If a random variable admits a
density function In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
, then the characteristic function is its Fourier dual, in the sense that each of them is a Fourier transform of the other. If a random variable has a moment-generating function M_X(t), then the domain of the characteristic function can be extended to the complex plane, and : \varphi_X(-it) = M_X(t). Note however that the characteristic function of a distribution always exists, even when the probability density function or moment-generating function do not. The characteristic function approach is particularly useful in analysis of linear combinations of independent random variables: a classical proof of the Central Limit Theorem uses characteristic functions and
Lévy's continuity theorem In probability theory, Lévy’s continuity theorem, or Lévy's convergence theorem, named after the French mathematician Paul Lévy, connects convergence in distribution of the sequence of random variables with pointwise convergence of their cha ...
. Another important application is to the theory of the decomposability of random variables.


Definition

For a scalar random variable ''X'' the characteristic function is defined as the expected value of ''eitX'', where ''i'' is the imaginary unit, and is the argument of the characteristic function: :\begin \displaystyle \varphi_X\!:\mathbb\to\mathbb \\ \displaystyle \varphi_X(t) = \operatorname\left ^\right= \int_ e^\,dF_X(x) = \int_ e^ f_X(x)\,dx = \int_0^1 e^\,dp \end Here ''FX'' is the cumulative distribution function of ''X'', and the integral is of the Riemann–Stieltjes kind. If a random variable ''X'' has a probability density function ''fX'', then the characteristic function is its Fourier transform with sign reversal in the complex exponential. ''QX''(''p'') is the inverse cumulative distribution function of ''X'' also called the quantile function of ''X''. This convention for the constants appearing in the definition of the characteristic function differs from the usual convention for the Fourier transform. For example, some authors define , which is essentially a change of parameter. Other notation may be encountered in the literature: \scriptstyle\hat p as the characteristic function for a probability measure ''p'', or \scriptstyle\hat f as the characteristic function corresponding to a density ''f''.


Generalizations

The notion of characteristic functions generalizes to multivariate random variables and more complicated
random element In probability theory, random element is a generalization of the concept of random variable to more complicated spaces than the simple real line. The concept was introduced by who commented that the “development of probability theory and expansi ...
s. The argument of the characteristic function will always belong to the continuous dual of the space where the random variable ''X'' takes its values. For common cases such definitions are listed below: * If ''X'' is a ''k''-dimensional random vector, then for \varphi_X(t) = \operatorname\left exp( i t^T\!X)\right where t^T is the transpose of the vector   t , * If ''X'' is a ''k'' × ''p''-dimensional random matrix, then for \varphi_X(t) = \operatorname\left exp \left( i \operatorname(t^T\!X) \right )\right where \operatorname(\cdot) is the trace operator, * If ''X'' is a complex random variable, then for \varphi_X(t) = \operatorname\left exp\left( i \operatorname\left(\overlineX\right) \right)\right where \overline t is the complex conjugate of t and \operatorname(z) is the real part of the complex number z , * If ''X'' is a ''k''-dimensional complex random vector, then for    \varphi_X(t) = \operatorname\left exp(i\operatorname(t^*\!X))\right where t^* is the conjugate transpose of the vector   t, * If ''X''(''s'') is a stochastic process, then for all functions ''t''(''s'') such that the integral \int_ t(s)X(s)\,\mathrms converges for almost all realizations of ''X'' \varphi_X(t) = \operatorname\left exp \left ( i\int_\mathbf t(s)X(s) \, ds \right ) \right


Examples

Oberhettinger (1973) provides extensive tables of characteristic functions.


Properties

* The characteristic function of a real-valued random variable always exists, since it is an integral of a bounded continuous function over a space whose measure is finite. * A characteristic function is uniformly continuous on the entire space * It is non-vanishing in a region around zero: φ(0) = 1. * It is bounded: , φ(''t''), ≤ 1. * It is Hermitian: . In particular, the characteristic function of a symmetric (around the origin) random variable is real-valued and
even Even may refer to: General * Even (given name), a Norwegian male personal name * Even (surname) * Even (people), an ethnic group from Siberia and Russian Far East **Even language, a language spoken by the Evens * Odd and Even, a solitaire game wh ...
. * There is a bijection between probability distributions and characteristic functions. That is, for any two random variables ''X''1, ''X''2, both have the same probability distribution if and only if \varphi_=\varphi_. * If a random variable ''X'' has moments up to ''k''-th order, then the characteristic function φ''X'' is ''k'' times continuously differentiable on the entire real line. In this case \operatorname ^k= i^ \varphi_X^(0). * If a characteristic function φ''X'' has a ''k''-th derivative at zero, then the random variable ''X'' has all moments up to ''k'' if ''k'' is even, but only up to if ''k'' is odd. \varphi_X^(0) = i^k \operatorname ^k * If ''X''1, ..., ''Xn'' are independent random variables, and ''a''1, ..., ''an'' are some constants, then the characteristic function of the linear combination of the ''X''''i'' 's is \varphi_(t) = \varphi_(a_1t)\cdots \varphi_(a_nt). One specific case is the sum of two independent random variables ''X''1 and ''X''2 in which case one has \varphi_(t) = \varphi_(t)\cdot\varphi_(t). * Let X and Y be two random variables with characteristic functions \varphi_ and \varphi_. X and Y are independent if and only if \varphi_(s, t)= \varphi_(s) \varphi_(t) \quad \text \quad(s, t) \in \mathbb^. * The tail behavior of the characteristic function determines the smoothness of the corresponding density function. * Let the random variable Y = aX + b be the linear transformation of a random variable X. The characteristic function of Y is \varphi_Y(t)=e^\varphi_X(at). For random vectors X and Y = AX + B (where ''A'' is a constant matrix and ''B'' a constant vector), we have \varphi_Y(t) = e^\varphi_X(A^\top t).


Continuity

The bijection stated above between probability distributions and characteristic functions is ''sequentially continuous''. That is, whenever a sequence of distribution functions ''Fj''(''x'') converges (weakly) to some distribution ''F''(''x''), the corresponding sequence of characteristic functions φ''j''(''t'') will also converge, and the limit φ(''t'') will correspond to the characteristic function of law ''F''. More formally, this is stated as : Lévy’s continuity theorem: A sequence ''Xj'' of ''n''-variate random variables converges in distribution to random variable ''X'' if and only if the sequence φ''Xj'' converges pointwise to a function φ which is continuous at the origin. Where φ is the characteristic function of ''X''. This theorem can be used to prove the law of large numbers and the central limit theorem.


Inversion formula

There is a one-to-one correspondence between cumulative distribution functions and characteristic functions, so it is possible to find one of these functions if we know the other. The formula in the definition of characteristic function allows us to compute ''φ'' when we know the distribution function ''F'' (or density ''f''). If, on the other hand, we know the characteristic function ''φ'' and want to find the corresponding distribution function, then one of the following inversion theorems can be used. Theorem. If the characteristic function ''φX'' of a random variable ''X'' is integrable, then ''FX'' is absolutely continuous, and therefore ''X'' has a probability density function. In the univariate case (i.e. when ''X'' is scalar-valued) the density function is given by f_X(x) = F_X'(x) = \frac\int_ e^\varphi_X(t)\,dt. In the multivariate case it is f_X(x) = \frac \int_ e^\varphi_X(t)\lambda(dt) where t\cdot x is the dot-product. The pdf is the Radon–Nikodym derivative of the distribution ''μX'' with respect to the Lebesgue measure ''λ'': f_X(x) = \frac(x). Theorem (Lévy). If ''φ''''X'' is characteristic function of distribution function ''FX'', two points ''a'' < ''b'' are such that is a
continuity set In measure theory, a branch of mathematics, a continuity set of a measure ''μ'' is any Borel set ''B'' such that : \mu(\partial B) = 0\,, where \partial B is the (topological) boundary of ''B''. For signed measures, one asks that : , \mu, (\ ...
of ''μ''''X'' (in the univariate case this condition is equivalent to continuity of ''FX'' at points ''a'' and ''b''), then * If ''X'' is scalar: F_X(b) - F_X(a) = \frac \lim_ \int_^ \frac \, \varphi_X(t)\, dt. This formula can be re-stated in a form more convenient for numerical computation as \frac = \frac \int_^ \frac e^ \varphi_X(t) \, dt . For a random variable bounded from below one can obtain F(b) by taking a such that F(a)=0. Otherwise, if a random variable is not bounded from below, the limit for a\to-\infty gives F(b), but is numerically impractical. * If ''X'' is a vector random variable: \mu_X\big(\\big) = \frac \lim_\cdots\lim_ \int\limits_ \cdots \int\limits_ \prod_^n\left(\frac\right)\varphi_X(t)\lambda(dt_1 \times \cdots \times dt_n) Theorem. If ''a'' is (possibly) an atom of ''X'' (in the univariate case this means a point of discontinuity of ''FX'' ) then * If ''X'' is scalar: F_X(a) - F_X(a-0) = \lim_\frac \int_^ e^\varphi_X(t)\,dt * If ''X'' is a vector random variable: \mu_X(\) = \lim_\cdots\lim_ \left(\prod_^n\frac\right) \int\limits_ e^\varphi_X(t)\lambda(dt) Theorem (Gil-Pelaez). For a univariate random variable ''X'', if ''x'' is a
continuity point Continuity or continuous may refer to: Mathematics * Continuity (mathematics), the opposing concept to discreteness; common examples include ** Continuous probability distribution or random variable in probability and statistics ** Continuous g ...
of ''FX'' then : F_X(x) = \frac - \frac\int_0^\infty \frac\,dt. where the imaginary part of a complex number z is given by \mathrm(z) = (z - z^*)/2i. The integral may be not
Lebesgue-integrable In mathematics, the integral of a non-negative function of a single variable can be regarded, in the simplest case, as the area between the graph of that function and the -axis. The Lebesgue integral, named after French mathematician Henri Le ...
; for example, when ''X'' is the
discrete random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
that is always 0, it becomes the
Dirichlet integral In mathematics, there are several integrals known as the Dirichlet integral, after the German mathematician Peter Gustav Lejeune Dirichlet, one of which is the improper integral of the sinc function over the positive real line: : \int_0^ ...
. Inversion formulas for multivariate distributions are available.


Criteria for characteristic functions

The set of all characteristic functions is closed under certain operations: *A convex linear combination \sum_n a_n\varphi_n(t) (with a_n\geq0,\ \sum_n a_n=1) of a finite or a countable number of characteristic functions is also a characteristic function. * The product of a finite number of characteristic functions is also a characteristic function. The same holds for an infinite product provided that it converges to a function continuous at the origin. *If ''φ'' is a characteristic function and α is a real number, then \bar, Re(''φ''), , ''φ'', 2, and ''φ''(''αt'') are also characteristic functions. It is well known that any non-decreasing càdlàg function ''F'' with limits ''F''(−∞) = 0, ''F''(+∞) = 1 corresponds to a cumulative distribution function of some random variable. There is also interest in finding similar simple criteria for when a given function ''φ'' could be the characteristic function of some random variable. The central result here is Bochner’s theorem, although its usefulness is limited because the main condition of the theorem, non-negative definiteness, is very hard to verify. Other theorems also exist, such as Khinchine’s, Mathias’s, or Cramér’s, although their application is just as difficult. Pólya’s theorem, on the other hand, provides a very simple convexity condition which is sufficient but not necessary. Characteristic functions which satisfy this condition are called Pólya-type. Bochner’s theorem. An arbitrary function ''φ'' : R''n'' → C is the characteristic function of some random variable if and only if ''φ'' is
positive definite In mathematics, positive definiteness is a property of any object to which a bilinear form or a sesquilinear form may be naturally associated, which is positive-definite. See, in particular: * Positive-definite bilinear form * Positive-definite fu ...
, continuous at the origin, and if ''φ''(0) = 1. Khinchine’s criterion. A complex-valued, absolutely continuous function ''φ'', with ''φ''(0) = 1, is a characteristic function if and only if it admits the representation : \varphi(t) = \int_ g(t+\theta)\overline \, d\theta . Mathias’ theorem. A real-valued, even, continuous, absolutely integrable function ''φ'', with ''φ''(0) = 1, is a characteristic function if and only if :(-1)^n \left ( \int_ \varphi(pt)e^ H_(t) \, dt \right ) \geq 0 for ''n'' = 0,1,2,..., and all ''p'' > 0. Here ''H''2''n'' denotes the Hermite polynomial of degree 2''n''. Pólya’s theorem. If \varphi is a real-valued, even, continuous function which satisfies the conditions * \varphi(0) = 1 , * \varphi is convex for t>0 , * \varphi(\infty) = 0 , then ''φ''(''t'') is the characteristic function of an absolutely continuous distribution symmetric about 0.


Uses

Because of the continuity theorem, characteristic functions are used in the most frequently seen proof of the central limit theorem. The main technique involved in making calculations with a characteristic function is recognizing the function as the characteristic function of a particular distribution.


Basic manipulations of distributions

Characteristic functions are particularly useful for dealing with linear functions of
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s * Independe ...
random variables. For example, if is a sequence of independent (and not necessarily identically distributed) random variables, and :S_n = \sum_^n a_i X_i,\,\! where the ''a''''i'' are constants, then the characteristic function for ''S''''n'' is given by :\varphi_(t)=\varphi_(a_1t)\varphi_(a_2t)\cdots \varphi_(a_nt) \,\! In particular, . To see this, write out the definition of characteristic function: : \varphi_(t)= \operatorname\left ^\right \operatorname\left ^e^\right= \operatorname\left ^\right\operatorname\left ^\right=\varphi_X(t) \varphi_Y(t) The independence of ''X'' and ''Y'' is required to establish the equality of the third and fourth expressions. Another special case of interest for identically distributed random variables is when and then ''Sn'' is the sample mean. In this case, writing for the mean, : \varphi_(t)= \varphi_X\!\left(\tfrac \right)^n


Moments

Characteristic functions can also be used to find moments of a random variable. Provided that the ''n''th moment exists, the characteristic function can be differentiated ''n'' times and : \left frac \varphi_X(t)\right = i^ \operatorname\left X^n\right \Rightarrow \operatorname\left X^n\right= i^\left frac\varphi_X(t)\right = i^\varphi_X^(0) ,\! For example, suppose ''X'' has a standard Cauchy distribution. Then . This is not differentiable at ''t'' = 0, showing that the Cauchy distribution has no expectation. Also, the characteristic function of the sample mean of ''n''
independent Independent or Independents may refer to: Arts, entertainment, and media Artist groups * Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s * Independe ...
observations has characteristic function , using the result from the previous section. This is the characteristic function of the standard Cauchy distribution: thus, the sample mean has the same distribution as the population itself. As a further example, suppose ''X'' follows a
Gaussian distribution In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is : f(x) = \frac e^ The parameter \mu ...
i.e. X \sim \mathcal(\mu,\sigma^2). Then \varphi_(t) = e^ and :\operatorname\left X\right= i^ \left frac\varphi_X(t)\right = i^ \left i \mu - \sigma^2 t) \varphi_X(t) \right = \mu A similar calculation shows \operatorname\left X^2\right= \mu^2 + \sigma^2 and is easier to carry out than applying the definition of expectation and using integration by parts to evaluate \operatorname\left X^2\right. The logarithm of a characteristic function is a
cumulant generating function In probability theory and statistics, the cumulants of a probability distribution are a set of quantities that provide an alternative to the '' moments'' of the distribution. Any two probability distributions whose moments are identical will have ...
, which is useful for finding
cumulant In probability theory and statistics, the cumulants of a probability distribution are a set of quantities that provide an alternative to the '' moments'' of the distribution. Any two probability distributions whose moments are identical will have ...
s; some instead define the cumulant generating function as the logarithm of the moment-generating function, and call the logarithm of the characteristic function the ''second'' cumulant generating function.


Data analysis

Characteristic functions can be used as part of procedures for fitting probability distributions to samples of data. Cases where this provides a practicable option compared to other possibilities include fitting the
stable distribution In probability theory, a distribution is said to be stable if a linear combination of two independent random variables with this distribution has the same distribution, up to location and scale parameters. A random variable is said to be sta ...
since closed form expressions for the density are not available which makes implementation of
maximum likelihood In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed stat ...
estimation difficult. Estimation procedures are available which match the theoretical characteristic function to the
empirical characteristic function Let (X_1,...,X_n) be independent, identically distributed real-valued random variables with common characteristic function \varphi(t). The empirical characteristic function (ECF) defined as : \varphi_(t)= \frac \sum_^ e^, \ =\sqrt, is an unbia ...
, calculated from the data. Paulson et al. (1975) and Heathcote (1977) provide some theoretical background for such an estimation procedure. In addition, Yu (2004) describes applications of empirical characteristic functions to fit
time series In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Ex ...
models where likelihood procedures are impractical. Empirical characteristic functions have also been used by Ansari et al. (2020) and Li et al. (2020) for training
generative adversarial networks A generative adversarial network (GAN) is a class of machine learning frameworks designed by Ian Goodfellow and his colleagues in June 2014. Two neural networks contest with each other in the form of a zero-sum game, where one agent's gain is a ...
.


Example

The
gamma distribution In probability theory and statistics, the gamma distribution is a two- parameter family of continuous probability distributions. The exponential distribution, Erlang distribution, and chi-square distribution are special cases of the gamma di ...
with scale parameter θ and a shape parameter ''k'' has the characteristic function : (1 - \theta i t)^. Now suppose that we have : X ~\sim \Gamma(k_1,\theta) \mbox Y \sim \Gamma(k_2,\theta) with ''X'' and ''Y'' independent from each other, and we wish to know what the distribution of ''X'' + ''Y'' is. The characteristic functions are : \varphi_X(t)=(1 - \theta i t)^,\,\qquad \varphi_Y(t)=(1 - \theta it)^ which by independence and the basic properties of characteristic function leads to : \varphi_(t)=\varphi_X(t)\varphi_Y(t)=(1 - \theta i t)^(1 - \theta i t)^=\left(1 - \theta i t\right)^. This is the characteristic function of the gamma distribution scale parameter ''θ'' and shape parameter ''k''1 + ''k''2, and we therefore conclude : X+Y \sim \Gamma(k_1+k_2,\theta) The result can be expanded to ''n'' independent gamma distributed random variables with the same scale parameter and we get : \forall i \in \ : X_i \sim \Gamma(k_i,\theta) \qquad \Rightarrow \qquad \sum_^n X_i \sim \Gamma\left(\sum_^nk_i,\theta\right).


Entire characteristic functions

As defined above, the argument of the characteristic function is treated as a real number: however, certain aspects of the theory of characteristic functions are advanced by extending the definition into the complex plane by analytical continuation, in cases where this is possible.


Related concepts

Related concepts include the moment-generating function and the probability-generating function. The characteristic function exists for all probability distributions. This is not the case for the moment-generating function. The characteristic function is closely related to the Fourier transform: the characteristic function of a probability density function ''p''(''x'') is the complex conjugate of the
continuous Fourier transform A Fourier transform (FT) is a mathematical transform that decomposes functions into frequency components, which are represented by the output of the transform as a function of frequency. Most commonly functions of time or space are transformed, ...
of ''p''(''x'') (according to the usual convention; see continuous Fourier transform – other conventions). : \varphi_X(t) = \langle e^ \rangle = \int_ e^p(x)\, dx = \overline = \overline, where ''P''(''t'') denotes the
continuous Fourier transform A Fourier transform (FT) is a mathematical transform that decomposes functions into frequency components, which are represented by the output of the transform as a function of frequency. Most commonly functions of time or space are transformed, ...
of the probability density function ''p''(''x''). Likewise, ''p''(''x'') may be recovered from ''φX''(''t'') through the inverse Fourier transform: :p(x) = \frac \int_ e^ P(t)\, dt = \frac \int_ e^ \overline\, dt. Indeed, even when the random variable does not have a density, the characteristic function may be seen as the Fourier transform of the measure corresponding to the random variable. Another related concept is the representation of probability distributions as elements of a
reproducing kernel Hilbert space In functional analysis (a branch of mathematics), a reproducing kernel Hilbert space (RKHS) is a Hilbert space of functions in which point evaluation is a continuous linear functional. Roughly speaking, this means that if two functions f and g in ...
via the
kernel embedding of distributions In machine learning, the kernel embedding of distributions (also called the kernel mean or mean map) comprises a class of nonparametric methods in which a probability distribution is represented as an element of a reproducing kernel Hilbert spac ...
. This framework may be viewed as a generalization of the characteristic function under specific choices of the
kernel function In operator theory, a branch of mathematics, a positive-definite kernel is a generalization of a positive-definite function or a positive-definite matrix. It was first introduced by James Mercer in the early 20th century, in the context of solvi ...
.


See also

*
Subindependence In probability theory and statistics, subindependence is a weak form of independence. Two random variables ''X'' and ''Y'' are said to be subindependent if the characteristic function of their sum is equal to the product of their marginal characte ...
, a weaker condition than independence, that is defined in terms of characteristic functions. *
Cumulant In probability theory and statistics, the cumulants of a probability distribution are a set of quantities that provide an alternative to the '' moments'' of the distribution. Any two probability distributions whose moments are identical will have ...
, a term of the ''cumulant generating functions'', which are logs of the characteristic functions.


Notes


References


Citations


Sources

* * * * * * * * * * * * * * * * * * *


External links

* {{Theory of probability distributions Functions related to probability distributions