Taylor expansions for the moments of functions of random variables
   HOME

TheInfoList



OR:

In
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
, it is possible to approximate the moments of a function ''f'' of a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
''X'' using
Taylor expansion In mathematics, the Taylor series or Taylor expansion of a function is an infinite sum of terms that are expressed in terms of the function's derivatives at a single point. For most common functions, the function and the sum of its Taylor serie ...
s, provided that ''f'' is sufficiently differentiable and that the moments of ''X'' are finite.


First moment

Given \mu_X and \sigma^2_X, the mean and the variance of X, respectively,Haym Benaroya, Seon Mi Han, and Mark Nagurka. ''Probability Models in Engineering and Science''. CRC Press, 2005, p166. a Taylor expansion of the
expected value In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a l ...
of f(X) can be found via : \begin \operatorname\left (X)\right& = \operatorname\left \left(\mu_X + \left(X - \mu_X\right)\right)\right\\ & \approx \operatorname\left (\mu_X) + f'(\mu_X)\left(X-\mu_X\right) + \fracf''(\mu_X) \left(X - \mu_X\right)^2 \right\\ & = f(\mu_X) + f'(\mu_X) \operatorname \left X-\mu_X \right+ \fracf''(\mu_X) \operatorname \left \left(X - \mu_X\right)^2 \right \end Since E -\mu_X0, the second term vanishes. Also, E X-\mu_X)^2/math> is \sigma_X^2. Therefore, :\operatorname\left (X)\rightapprox f(\mu_X) +\frac\sigma_X^2. It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example, :\operatorname\left frac\rightapprox\frac -\frac+\frac\operatorname\left \right/math>


Second moment

Similarly, :\operatorname\left (X)\rightapprox \left(f'(\operatorname\left \right\right)^2\operatorname\left \right= \left(f'(\mu_X)\right)^2\sigma^2_X -\frac\left(f''(\mu_X)\right)^2\sigma_X^4 The above is obtained using a second order approximation, following the method used in estimating the first moment. It will be a poor approximation in cases where f(X) is highly non-linear. This is a special case of the
delta method In statistics, the delta method is a result concerning the approximate probability distribution for a function of an asymptotically normal statistical estimator from knowledge of the limiting variance of that estimator. History The delta method ...
. Indeed, we take \operatorname\left (X)\rightapprox f(\mu_X) +\frac\sigma_X^2. With f(X) = g(X)^2 , we get \operatorname\left ^2\right/math>. The variance is then computed using the formula \operatorname\left \right= \operatorname\left ^2\right- \mu_Y^2. An example is, :\operatorname\left frac\rightapprox\frac-\frac\operatorname\left ,Y\right\frac\operatorname\left \right The second order approximation, when X follows a normal distribution, is: :\operatorname\left (X)\rightapprox \left(f'(\operatorname\left \right\right)^2\operatorname\left \right+ \frac\left(\operatorname\left \rightright)^2 = \left(f'(\mu_X)\right)^2\sigma^2_X + \frac\left(f''(\mu_X)\right)^2\sigma_X^4 + \left(f'(\mu_X)\right)\left(f(\mu_X)\right)\sigma_X^4


First product moment

To find a second-order approximation for the covariance of functions of two random variables (with the same function applied to both), one can proceed as follows. First, note that \operatorname\left (X),f(Y)\right\operatorname\left (X)f(Y)\right\operatorname\left (X)\rightoperatorname\left (Y)\right/math>. Since a second-order expansion for \operatorname\left (X)\right/math> has already been derived above, it only remains to find \operatorname\left (X)f(Y)\right/math>. Treating f(X)f(Y) as a two-variable function, the second-order Taylor expansion is as follows: : \begin f(X)f(Y) & \approx f(\mu_X) f(\mu_Y) + (X-\mu_X) f'(\mu_X)f(\mu_Y) + (Y - \mu_Y)f(\mu_X)f'(\mu_Y) + \frac\left X-\mu_X)^2 f''(\mu_X)f(\mu_Y) + 2(X-\mu_X)(Y-\mu_Y)f'(\mu_X)f'(\mu_Y) + (Y-\mu_Y)^2 f(\mu_X)f''(\mu_Y) \right\end Taking expectation of the above and simplifying—making use of the identities \operatorname(X^2)=\operatorname(X)+\left operatorname(X)\right2 and \operatorname(XY)=\operatorname(X,Y)+\left operatorname(X)\rightleft operatorname(Y)\right/math>—leads to \operatorname\left (X)f(Y)\rightapprox f(\mu_X)f(\mu_Y)+f'(\mu_X)f'(\mu_Y)\operatorname(X,Y)+\fracf''(\mu_X)f(\mu_Y)\operatorname(X)+\fracf(\mu_X)f''(\mu_Y)\operatorname(Y). Hence, : \begin \operatorname\left (X),f(Y)\right& \approx f(\mu_X)f(\mu_Y)+f'(\mu_X)f'(\mu_Y)\operatorname(X,Y)+\fracf''(\mu_X)f(\mu_Y)\operatorname(X)+\fracf(\mu_X)f''(\mu_Y)\operatorname(Y) - \left (\mu_X)+\fracf''(\mu_X)\operatorname(X)\right\left (\mu_Y)+\fracf''(\mu_Y)\operatorname(Y) \right\\ & = f'(\mu_X)f'(\mu_Y) \operatorname(X,Y) - \fracf''(\mu_X)f''(\mu_Y)\operatorname(X)\operatorname(Y) \end


See also

*
Propagation of uncertainty In statistics, propagation of uncertainty (or propagation of error) is the effect of variables' uncertainties (or errors, more specifically random errors) on the uncertainty of a function based on them. When the variables are the values of exp ...
*
WKB approximation In mathematical physics, the WKB approximation or WKB method is a method for finding approximate solutions to linear differential equations with spatially varying coefficients. It is typically used for a semiclassical calculation in quantum mecha ...
*
Delta method In statistics, the delta method is a result concerning the approximate probability distribution for a function of an asymptotically normal statistical estimator from knowledge of the limiting variance of that estimator. History The delta method ...


Notes


Further reading

* {{DEFAULTSORT:Taylor Expansions For The Moments Of Functions Of Random Variables Statistical approximations Algebra of random variables Moment (mathematics)