Taylor expansions for the moments of functions of random variables

In probability theory, it is possible to approximate the moments of a function ''f'' of a random variable ''X'' using Taylor expansions, provided that ''f'' is sufficiently differentiable and that the moments of ''X'' are finite.

A simulation-based alternative to this approximation is the application of Monte Carlo simulations.

First moment

Given $\mu_X$ and $\sigma^2_X$, the mean and the variance of $X$, respectively,Haym Benaroya, Seon Mi Han, and Mark Nagurka. ''Probability Models in Engineering and Science''. CRC Press, 2005, p166. a Taylor expansion of the expected value of $f(X)$ can be found via

:
\begin
\operatorname\left (X)\right& = \operatorname\left \left(\mu_X + \left(X - \mu_X\right)\right)\right\\
& \approx \operatorname\left (\mu_X) + f'(\mu_X)\left(X-\mu_X\right) + \fracf''(\mu_X) \left(X - \mu_X\right)^2 \right\\
& = f(\mu_X) + f'(\mu_X) \operatorname \left X-\mu_X \right+ \fracf''(\mu_X) \operatorname \left \left(X - \mu_X\right)^2 \right
\end

Since E -\mu_X0, the second term vanishes. Also, E X-\mu_X)^2/math> is $\sigma_X^2$. Therefore,

:\operatorname\left (X)\rightapprox f(\mu_X) +\frac\sigma_X^2.

It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions. For example,

:\operatorname\left frac\rightapprox\frac -\frac+\frac\operatorname\left \right/math>

Second moment

Similarly,

:\operatorname\left (X)\rightapprox \left(f'(\operatorname\left \right\right)^2\operatorname\left \right= \left(f'(\mu_X)\right)^2\sigma^2_X -\frac\left(f''(\mu_X)\right)^2\sigma_X^4

The above is obtained using a second order approximation, following the method used in estimating the first moment. It will be a poor approximation in cases where $f(X)$ is highly non-linear. This is a special case of the delta method.

Indeed, we take \operatorname\left (X)\rightapprox f(\mu_X) +\frac\sigma_X^2.

With $f(X) = g(X)^2$, we get \operatorname\left ^2\right/math>. The variance is then computed using the formula
\operatorname\left \right= \operatorname\left ^2\right- \mu_Y^2.

An example is,

:\operatorname\left frac\rightapprox\frac-\frac\operatorname\left ,Y\right\frac\operatorname\left \right

The second order approximation, when X follows a normal distribution, is:

:\operatorname\left (X)\rightapprox \left(f'(\operatorname\left \right\right)^2\operatorname\left \right+ \frac\left(\operatorname\left \rightright)^2 = \left(f'(\mu_X)\right)^2\sigma^2_X + \frac\left(f''(\mu_X)\right)^2\sigma_X^4 + \left(f'(\mu_X)\right)\left(f(\mu_X)\right)\sigma_X^4

First product moment

To find a second-order approximation for the covariance of functions of two random variables (with the same function applied to both), one can proceed as follows. First, note that \operatorname\left (X),f(Y)\right\operatorname\left (X)f(Y)\right\operatorname\left (X)\rightoperatorname\left (Y)\right/math>. Since a second-order expansion for \operatorname\left (X)\right/math> has already been derived above, it only remains to find \operatorname\left (X)f(Y)\right/math>. Treating $f(X)f(Y)$ as a two-variable function, the second-order Taylor expansion is as follows:

:
\begin
f(X)f(Y) & \approx f(\mu_X) f(\mu_Y) + (X-\mu_X) f'(\mu_X)f(\mu_Y) + (Y - \mu_Y)f(\mu_X)f'(\mu_Y) + \frac\left X-\mu_X)^2 f''(\mu_X)f(\mu_Y) + 2(X-\mu_X)(Y-\mu_Y)f'(\mu_X)f'(\mu_Y) + (Y-\mu_Y)^2 f(\mu_X)f''(\mu_Y) \right\end

Taking expectation of the above and simplifying—making use of the identities \operatorname(X^2)=\operatorname(X)+\left operatorname(X)\right2 and \operatorname(XY)=\operatorname(X,Y)+\left operatorname(X)\rightleft operatorname(Y)\right/math>—leads to \operatorname\left (X)f(Y)\rightapprox f(\mu_X)f(\mu_Y)+f'(\mu_X)f'(\mu_Y)\operatorname(X,Y)+\fracf''(\mu_X)f(\mu_Y)\operatorname(X)+\fracf(\mu_X)f''(\mu_Y)\operatorname(Y). Hence,

:
\begin
\operatorname\left (X),f(Y)\right& \approx f(\mu_X)f(\mu_Y)+f'(\mu_X)f'(\mu_Y)\operatorname(X,Y)+\fracf''(\mu_X)f(\mu_Y)\operatorname(X)+\fracf(\mu_X)f''(\mu_Y)\operatorname(Y) - \left (\mu_X)+\fracf''(\mu_X)\operatorname(X)\right\left (\mu_Y)+\fracf''(\mu_Y)\operatorname(Y) \right\\
& = f'(\mu_X)f'(\mu_Y) \operatorname(X,Y) - \fracf''(\mu_X)f''(\mu_Y)\operatorname(X)\operatorname(Y)
\end

Random vectors

If ''X'' is a random vector, the approximations for the mean and variance of $f(X)$ are given by
: $\begin \operatorname(f(X)) &= f(\mu_X) + \frac \operatorname(H_f(\mu_X) \Sigma_X) \\ \operatorname(f(X)) &= \nabla f(\mu_X)^t \Sigma_X \nabla f(\mu_X) + \frac \operatorname \left( H_f(\mu_X) \Sigma_X H_f(\mu_X) \Sigma_X \right). \end$
Here $\nabla f$ and $H_f$ denote the gradient and the Hessian matrix respectively, and $\Sigma_X$ is the covariance matrix of ''X''.

See also

*Propagation of uncertainty
*WKB approximation
*Delta method

Notes

Further reading

*

{{DEFAULTSORT:Taylor Expansions For The Moments Of Functions Of Random Variables
Statistical approximations
Algebra of random variables
Moments (mathematics)