Autocovariance Matrix

In probability theory and statistics, given a stochastic process, the autocovariance is a function that gives the covariance of the process with itself at pairs of time points. Autocovariance is closely related to the autocorrelation of the process in question.

Auto-covariance of stochastic processes

Definition

With the usual notation $\operatorname$ for the expectation operator, if the stochastic process $\left\$ has the mean function \mu_t = \operatorname _t/math>, then the autocovariance is given by

where $t_1$ and $t_2$ are two moments in time.

Definition for weakly stationary process

If $\left\$ is a weakly stationary (WSS) process, then the following are true:

:$\mu_ = \mu_ \triangleq \mu$ for all $t_1,t_2$

and

:\operatorname X__-_\mu_)(X__-_\mu_)=_\operatorname __X_t-_\mu^2_.

__Normalization_

It_is_common_practice_in_some_disciplines_(e.g._statistics_and_time_series_analysis)_to_normalize_the_autocovariance_function_to_get_a_time-dependent_Pearson_correlation_coefficient._However_in_other_disciplines_(e.g._engineering)_the_normalization_is_usually_dropped_and_the_terms_"autocorrelation"_and_"autocovariance"_are_used_interchangeably.

The_definition_of_the_normalized_auto-correlation_of_a_stochastic_process_is

:$\rho_(t_1,t_2)_=_\frac_=_\frac$.

If_the_function_$\rho_$_is_well-defined,_its_value_must_lie_in_the_range_ 1,1/math>,_with_1_indicating_perfect_correlation_and_−1_indicating_perfect_anti-correlation.

For_a_WSS_process,_the_definition_is

:$\rho_(\tau)_=_\frac_=_\frac$.

where

:$\operatorname_(0)_=_\sigma^2$.

_Properties

_Symmetry_property

:$\operatorname_(t_1,t_2)_=_\overline$Kun_Il_Park,_Fundamentals_of_Probability_and_Stochastic_Processes_with_Applications_to_Communications,_Springer,_2018,_978-3-319-68074-3
respectively_for_a_WSS_process:
:$\operatorname_(\tau)_=_\overline$

_Linear_filtering

The_autocovariance_of_a_linearly_filtered_process_$\left\$
:$Y_t_=_\sum_^\infty_a_k_X_\,$
is
:$K_(\tau)_=_\sum_^\infty_a_k_a_l_K_(\tau+k-l).\,$

__Calculating_turbulent_diffusivity

Autocovariance_can_be_used_to_calculate_turbulent_diffusivity._Turbulence_in_a_flow_can_cause_the_fluctuation_of_velocity_in_space_and_time._Thus,_we_are_able_to_identify_turbulence_through_the_statistics_of_those_fluctuations.

Reynolds_decomposition_is_used_to_define_the_velocity_fluctuations_$u'(x,t)$_(assume_we_are_now_working_with_1D_problem_and__$U(x,t)$_is_the_velocity_along_$x$_direction):

:$U(x,t)_=_\langle_U(x,t)_\rangle_+_u'(x,t),$

where_$U(x,t)$_is_the_true_velocity,_and_$\langle_U(x,t)_\rangle$_is_the_ expected_value_of_velocity._If_we_choose_a_correct_$\langle_U(x,t)_\rangle$,_all_of_the_stochastic_components_of_the_turbulent_velocity_will_be_included_in_$u'(x,t)$._To_determine_$\langle_U(x,t)_\rangle$,_a_set_of_velocity_measurements_that_are_assembled_from_points_in_space,_moments_in_time_or_repeated_experiments_is_required.

If_we_assume_the_turbulent_flux_$\langle_u'c'_\rangle$_($c'_=_c_-_\langle_c_\rangle$,_and_''c''_is_the_concentration_term)_can_be_caused_by_a_random_walk,_we_can_use_Fick's_laws_of_diffusion_to_express_the_turbulent_flux_term:

:$J__=_\langle_u'c'_\rangle_\approx_D__\frac.$

The_velocity_autocovariance_is_defined_as

:$K__\equiv_\langle_u'(t_0)_u'(t_0_+_\tau)\rangle$_or_$K__\equiv_\langle_u'(x_0)_u'(x_0_+_r)\rangle,$

where_$\tau$_is_the_lag_time,_and_$r$_is_the_lag_distance.

The_turbulent_diffusivity_$D_$_can_be_calculated_using_the_following_3_methods:

__Auto-covariance_of_random_vectors_

__See_also_

*_Autoregressive_process
*_Correlation
*_Cross-covariance
*_Cross-correlation
*_ Noise_covariance_estimation_(as_an_application_example)

__References_

__Further_reading_

*_{{cite_book_, first=P._G._, last=Hoel_, title=Mathematical_Statistics_, publisher=Wiley_, location=New_York_, year=1984_, edition=Fifth_, isbn=978-0-471-89045-4_

Lecture_notes_on_autocovariance_from_WHOI

Fourier_analysis
Autocorrelationhtml" ;"title="X_t, ^2] < \infty for all $t$

and

:$\operatorname_(t_1,t_2) = \operatorname_(t_2 - t_1,0) \triangleq \operatorname_(t_2 - t_1) = \operatorname_(\tau),$

where $\tau = t_2 - t_1$ is the lag time, or the amount of time by which the signal has been shifted.

The autocovariance function of a WSS process is therefore given by:

which is equivalent to

:\operatorname_(\tau) = \operatorname X_ - \mu_)(X_ - \mu_)= \operatorname _ X_t- \mu^2 .

Normalization

It is common practice in some disciplines (e.g. statistics and time series analysis) to normalize the autocovariance function to get a time-dependent Pearson correlation coefficient. However in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "autocorrelation" and "autocovariance" are used interchangeably.

The definition of the normalized auto-correlation of a stochastic process is

:$\rho_(t_1,t_2) = \frac = \frac$.

If the function $\rho_$ is well-defined, its value must lie in the range 1,1/math>, with 1 indicating perfect correlation and −1 indicating perfect anti-correlation.

For a WSS process, the definition is

:$\rho_(\tau) = \frac = \frac$.

where

:$\operatorname_(0) = \sigma^2$.

Properties

Symmetry property

:$\operatorname_(t_1,t_2) = \overline$Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3
respectively for a WSS process:
:$\operatorname_(\tau) = \overline$

Linear filtering

The autocovariance of a linearly filtered process $\left\$
:$Y_t = \sum_^\infty a_k X_\,$
is
:$K_(\tau) = \sum_^\infty a_k a_l K_(\tau+k-l).\,$

Calculating turbulent diffusivity

Autocovariance can be used to calculate turbulent diffusivity. Turbulence in a flow can cause the fluctuation of velocity in space and time. Thus, we are able to identify turbulence through the statistics of those fluctuations.

Reynolds decomposition is used to define the velocity fluctuations $u'(x,t)$ (assume we are now working with 1D problem and $U(x,t)$ is the velocity along $x$ direction):

:$U(x,t) = \langle U(x,t) \rangle + u'(x,t),$

where $U(x,t)$ is the true velocity, and $\langle U(x,t) \rangle$ is the expected value of velocity. If we choose a correct $\langle U(x,t) \rangle$, all of the stochastic components of the turbulent velocity will be included in $u'(x,t)$. To determine $\langle U(x,t) \rangle$, a set of velocity measurements that are assembled from points in space, moments in time or repeated experiments is required.

If we assume the turbulent flux $\langle u'c' \rangle$ ($c' = c - \langle c \rangle$, and ''c'' is the concentration term) can be caused by a random walk, we can use Fick's laws of diffusion to express the turbulent flux term:

:$J_ = \langle u'c' \rangle \approx D_ \frac.$

The velocity autocovariance is defined as

:$K_ \equiv \langle u'(t_0) u'(t_0 + \tau)\rangle$ or $K_ \equiv \langle u'(x_0) u'(x_0 + r)\rangle,$

where $\tau$ is the lag time, and $r$ is the lag distance.

The turbulent diffusivity $D_$ can be calculated using the following 3 methods:

Auto-covariance of random vectors

See also

* Autoregressive process
* Correlation
* Cross-covariance
* Cross-correlation
* Noise covariance estimation (as an application example)

References

Further reading

* {{cite book , first=P. G. , last=Hoel , title=Mathematical Statistics , publisher=Wiley , location=New York , year=1984 , edition=Fifth , isbn=978-0-471-89045-4

Lecture notes on autocovariance from WHOI

Fourier analysis
Autocorrelation>X_t, ^2< \infty for all $t$

and

:$\operatorname_(t_1,t_2) = \operatorname_(t_2 - t_1,0) \triangleq \operatorname_(t_2 - t_1) = \operatorname_(\tau),$

where $\tau = t_2 - t_1$ is the lag time, or the amount of time by which the signal has been shifted.

The autocovariance function of a WSS process is therefore given by:

which is equivalent to

:\operatorname_(\tau) = \operatorname X_ - \mu_)(X_ - \mu_)= \operatorname _ X_t- \mu^2 .

Normalization

It is common practice in some disciplines (e.g. statistics and time series analysis) to normalize the autocovariance function to get a time-dependent Pearson correlation coefficient. However in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "autocorrelation" and "autocovariance" are used interchangeably.

The definition of the normalized auto-correlation of a stochastic process is

:$\rho_(t_1,t_2) = \frac = \frac$.

If the function $\rho_$ is well-defined, its value must lie in the range 1,1/math>, with 1 indicating perfect correlation and −1 indicating perfect anti-correlation.

For a WSS process, the definition is

:$\rho_(\tau) = \frac = \frac$.

where

:$\operatorname_(0) = \sigma^2$.

Properties

Symmetry property

:$\operatorname_(t_1,t_2) = \overline$Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3
respectively for a WSS process:
:$\operatorname_(\tau) = \overline$

Linear filtering

The autocovariance of a linearly filtered process $\left\$
:$Y_t = \sum_^\infty a_k X_\,$
is
:$K_(\tau) = \sum_^\infty a_k a_l K_(\tau+k-l).\,$

Calculating turbulent diffusivity

Autocovariance can be used to calculate turbulent diffusivity. Turbulence in a flow can cause the fluctuation of velocity in space and time. Thus, we are able to identify turbulence through the statistics of those fluctuations.

Reynolds decomposition is used to define the velocity fluctuations $u'(x,t)$ (assume we are now working with 1D problem and $U(x,t)$ is the velocity along $x$ direction):

:$U(x,t) = \langle U(x,t) \rangle + u'(x,t),$

where $U(x,t)$ is the true velocity, and $\langle U(x,t) \rangle$ is the expected value of velocity. If we choose a correct $\langle U(x,t) \rangle$, all of the stochastic components of the turbulent velocity will be included in $u'(x,t)$. To determine $\langle U(x,t) \rangle$, a set of velocity measurements that are assembled from points in space, moments in time or repeated experiments is required.

If we assume the turbulent flux $\langle u'c' \rangle$ ($c' = c - \langle c \rangle$, and ''c'' is the concentration term) can be caused by a random walk, we can use Fick's laws of diffusion to express the turbulent flux term:

:$J_ = \langle u'c' \rangle \approx D_ \frac.$

The velocity autocovariance is defined as

:$K_ \equiv \langle u'(t_0) u'(t_0 + \tau)\rangle$ or $K_ \equiv \langle u'(x_0) u'(x_0 + r)\rangle,$

where $\tau$ is the lag time, and $r$ is the lag distance.

The turbulent diffusivity $D_$ can be calculated using the following 3 methods:

Auto-covariance of random vectors

See also

* Autoregressive process
* Correlation
* Cross-covariance
* Cross-correlation
* Noise covariance estimation (as an application example)

References

Further reading

* {{cite book , first=P. G. , last=Hoel , title=Mathematical Statistics , publisher=Wiley , location=New York , year=1984 , edition=Fifth , isbn=978-0-471-89045-4

Lecture notes on autocovariance from WHOI

Fourier analysis
Autocorrelation