Autocorrelation

Autocorrelation, sometimes known as serial correlation in the discrete time case, is the correlation of a signal with a delayed copy of itself as a function of delay. Informally, it is the similarity between observations of a random variable as a function of the time lag between them. The analysis of autocorrelation is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise, or identifying the missing fundamental frequency in a signal implied by its harmonic frequencies. It is often used in signal processing for analyzing functions or series of values, such as time domain signals.

Different fields of study define autocorrelation differently, and not all of these definitions are equivalent. In some fields, the term is used interchangeably with autocovariance.

Unit root processes, trend-stationary processes, autoregressive processes, and moving average processes are specific forms of processes with autocorrelation.

Auto-correlation of stochastic processes

In statistics, the autocorrelation of a real or complex random process is the Pearson correlation between values of the process at different times, as a function of the two times or of the time lag. Let $\left\$ be a random process, and $t$ be any point in time ($t$ may be an integer for a discrete-time process or a real number for a continuous-time process). Then $X_t$ is the value (or realization) produced by a given run of the process at time $t$. Suppose that the process has mean $\mu_t$ and variance $\sigma_t^2$ at time $t$, for each $t$. Then the definition of the auto-correlation function between times $t_1$ and $t_2$ isKun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018,

where $\operatorname$ is the expected value operator and the bar represents complex conjugation. Note that the expectation may not be well defined.

Subtracting the mean before multiplication yields the auto-covariance function between times $t_1$ and $t_2$:

Note that this expression is not well defined for all time series or processes, because the mean may not exist, or the variance may be zero (for a constant process) or infinite (for processes with distribution lacking well-behaved moments, such as certain types of power law).

Definition for wide-sense stationary stochastic process

If $\left\$ is a wide-sense stationary process then the mean $\mu$ and the variance $\sigma^2$ are time-independent, and further the autocovariance function depends only on the lag between $t_1$ and $t_2$: the autocovariance depends only on the time-distance between the pair of values but not on their position in time. This further implies that the autocovariance and auto-correlation can be expressed as a function of the time-lag, and that this would be an even function of the lag $\tau=t_2-t_1$. This gives the more familiar forms for the auto-correlation function

and the auto-covariance function:

In particular, note that

$\operatorname_(0) = \sigma^2 .$

Normalization

It is common practice in some disciplines (e.g. statistics and time series analysis) to normalize the autocovariance function to get a time-dependent Pearson correlation coefficient. However, in other disciplines (e.g. engineering) the normalization is usually dropped and the terms "autocorrelation" and "autocovariance" are used interchangeably.

The definition of the auto-correlation coefficient of a stochastic process is

$\rho_(t_1,t_2) = \frac = \frac .$

If the function $\rho_$ is well defined, its value must lie in the range 1,1/math>, with 1 indicating perfect correlation and −1 indicating perfect anti-correlation.

For a wide-sense stationary (WSS) process, the definition is

$\rho_(\tau) = \frac = \frac$.

The normalization is important both because the interpretation of the autocorrelation as a correlation provides a scale-free measure of the strength of statistical dependence, and because the normalization has an effect on the statistical properties of the estimated autocorrelations.

Properties

Symmetry property

The fact that the auto-correlation function $\operatorname_$ is an even function can be stated as
$\operatorname_(t_1,t_2) = \overline$
respectively for a WSS process:
$\operatorname_(\tau) = \overline .$

Maximum at zero

For a WSS process:
$\left, \operatorname_(\tau)\ \leq \operatorname_(0)$
Notice that $\operatorname_(0)$ is always real.

Cauchy–Schwarz inequality

The Cauchy–Schwarz inequality, inequality for stochastic processes:
\left, \operatorname_(t_1,t_2)\^2 \leq \operatorname\left X_, ^2\right\operatorname\left white noise signal will have a strong peak (represented by a Dirac delta function) at $\tau=0$ and will be exactly $0$ for all other $\tau$.

Wiener–Khinchin theorem

The Wiener–Khinchin theorem relates the autocorrelation function $\operatorname_$ to the power spectral density $S_$ via the Fourier transform:

$\operatorname_(\tau) = \int_^\infty S_(f) e^ \, f$

$S_(f) = \int_^\infty \operatorname_(\tau) e^ \, \tau .$

For real-valued functions, the symmetric autocorrelation function has a real symmetric transform, so the Wiener–Khinchin theorem can be re-expressed in terms of real cosines only:

$\operatorname_(\tau) = \int_^\infty S_(f) \cos(2 \pi f \tau) \, f$

$S_(f) = \int_^\infty \operatorname_(\tau) \cos(2 \pi f \tau) \, \tau .$

Auto-correlation of random vectors

The (potentially time-dependent) auto-correlation matrix (also called second moment) of a (potentially time-dependent) random vector $\mathbf = (X_1,\ldots,X_n)^$ is an $n \times n$ matrix containing as elements the autocorrelations of all pairs of elements of the random vector $\mathbf$. The autocorrelation matrix is used in various digital signal processing algorithms.

For a random vector $\mathbf = (X_1,\ldots,X_n)^$ containing random elements whose expected value and variance exist, the auto-correlation matrix is defined byPapoulis, Athanasius, ''Probability, Random variables and Stochastic processes'', McGraw-Hill, 1991

where $^$ denotes transposition and has dimensions $n \times n$.

Written component-wise:

\operatorname_ =
\begin
\operatorname _1 X_1& \operatorname_1 X_2& \cdots & \operatorname _1 X_n\\ \\
\operatorname _2 X_1& \operatorname_2 X_2& \cdots & \operatorname _2 X_n\\ \\
\vdots & \vdots & \ddots & \vdots \\ \\
\operatorname_n X_1& \operatorname