In
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, Whittle likelihood is an approximation to the
likelihood function
The likelihood function (often simply called the likelihood) represents the probability of random variable realizations conditional on particular values of the statistical parameters. Thus, when evaluated on a given sample, the likelihood funct ...
of a stationary Gaussian
time series
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Exa ...
. It is named after the mathematician and statistician
Peter Whittle, who introduced it in his PhD thesis in 1951.
It is commonly used in
time series analysis
In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Exa ...
and
signal processing
Signal processing is an electrical engineering subfield that focuses on analyzing, modifying and synthesizing ''signals'', such as audio signal processing, sound, image processing, images, and scientific measurements. Signal processing techniq ...
for parameter estimation and signal detection.
Context
In a
stationary Gaussian time series model, the
likelihood function
The likelihood function (often simply called the likelihood) represents the probability of random variable realizations conditional on particular values of the statistical parameters. Thus, when evaluated on a given sample, the likelihood funct ...
is (as usual in Gaussian models) a function of the associated mean and covariance parameters. With a large number (
) of observations, the (
) covariance matrix may become very large, making computations very costly in practice. However, due to stationarity, the covariance matrix has a rather simple structure, and by using an approximation, computations may be simplified considerably (from
to
). The idea effectively boils down to assuming a
heteroscedastic
In statistics, a sequence (or a vector) of random variables is homoscedastic () if all its random variables have the same finite variance. This is also known as homogeneity of variance. The complementary notion is called heteroscedasticity. The s ...
zero-mean Gaussian model in
Fourier domain
In physics, electronics, control systems engineering, and statistics, the frequency domain refers to the analysis of mathematical functions or signals with respect to frequency, rather than time. Put simply, a time-domain graph shows how a signa ...
; the model formulation is based on the time series'
discrete Fourier transform
In mathematics, the discrete Fourier transform (DFT) converts a finite sequence of equally-spaced samples of a function into a same-length sequence of equally-spaced samples of the discrete-time Fourier transform (DTFT), which is a complex- ...
and its
power spectral density
The power spectrum S_(f) of a time series x(t) describes the distribution of power into frequency components composing that signal. According to Fourier analysis, any physical signal can be decomposed into a number of discrete frequencies, o ...
.
[ ]
See also:
Definition
Let
be a stationary Gaussian time series with (''one-sided'') power spectral density
, where
is even and samples are taken at constant sampling intervals
.
Let
be the (complex-valued)
discrete Fourier transform
In mathematics, the discrete Fourier transform (DFT) converts a finite sequence of equally-spaced samples of a function into a same-length sequence of equally-spaced samples of the discrete-time Fourier transform (DTFT), which is a complex- ...
(DFT) of the time series. Then for the Whittle likelihood one effectively assumes independent zero-mean
Gaussian distributions for all
with variances for the real and imaginary parts given by
:
where
is the
th Fourier frequency. This approximate model immediately leads to the (logarithmic) likelihood function
:
where
denotes the absolute value with
.
Special case of a known noise spectrum
In case the noise spectrum is assumed a-priori ''known'', and noise properties are not to be inferred from the data, the likelihood function may be simplified further by ignoring constant terms, leading to the sum-of-squares expression
:
This expression also is the basis for the common
matched filter
In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
.
Accuracy of approximation
The Whittle likelihood in general is only an approximation, it is only exact if the spectrum is constant, i.e., in the trivial case of
white noise
In signal processing, white noise is a random signal having equal intensity at different frequencies, giving it a constant power spectral density. The term is used, with this or similar meanings, in many scientific and technical disciplines, ...
.
The
efficiency
Efficiency is the often measurable ability to avoid wasting materials, energy, efforts, money, and time in doing something or in producing a desired result. In a more general sense, it is the ability to do things well, successfully, and without ...
of the Whittle approximation always depends on the particular circumstances.
Note that due to
linearity
Linearity is the property of a mathematical relationship (''function'') that can be graphically represented as a straight line. Linearity is closely related to '' proportionality''. Examples in physics include rectilinear motion, the linear r ...
of the Fourier transform, Gaussianity in Fourier domain implies Gaussianity in time domain and vice versa. What makes the Whittle likelihood only approximately accurate is related to the
sampling theorem
Sampling may refer to:
*Sampling (signal processing), converting a continuous signal into a discrete signal
* Sampling (graphics), converting continuous colors into discrete color components
*Sampling (music), the reuse of a sound recording in ano ...
—the effect of Fourier-transforming only a ''finite'' number of data points, which also manifests itself as
spectral leakage
The Fourier transform of a function of time, s(t), is a complex-valued function of frequency, S(f), often referred to as a frequency spectrum. Any linear time-invariant operation on s(t) produces a new spectrum of the form H(f)•S(f), which chang ...
in related problems (and which may be ameliorated using the same methods, namely,
windowing). In the present case, the implicit periodicity assumption implies correlation between the first and last samples (
and
), which are effectively treated as "neighbouring" samples (like
and
).
Applications
Parameter estimation
Whittle's likelihood is commonly used to estimate signal parameters for signals that are buried in non-white noise. The
noise spectrum
In audio engineering, electronics, physics, and many other fields, the color of noise or noise spectrum refers to the power spectrum of a noise signal (a signal produced by a stochastic process). Different colors of noise have significantly ...
then may be assumed known,
or it may be inferred along with the signal parameters.
Signal detection
Signal detection is commonly performed with the
matched filter
In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
, which is based on the Whittle likelihood for the case of a ''known'' noise power spectral density.
The matched filter effectively does a
maximum-likelihood
In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed stati ...
fit of the signal to the noisy data and uses the resulting
likelihood ratio
The likelihood function (often simply called the likelihood) represents the probability of random variable realizations conditional on particular values of the statistical parameters. Thus, when evaluated on a given sample, the likelihood functi ...
as the detection statistic.
The matched filter may be generalized to an analogous procedure based on a
Student-t distribution
In probability and statistics, Student's ''t''-distribution (or simply the ''t''-distribution) is any member of a family of continuous probability distributions that arise when estimating the mean of a normally distributed population in sit ...
by also considering uncertainty (e.g.
estimation
Estimation (or estimating) is the process of finding an estimate or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is der ...
uncertainty) in the noise spectrum. On the technical side, this entails repeated or iterative matched-filtering.
Spectrum estimation
The Whittle likelihood is also applicable for estimation of the
noise spectrum
In audio engineering, electronics, physics, and many other fields, the color of noise or noise spectrum refers to the power spectrum of a noise signal (a signal produced by a stochastic process). Different colors of noise have significantly ...
, either alone or in conjunction with signal parameters.
See also
*
Coloured noise
*
Discrete Fourier transform
In mathematics, the discrete Fourier transform (DFT) converts a finite sequence of equally-spaced samples of a function into a same-length sequence of equally-spaced samples of the discrete-time Fourier transform (DTFT), which is a complex- ...
*
Likelihood function
The likelihood function (often simply called the likelihood) represents the probability of random variable realizations conditional on particular values of the statistical parameters. Thus, when evaluated on a given sample, the likelihood funct ...
*
Matched filter
In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
*
Power spectral density
The power spectrum S_(f) of a time series x(t) describes the distribution of power into frequency components composing that signal. According to Fourier analysis, any physical signal can be decomposed into a number of discrete frequencies, o ...
*
Statistical signal processing
Signal processing is an electrical engineering subfield that focuses on analyzing, modifying and synthesizing ''signals'', such as sound, images, and scientific measurements. Signal processing techniques are used to optimize transmissions, di ...
*
Weighted least squares
Weighted least squares (WLS), also known as weighted linear regression, is a generalization of ordinary least squares and linear regression in which knowledge of the variance of observations is incorporated into the regression.
WLS is also a speci ...
References
{{Statistics, analysis
Time series
Time series models
Frequency-domain analysis
Statistical inference
Statistical models
Statistical signal processing
Signal estimation
Normal distribution