Description
The goal of the Wiener filter is to compute a statistical estimate of an unknown signal using a related signal as an input and filtering that known signal to produce the estimate as an output. For example, the known signal might consist of an unknown signal of interest that has been corrupted by additiveWiener filter solutions
Let be an unknown signal which must be estimated from a measurement signal . Where alpha is a tunable parameter. is known as prediction, is known as filtering, and is known as smoothing (see Wiener filtering chapter of for more details). The Wiener filter problem has solutions for three possible cases: one where a noncausal filter is acceptable (requiring an infinite amount of both past and future data), the case where aNoncausal solution
: where are spectral densities. Provided that is optimal, then theCausal solution
: where * consists of the causal part of (that is, that part of this fraction having a positive time solution under the inverse Laplace transform) * is the causal component of (i.e., the inverse Laplace transform of is non-zero only for ) * is the anti-causal component of (i.e., the inverse Laplace transform of is non-zero only for ) This general formula is complicated and deserves a more detailed explanation. To write down the solution in a specific case, one should follow these steps: # Start with the spectrum in rational form and factor it into causal and anti-causal components: where contains all the zeros and poles in the left half plane (LHP) and contains the zeroes and poles in the right half plane (RHP). This is called the Wiener–Hopf factorization. # Divide by and write out the result as aFinite impulse response Wiener filter for discrete series
The causal_References
_Further_reading
*__External_links
*MathematicApplications
The Wiener filter has a variety of applications in signal processing, image processing, control systems, and digital communications. These applications generally fall into one of four main categories: * System identification *WienerFilter mage,2/code> on the first image on the right, produces the filtered image below it.
It is commonly used to denoise audio signals, especially speech, as a preprocessor before speech recognition
Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
.
History
The filter was proposed by Norbert Wiener
Norbert Wiener (November 26, 1894 – March 18, 1964) was an American mathematician and philosopher. He was a professor of mathematics at the Massachusetts Institute of Technology (MIT). A child prodigy, Wiener later became an early researcher i ...
during the 1940s and published in 1949. The discrete-time equivalent of Wiener's work was derived independently by Andrey Kolmogorov
Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
and published in 1941.Kolmogorov A.N: 'Stationary sequences in Hilbert space', (In Russian) Bull. Moscow Univ. 1941 vol.2 no.6 1-40. English translation in Kailath T. (ed.) ''Linear least squares estimation'' Dowden, Hutchinson & Ross 1977 Hence the theory is often called the ''Wiener–Kolmogorov'' filtering theory (''cf.'' Kriging
In statistics, originally in geostatistics, kriging or Kriging, also known as Gaussian process regression, is a method of interpolation based on Gaussian process governed by prior covariances. Under suitable assumptions of the prior, kriging giv ...
). The Wiener filter was the first statistically designed filter to be proposed and subsequently gave rise to many others including the Kalman filter
For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
.
See also
* Wiener deconvolution
* least mean squares filter
Least mean squares (LMS) algorithms are a class of adaptive filter used to mimic a desired filter by finding the filter coefficients that relate to producing the least mean square of the error signal (difference between the desired and the actual ...
* similarities between Wiener and LMS
The Least mean squares filter solution converges to the Wiener filter solution, assuming that the unknown system is LTI and the noise is stationary. Both filters can be used to identify the impulse response of an unknown system, knowing only the ...
* linear prediction
Linear prediction is a mathematical operation where future values of a discrete-time signal are estimated as a linear function of previous samples.
In digital signal processing, linear prediction is often called linear predictive coding (LPC) and ...
* MMSE estimator
In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
* Kalman filter
For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
* generalized Wiener filter
The Wiener filter as originally proposed by Norbert Wiener is a signal processing filter which uses knowledge of the statistical properties of both the signal and the noise to reconstruct an optimal estimate of the signal from a noisy one-dimensio ...
* matched filter
In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
* Information field theory Information field theory (IFT) is a Bayesian statistical field theory relating to signal reconstruction, cosmography, and other related areas. IFT summarizes the information available on a physical field using Bayesian probabilities. It uses comput ...
References
Further reading
* Thomas Kailath
Thomas Kailath (born June 7, 1935) is an electrical engineer, information theorist, control engineer, entrepreneur and the Hitachi America Professor of Engineering, Emeritus, at Stanford University. Professor Kailath has authored several books, i ...
, Ali H. Sayed Ali H. Sayed (born Sao Paulo, Brazil, to parents of Lebanese people, Lebanese descent) is the dean of engineering at École Polytechnique Fédérale de Lausanne, EPFL (École polytechnique fédérale de Lausanne), where he teaches and conducts resea ...
, and Babak Hassibi
Babak Hassibi ( fa, بابک حسیبی, born in Tehran, Iran) is an Iranian-American electrical engineer, computer scientist, and applied mathematician who is the inaugural Mose and Lillian S. Bohn Professor of Electrical Engineering and Compu ...
, Linear Estimation, Prentice-Hall, NJ, 2000, .
External links
*Mathematic
WienerFilter
function
{{DEFAULTSORT:Wiener Filter
Linear filters
Image noise reduction techniques
Signal estimation>e ^2 \right ">^2 \right /math> =E \left [n^*_\right_.html" ;"title=".html" ;"title="[n">[n^* \right ">.html" ;"title="[n">[n^* \right /math>. This involves computing partial derivatives with respect to both the real and imaginary parts of a_i , and requiring them both to be zero.
The resulting Wiener-Hopf equations are:
:\sum_^N R_w -ia_j^* = R_ \qquad i = 0,\cdots, N.
which can be rewritten in matrix form:
:\underbrace_ \underbrace_ = \underbrace_
Note here that:\begin
R_w[-k] &= R_w^* \\
R_ &= R_^*[-k]
\end
The Wiener coefficient vector is then computed as:\mathbf = ^*
Applications
The Wiener filter has a variety of applications in signal processing, image processing, control systems, and digital communications. These applications generally fall into one of four main categories:
* System identification
* Deconvolution
In mathematics, deconvolution is the operation inverse to convolution. Both operations are used in signal processing and image processing. For example, it may be possible to recover the original signal after a filter (convolution) by using a deco ...
* Noise reduction
Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree. Noise rejection is the ability of a circuit to isolate an und ...
* Signal detection
Detection theory or signal detection theory is a means to measure the ability to differentiate between information-bearing patterns (called Stimulus (psychology), stimulus in living organisms, Signal (electronics), signal in machines) and random pa ...
For example, the Wiener filter can be used in image processing to remove noise from a picture. For example, using the Mathematica function:
WienerFilter mage,2/code> on the first image on the right, produces the filtered image below it.
It is commonly used to denoise audio signals, especially speech, as a preprocessor before speech recognition
Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
.
History
The filter was proposed by Norbert Wiener
Norbert Wiener (November 26, 1894 – March 18, 1964) was an American mathematician and philosopher. He was a professor of mathematics at the Massachusetts Institute of Technology (MIT). A child prodigy, Wiener later became an early researcher i ...
during the 1940s and published in 1949. The discrete-time equivalent of Wiener's work was derived independently by Andrey Kolmogorov
Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
and published in 1941.Kolmogorov A.N: 'Stationary sequences in Hilbert space', (In Russian) Bull. Moscow Univ. 1941 vol.2 no.6 1-40. English translation in Kailath T. (ed.) ''Linear least squares estimation'' Dowden, Hutchinson & Ross 1977 Hence the theory is often called the ''Wiener–Kolmogorov'' filtering theory (''cf.'' Kriging
In statistics, originally in geostatistics, kriging or Kriging, also known as Gaussian process regression, is a method of interpolation based on Gaussian process governed by prior covariances. Under suitable assumptions of the prior, kriging giv ...
). The Wiener filter was the first statistically designed filter to be proposed and subsequently gave rise to many others including the Kalman filter
For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
.
See also
* Wiener deconvolution
* least mean squares filter
Least mean squares (LMS) algorithms are a class of adaptive filter used to mimic a desired filter by finding the filter coefficients that relate to producing the least mean square of the error signal (difference between the desired and the actual ...
* similarities between Wiener and LMS
The Least mean squares filter solution converges to the Wiener filter solution, assuming that the unknown system is LTI and the noise is stationary. Both filters can be used to identify the impulse response of an unknown system, knowing only the ...
* linear prediction
Linear prediction is a mathematical operation where future values of a discrete-time signal are estimated as a linear function of previous samples.
In digital signal processing, linear prediction is often called linear predictive coding (LPC) and ...
* MMSE estimator
In statistics and signal processing, a minimum mean square error (MMSE) estimator is an estimation method which minimizes the mean square error (MSE), which is a common measure of estimator quality, of the fitted values of a dependent variable. In ...
* Kalman filter
For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
* generalized Wiener filter
The Wiener filter as originally proposed by Norbert Wiener is a signal processing filter which uses knowledge of the statistical properties of both the signal and the noise to reconstruct an optimal estimate of the signal from a noisy one-dimensio ...
* matched filter
In signal processing, a matched filter is obtained by correlating a known delayed signal, or ''template'', with an unknown signal to detect the presence of the template in the unknown signal. This is equivalent to convolving the unknown signal wi ...
* Information field theory Information field theory (IFT) is a Bayesian statistical field theory relating to signal reconstruction, cosmography, and other related areas. IFT summarizes the information available on a physical field using Bayesian probabilities. It uses comput ...
References
Further reading
* Thomas Kailath
Thomas Kailath (born June 7, 1935) is an electrical engineer, information theorist, control engineer, entrepreneur and the Hitachi America Professor of Engineering, Emeritus, at Stanford University. Professor Kailath has authored several books, i ...
, Ali H. Sayed Ali H. Sayed (born Sao Paulo, Brazil, to parents of Lebanese people, Lebanese descent) is the dean of engineering at École Polytechnique Fédérale de Lausanne, EPFL (École polytechnique fédérale de Lausanne), where he teaches and conducts resea ...
, and Babak Hassibi
Babak Hassibi ( fa, بابک حسیبی, born in Tehran, Iran) is an Iranian-American electrical engineer, computer scientist, and applied mathematician who is the inaugural Mose and Lillian S. Bohn Professor of Electrical Engineering and Compu ...
, Linear Estimation, Prentice-Hall, NJ, 2000, .
External links
*Mathematic
WienerFilter
function
{{DEFAULTSORT:Wiener Filter
Linear filters
Image noise reduction techniques
Signal estimation