information theory Information theory is the mathematical study of the quantification (science), quantification, Data storage, storage, and telecommunications, communication of information. The field was established and formalized by Claude Shannon in the 1940s, ...

, the Cheung–Marks theorem, named after K. F. Cheung and Robert J. Marks II, specifies conditions where restoration of a

signal A signal is both the process and the result of transmission of data over some media accomplished by embedding some variation. Signals are important in multiple subject fields including signal processing, information theory and biology. In ...

by the sampling theorem can become ill-posed. It offers conditions whereby "reconstruction error with unbounded

variance In probability theory and statistics, variance is the expected value of the squared deviation from the mean of a random variable. The standard deviation (SD) is obtained as the square root of the variance. Variance is a measure of dispersion ...

esultswhen a bounded variance noise is added to the samples."

Background

In the sampling theorem, the uncertainty of the interpolation as measured by noise variance is the same as the uncertainty of the sample data when the noise is i.i.d. In his classic 1948 paper founding

Claude Shannon Claude Elwood Shannon (April 30, 1916 – February 24, 2001) was an American mathematician, electrical engineer, computer scientist, cryptographer and inventor known as the "father of information theory" and the man who laid the foundations of th ...

offered the following generalization of the sampling theorem: Also Although true in the absence of noise, many of the expansions proposed by Shannon become ill-posed. An arbitrarily small amount of noise on the data renders restoration unstable. Such sampling expansions are not useful in practice since sampling noise, such as quantization noise, rules out stable interpolation and therefore any practical use.

Example

Shannon's suggestion of simultaneous sampling of the signal and its derivative at half the Nyquist rate results in well behaved interpolation. The Cheung–Marks theorem shows counter-intuitively that interlacing signal and derivative samples makes the restoration problem ill-posed. The theorem also shows sensitivity increases with derivative order.

The theorem

Generally, the Cheung–Marks theorem shows the sampling theorem becomes ill-posed when the area (

integral In mathematics, an integral is the continuous analog of a Summation, sum, which is used to calculate area, areas, volume, volumes, and their generalizations. Integration, the process of computing an integral, is one of the two fundamental oper ...

) of the squared magnitude of the interpolation function over all time is not finite. "While the generalized sampling concept is relatively straightforward, the reconstruction is not always feasible because of potential instabilities."

References

{{DEFAULTSORT:Cheung-Marks theorem Information theory Digital signal processing Mathematical theorems in theoretical computer science