In
information theory, an entropy coding (or entropy encoding) is any
lossless data compression
Lossless compression is a class of data compression that allows the original data to be perfectly reconstructed from the compressed data with no loss of information. Lossless compression is possible because most real-world data exhibits statistic ...
method that attempts to approach the lower bound declared by
Shannon's source coding theorem
In information theory, Shannon's source coding theorem (or noiseless coding theorem) establishes the limits to possible data compression, and the operational meaning of the Shannon entropy.
Named after Claude Shannon, the source coding theorem ...
, which states that any lossless data compression method must have expected code length greater or equal to the entropy of the source.
More precisely, the source coding theorem states that for any source distribution, the expected code length satisfies