Coding theory is the study of the properties of

_{min}'') where
# n is the length of the codeword, in symbols,
# m is the number of source symbols that will be used for encoding at once,
# ''d_{min}'' is the minimum hamming distance for the code.
There are many types of linear block codes, such as
# ^{''r''} – 1, 2^{''r''} – 1 – ''r'', 3), and the 3,12,7binary and 1,6,5ternary Golay codes.
Another code property is the number of neighbors that a single codeword may have.
Again, consider pennies as an example. First we pack the pennies in a rectangular grid. Each penny will have 4 near neighbors (and 4 at the corners which are farther away). In a hexagon, each penny will have 6 near neighbors. When we increase the dimensions, the number of near neighbors increases very rapidly. The result is the number of ways for noise to make the receiver choose a neighbor (hence an error) grows as well. This is a fundamental limitation of block codes, and indeed all codes. It may be harder to cause an error to a single neighbor, but the number of neighbors can be large enough so the total error probability actually suffers.
Properties of linear block codes are used in many applications. For example, the syndrome-coset uniqueness property of linear block codes is used in trellis shaping, one of the best-known

Information Theory, Inference, and Learning Algorithms

' Cambridge: Cambridge University Press, 2003. * Vera Pless (1982), ''Introduction to the Theory of Error-Correcting Codes'', John Wiley & Sons, Inc., . * Randy Yates,

A Coding Theory Tutorial

'. {{Authority control Coding theory, Error detection and correction

code
In communication
Communication (from Latin
Latin (, or , ) is a classical language
A classical language is a language
A language is a structured system of communication
Communication (from Latin ''communicare'', mean ...

s and their respective fitness for specific applications. Codes are used for data compression
In signal processing
Signal processing is an electrical engineering
Electrical engineering is an engineering discipline concerned with the study, design, and application of equipment, devices, and systems which use electricity, electro ...

, cryptography
Cryptography, or cryptology (from grc, , translit=kryptós "hidden, secret"; and ''graphein'', "to write", or ''-logia
''-logy'' is a suffix in the English language, used with words originally adapted from Ancient Greek ending in (''- ...

, error detection and correction
In information theory
Information theory is the scientific study of the quantification, storage, and communication
Communication (from Latin ''communicare'', meaning "to share" or "to be in relation with") is "an apparent answer to ...

, data transmission
Data transmission and data reception or, more broadly, data communication or digital communications is the transfer and reception of data
Data (; ) are individual facts, statistics, or items of information, often numeric. In a more techn ...

and data storage
Data
Data (; ) are individual facts, statistics, or items of information, often numeric. In a more technical sense, data are a set of values of qualitative property, qualitative or quantity, quantitative variable (research), variables ...

. Codes are studied by various scientific disciplines—such as information theory
Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of Digital data, digital information. The field was fundamentally established by the ...

, electrical engineering
Electrical engineering is an engineering discipline concerned with the study, design, and application of equipment, devices, and systems which use electricity, electronics
The field of electronics is a branch of physics and electrical enginee ...

, mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and their changes (cal ...

, linguistics
Linguistics is the scientific study of language
A language is a structured system of communication
Communication (from Latin
Latin (, or , ) is a classical language belonging to the Italic languages, Italic branch of the Indo ...

, and computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , , and . Computer science ...

—for the purpose of designing efficient and reliable data transmission
Data transmission and data reception or, more broadly, data communication or digital communications is the transfer and reception of data
Data (; ) are individual facts, statistics, or items of information, often numeric. In a more techn ...

methods. This typically involves the removal of redundancy and the correction or detection of errors in the transmitted data.
There are four types of coding:
# Data compression
In signal processing
Signal processing is an electrical engineering
Electrical engineering is an engineering discipline concerned with the study, design, and application of equipment, devices, and systems which use electricity, electro ...

(or ''source coding'')
# Error control
In information theory
Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of Digital data, digital information. The field was fun ...

(or ''channel coding'')
#
# Line coding
In telecommunication
Telecommunication is the transmission of information by various types of technologies over wire
A wire is a single usually cylindrical
A cylinder (from Greek
Greek may refer to:
Greece
Anything of, from, or ...

Data compression attempts to remove redundancy from the data from a source in order to transmit it more efficiently. For example, ZIP data compression makes data files smaller, for purposes such as to reduce Internet traffic. Data compression and error correction may be studied in combination.
Error correction
In information theory
Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of Digital data, digital information. The field was fun ...

adds extra data bits to make the transmission of data more robust to disturbances present on the transmission channel. The ordinary user may not be aware of many applications using error correction. A typical music compact disc (CD) uses the Reed–Solomon code to correct for scratches and dust. In this application the transmission channel is the CD itself. Cell phones also use coding techniques to correct for the fading
In wireless communication
Wireless communication (or just wireless, when the context allows) is the transfer of information between two or more points that do not use an electrical conductor
In physics
Physics (from grc, φυσ ...

and noise of high frequency radio transmission. Data modems, telephone transmissions, and the NASA Deep Space Network
The NASA Deep Space Network (DSN) is a worldwide network
Network, networking and networked may refer to:
Science and technology
* Network theory, the study of graphs as a representation of relations between discrete objects
* Network science, ...

all employ channel coding techniques to get the bits through, for example the turbo code
In information theory, turbo codes (originally in French ''Turbocodes'') are a class of high-performance forward error correction (FEC) codes developed around 1990–91, but first published in 1993. They were the first practical codes to closely ...

and LDPC code
In information theory, a low-density parity-check (LDPC) code is a Linear code, linear error correcting code, a method of transmitting a message over a signal noise, noisy transmission channel. An LDPC is constructed using a sparse Tanner graph ...

s.
History of coding theory

In 1948,Claude Shannon
Claude Elwood Shannon (April 30, 1916 – February 24, 2001) was an American mathematician
A mathematician is someone who uses an extensive knowledge of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbe ...

published "A Mathematical Theory of Communication
"A Mathematical Theory of Communication" is an article by mathematician
A mathematician is someone who uses an extensive knowledge of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and nu ...

", an article in two parts in the July and October issues of the ''Bell System Technical Journal''. This work focuses on the problem of how best to encode the information
Information is processed, organised and structured data
Data (; ) are individual facts
A fact is something that is truth, true. The usual test for a statement of fact is verifiability—that is whether it can be demonstrated to c ...

a sender wants to transmit. In this fundamental work he used tools in probability theory, developed by Norbert Wiener
Norbert Wiener (November 26, 1894 – March 18, 1964) was an American mathematician
A mathematician is someone who uses an extensive knowledge of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers ( an ...

, which were in their nascent stages of being applied to communication theory at that time. Shannon developed information entropy
In information theory
Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of Digital data, digital information. The field was fun ...

as a measure for the uncertainty in a message while essentially inventing the field of information theory
Information theory is the scientific study of the quantification (science), quantification, computer data storage, storage, and telecommunication, communication of Digital data, digital information. The field was fundamentally established by the ...

.
The binary Golay code
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and t ...

was developed in 1949. It is an error-correcting code capable of correcting up to three errors in each 24-bit word, and detecting a fourth.
Richard Hamming
Richard Wesley Hamming (February 11, 1915 – January 7, 1998) was an American mathematician whose work had many implications for computer engineering
Computer engineering (CoE or CpE) is a branch of engineering
Engineering is t ...

won the Turing Award
The ACM A. M. Turing Award is an annual prize given by the Association for Computing Machinery
The Association for Computing Machinery (ACM) is a US-based international for . It was founded in 1947 and is the world's largest scientific and e ...

in 1968 for his work at Bell Labs
Nokia Bell Labs (formerly named Bell Labs Innovations (1996–2007), AT&T Bell Laboratories (1984–1996) and Bell Telephone Laboratories (1925–1984)) is an American industrial research and scientific development company
A company, ab ...

in numerical methods, automatic coding systems, and error-detecting and error-correcting codes. He invented the concepts known as Hamming code
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , ...

s, Hamming window
In signal processing
Signal processing is an electrical engineering
Electrical engineering is an engineering discipline concerned with the study, design, and application of equipment, devices, and systems which use electricity, electro ...

s, Hamming numbers
Regular numbers are numbers that evenly divide powers of 60 (or, equivalently, powers of 30). Equivalently, they are the numbers whose only prime divisors are 2, 3, and 5. As an example, 602 = 3600 = 48 × ...

, and Hamming distance
In information theory
Information theory is the scientific study of the quantification, storage, and communication
Communication (from Latin ''communicare'', meaning "to share" or "to be in relation with") is "an apparent answer to ...

.
In 1972, Nasir Ahmed proposed the discrete cosine transform
A discrete cosine transform (DCT) expresses a finite sequence of data points
In statistics
Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data
Data (; ) are ...

(DCT), which he developed with T. Natarajan and K. R. Rao
Kamisetty Ramamohan Rao was an Indian-American electrical engineer. He was a professor of Electrical Engineering at the University of Texas at Arlington (UT Arlington).
Academically known as K. R. Rao, he is credited with the co-invention of ...

in 1973. The DCT is the most widely used lossy compression
In information technology, lossy compression or irreversible compression is the class of data compression, data encoding methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to r ...

algorithm, the basis for multimedia formats such as JPEG
JPEG ( ) is a commonly used method of lossy compression
In information technology, lossy compression or irreversible compression is the class of data encoding methods that uses inexact approximations and partial data discarding to represe ...

, MPEG
The Moving Picture Experts Group (MPEG) is an alliance of working groups established jointly by International Organization for Standardization, ISO and International Electrotechnical Commission, IEC that sets standards for media coding, includ ...

and MP3
MP3 (formally MPEG-1 Audio Layer III or MPEG-2 Audio Layer III) is a coding format for digital audio
Digital audio is a representation of sound recorded in, or converted into, Digital signal (signal processing), digital form. In digital a ...

.
Source coding

The aim of source coding is to take the source data and make it smaller.Definition

Data can be seen as arandom variable
A random variable is a variable whose values depend on outcomes of a random
In common parlance, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no ...

$X:\backslash Omega\backslash to\backslash mathcal$, where $x\; \backslash in\; \backslash mathcal$ appears with probability $\backslash mathbb;\; href="/html/ALL/s/=x.html"\; ;"title="=x">=x$alphabet
An alphabet is a standardized set of basic written symbols
A symbol is a mark, sign, or word
In linguistics, a word of a spoken language can be defined as the smallest sequence of phonemes that can be uttered in isolation with semanti ...

$\backslash Sigma$.
A code is a function
:$C:\backslash mathcal\backslash to\backslash Sigma^*$ (or $\backslash Sigma^+$ if the empty string is not part of the alphabet).
$C(x)$ is the code word associated with $x$.
Length of the code word is written as
:$l(C(x)).$
Expected length of a code is
:$l(C)\; =\; \backslash sum\_l(C(x))\backslash mathbb;\; href="/html/ALL/s/=x.html"\; ;"title="=x">=x$
The concatenation of code words $C(x\_1,\; \backslash ldots,\; x\_k)\; =\; C(x\_1)C(x\_2)\; \backslash cdots\; C(x\_k)$.
The code word of the empty string is the empty string itself:
:$C(\backslash epsilon)\; =\; \backslash epsilon$
Properties

# $C:\backslash mathcal\backslash to\backslash Sigma^*$ isnon-singular
In the mathematical
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities a ...

if injective
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and ...

.
# $C:\backslash mathcal^*\backslash to\backslash Sigma^*$ is uniquely decodable if injective.
# $C:\backslash mathcal\backslash to\backslash Sigma^*$ is instantaneous
In physics
Physics is the that studies , its , its and behavior through , and the related entities of and . "Physical science is that department of knowledge which relates to the order of nature, or, in other words, to the regular succe ...

if $C(x\_1)$ is not a prefix of $C(x\_2)$ (and vice versa).
Principle

Entropy
Entropy is a scientific concept as well as a measurable physical property that is most commonly associated with a state of disorder, randomness, or uncertainty. The term and the concept are used in diverse fields, from classical thermodynamic ...

of a source is the measure of information. Basically, source codes try to reduce the redundancy present in the source, and represent the source with fewer bits that carry more information.
Data compression which explicitly tries to minimize the average length of messages according to a particular assumed probability model is called entropy encoding
In information theory, an entropy coding (or entropy encoding) is a lossless compression , lossless data compression scheme that is independent of the specific characteristics of the medium.
One of the main types of entropy coding creates and assi ...

.
Various techniques used by source coding schemes try to achieve the limit of entropy of the source. ''C''(''x'') ≥ ''H''(''x''), where ''H''(''x'') is entropy of source (bitrate), and ''C''(''x'') is the bitrate after compression. In particular, no source coding scheme can be better than the entropy of the source.
Example

Facsimile
A facsimile (from Latin
Latin (, or , ) is a classical language belonging to the Italic languages, Italic branch of the Indo-European languages. Latin was originally spoken in the area around Rome, known as Latium. Through the power of the ...

transmission uses a simple run length code. Source coding removes all data superfluous to the need of the transmitter, decreasing the bandwidth required for transmission.
Channel coding

The purpose of channel coding theory is to find codes which transmit quickly, contain many validcode word
In communication, a code word is an element of a standardized code or Communications protocol, protocol. Each code word is assembled in accordance with the specific rules of the code and assigned a unique meaning. Code words are typically used fo ...

s and can correct or at least detect many errors. While not mutually exclusive, performance in these areas is a trade off. So, different codes are optimal for different applications. The needed properties of this code mainly depend on the probability of errors happening during transmission. In a typical CD, the impairment is mainly dust or scratches.
CDs use cross-interleaved Reed–Solomon coding In the compact disc system, forward error correction#Interleaving, cross-interleaved Reed–Solomon error correction, Reed–Solomon code (CIRC) provides error detection and error correction. CIRC adds to every three data bytes one redundancy (infor ...

to spread the data out over the disk.
Although not a very good code, a simple repeat code can serve as an understandable example. Suppose we take a block of data bits (representing sound) and send it three times. At the receiver we will examine the three repetitions bit by bit and take a majority vote. The twist on this is that we don't merely send the bits in order. We interleave them. The block of data bits is first divided into 4 smaller blocks. Then we cycle through the block and send one bit from the first, then the second, etc. This is done three times to spread the data out over the surface of the disk. In the context of the simple repeat code, this may not appear effective. However, there are more powerful codes known which are very effective at correcting the "burst" error of a scratch or a dust spot when this interleaving technique is used.
Other codes are more appropriate for different applications. Deep space communications are limited by the thermal noise
A thermal column (or thermal) is a column of rising air
File:Atmosphere gas proportions.svg, Composition of Earth's atmosphere by volume, excluding water vapor. Lower pie represents trace gases that together compose about 0.043391% of the ...

of the receiver which is more of a continuous nature than a bursty nature. Likewise, narrowband modems are limited by the noise, present in the telephone network and also modeled better as a continuous disturbance. Cell phones are subject to rapid fading
In wireless communication
Wireless communication (or just wireless, when the context allows) is the transfer of information between two or more points that do not use an electrical conductor
In physics
Physics (from grc, φυσ ...

. The high frequencies used can cause rapid fading of the signal even if the receiver is moved a few inches. Again there are a class of channel codes that are designed to combat fading.
Linear codes

The term algebraic coding theory denotes the sub-field of coding theory where the properties of codes are expressed in algebraic terms and then further researched. Algebraic coding theory is basically divided into two major types of codes: * Linear block codes * Convolutional codes It analyzes the following three properties of a code – mainly: * Code word length * Total number of valid code words * The minimumdistance
Distance is a numerical measurement
Measurement is the quantification (science), quantification of variable and attribute (research), attributes of an object or event, which can be used to compare with other objects or events. The scope and ...

between two valid code words, using mainly the Hamming distance
In information theory
Information theory is the scientific study of the quantification, storage, and communication
Communication (from Latin ''communicare'', meaning "to share" or "to be in relation with") is "an apparent answer to ...

, sometimes also other distances like the Lee distance In coding theory
Coding theory is the study of the properties of codes and their respective fitness for specific applications. Codes are used for data compression, cryptography, error detection and correction, data transmission and data storage. ...

Linear block codes

Linear block codes have the property oflinearity
Linearity is the property of a mathematical relationship (''function (mathematics), function'') that can be graph of a function, graphically represented as a straight Line (geometry), line. Linearity is closely related to ''Proportionality (math ...

, i.e. the sum of any two codewords is also a code word, and they are applied to the source bits in blocks, hence the name linear block codes. There are block codes that are not linear, but it is difficult to prove that a code is a good one without this property.
Linear block codes are summarized by their symbol alphabets (e.g., binary or ternary) and parameters (''n'',''m'',''dCyclic code
In coding theory, a cyclic code is a block code, where the circular shifts of each codeword gives another word that belongs to the code. They are error-correcting codes that have algebraic properties that are convenient for efficient error detecti ...

s (e.g., Hamming code
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , ...

s)
# Repetition code
In coding theory
Coding theory is the study of the properties of s and their respective fitness for specific applications. Codes are used for , , , and . Codes are studied by various scientific disciplines—such as , , , , and —for the purp ...

s
# Parity codes
# Polynomial codeIn coding theory, a polynomial code is a type of linear code whose set of valid code words consists of those polynomials (usually of some fixed length) that are polynomial long division, divisible by a given fixed polynomial (of shorter length, calle ...

s (e.g., BCH code In coding theory
Coding theory is the study of the properties of codes and their respective fitness for specific applications. Codes are used for data compression, cryptography, error detection and correction, data transmission and data storage. ...

s)
# Reed–Solomon codes
# Algebraic geometric codes
# Reed–Muller code
Reed–Muller codes are error-correcting codes that are used in wireless communications applications, particularly in deep-space communication. Moreover, the proposed 5G, 5G standard relies on the closely related Polar code (coding theory), polar ...

s
# Perfect codes
Block codes are tied to the sphere packing
In geometry
Geometry (from the grc, γεωμετρία; ' "earth", ' "measurement") is, with , one of the oldest branches of . It is concerned with properties of space that are related with distance, shape, size, and relative position of ...

problem, which has received some attention over the years. In two dimensions, it is easy to visualize. Take a bunch of pennies flat on the table and push them together. The result is a hexagon pattern like a bee's nest. But block codes rely on more dimensions which cannot easily be visualized. The powerful (24,12) Golay code used in deep space communications uses 24 dimensions. If used as a binary code (which it usually is) the dimensions refer to the length of the codeword as defined above.
The theory of coding uses the ''N''-dimensional sphere model. For example, how many pennies can be packed into a circle on a tabletop, or in 3 dimensions, how many marbles can be packed into a globe. Other considerations enter the choice of a code. For example, hexagon packing into the constraint of a rectangular box will leave empty space at the corners. As the dimensions get larger, the percentage of empty space grows smaller. But at certain dimensions, the packing uses all the space and these codes are the so-called "perfect" codes. The only nontrivial and useful perfect codes are the distance-3 Hamming codes with parameters satisfying (2shaping codes
In digital communications shaping codes are a method of encoding that changes the distribution of signals to improve efficiency.
Description
Typical digital communication systems uses M-Quadrature Amplitude Modulation(QAM) to communicate throug ...

.
Convolutional codes

The idea behind a convolutional code is to make every codeword symbol be the weighted sum of the various input message symbols. This is likeconvolution
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities a ...

used in LTI systems to find the output of a system, when you know the input and impulse response.
So we generally find the output of the system convolutional encoder, which is the convolution of the input bit, against the states of the convolution encoder, registers.
Fundamentally, convolutional codes do not offer more protection against noise than an equivalent block code. In many cases, they generally offer greater simplicity of implementation over a block code of equal power. The encoder is usually a simple circuit which has state memory and some feedback logic, normally XOR gate
XOR gate (sometimes EOR, or EXOR and pronounced as Exclusive OR) is a digital logic gate that gives a true (1 or HIGH) output when the number of true inputs is odd. An XOR gate implements an exclusive disjunction, exclusive or (\nleftrightarro ...

s. The decoder
Decoder may refer to:
Technology
* Audio decoder converts digital audio to analog form
* Binary decoder, digital circuits such as 1-of-N and seven-segment decoders
* codec, Decompress (compression decoder), converts compressed data (e.g., audio/v ...

can be implemented in software or firmware.
The Viterbi algorithm
The Viterbi algorithm is a dynamic programming
Dynamic programming is both a mathematical optimization
File:Nelder-Mead Simionescu.gif, Nelder-Mead minimum search of Test functions for optimization, Simionescu's function. Simplex vertices a ...

is the optimum algorithm used to decode convolutional codes. There are simplifications to reduce the computational load. They rely on searching only the most likely paths. Although not optimum, they have generally been found to give good results in low noise environments.
Convolutional codes are used in voiceband modems (V.32, V.17, V.34) and in GSM mobile phones, as well as satellite and military communication devices.
Cryptographic coding

Cryptography
Cryptography, or cryptology (from grc, , translit=kryptós "hidden, secret"; and ''graphein'', "to write", or ''-logia
''-logy'' is a suffix in the English language, used with words originally adapted from Ancient Greek ending in (''- ...

or cryptographic coding is the practice and study of techniques for secure communication
Secure communication is when two entities are communicating and do not want a third party to listen in. For this to be the case, the entities need to communicate in a way that is unsusceptible to eavesdropping
Eavesdropping is the act of secret ...

in the presence of third parties (called adversaries). More generally, it is about constructing and analyzing protocol
Protocol may refer to:
Sociology and politics
* Protocol (politics) Protocol originally (in Late Middle English, c. 15th century) meant the minutes or logbook taken at a meeting, upon which an agreement was based. The term now commonly refers to a ...

s that block adversaries; various aspects in information security #REDIRECT Information security #REDIRECT Information security
Information security, sometimes shortened to infosec, is the practice of protecting information by mitigating information risks. It is part of Risk management information systems, inform ...

such as data confidentiality
Confidentiality involves a set of rules or a promise usually executed through Non-disclosure agreement, confidentiality agreements that limits access or places restrictions on certain types of information.
Legal confidentiality
Lawyers are o ...

, data integrity Data integrity is the maintenance of, and the assurance of, data accuracy and consistency over its entire life-cycle and is a critical aspect to the design, implementation, and usage of any system that stores, processes, or retrieves data. The term ...

, authentication
Authentication (from ''authentikos'', "real, genuine", from αὐθέντης ''authentes'', "author") is the act of proof (truth), proving an Logical assertion, assertion, such as the Digital identity, identity of a computer system user. In ...

, and non-repudiationNon-repudiation refers to a situation where a statement's author cannot successfully dispute its authorship or the validity of an associated contract
A contract is a legally binding document between at least two parties that defines and governs th ...

are central to modern cryptography. Modern cryptography exists at the intersection of the disciplines of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and their changes (cal ...

, computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , , and . Computer science ...

, and electrical engineering
Electrical engineering is an engineering discipline concerned with the study, design, and application of equipment, devices, and systems which use electricity, electronics
The field of electronics is a branch of physics and electrical enginee ...

. Applications of cryptography include ATM cards
ATM or atm often refers to:
* Atmosphere (unit) or atm, a unit of atmospheric pressure
* Automated teller machine, a cash dispenser or cash machine
ATM or atm may also refer to:
Computing
* ATM (computer), a ZX Spectrum clone developed in Mos ...

, , and electronic commerce
E-commerce (electronic commerce) is the activity of electronically
The field of electronics is a branch of physics and electrical engineering that deals with the emission, behaviour and effects of electrons using electronic devices. Electronic ...

.
Cryptography prior to the modern age was effectively synonymous with ''encryption
In cryptography
Cryptography, or cryptology (from grc, , translit=kryptós "hidden, secret"; and ''graphein'', "to write", or ''-logy, -logia'', "study", respectively), is the practice and study of techniques for secure communication in ...

'', the conversion of information from a readable state to apparent nonsense
Nonsense is a communication
Communication (from Latin
Latin (, or , ) is a classical language belonging to the Italic languages, Italic branch of the Indo-European languages. Latin was originally spoken in the area around Rome, know ...

. The originator of an encrypted message shared the decoding technique needed to recover the original information only with intended recipients, thereby precluding unwanted persons from doing the same. Since World War I
World War I or the First World War, often abbreviated as WWI or WW1, was a global war originating in Europe that lasted from 28 July 1914 to 11 November 1918. Contemporaneously known as the Great War, the World War, and "The war t ...

and the advent of the computer
A computer is a machine that can be programmed to Execution (computing), carry out sequences of arithmetic or logical operations automatically. Modern computers can perform generic sets of operations known as Computer program, programs. These ...

, the methods used to carry out cryptology have become increasingly complex and its application more widespread.
Modern cryptography is heavily based on mathematical theory and computer science practice; cryptographic algorithms are designed around computational hardness assumption
In computational complexity theory
Computational complexity theory focuses on classifying computational problems according to their resource usage, and relating these classes to each other. A computational problem is a task solved by a computer. ...

s, making such algorithms hard to break in practice by any adversary. It is theoretically possible to break such a system, but it is infeasible to do so by any known practical means. These schemes are therefore termed computationally secure; theoretical advances, e.g., improvements in integer factorization
In number theory
Number theory (or arithmetic or higher arithmetic in older usage) is a branch of devoted primarily to the study of the s and . German mathematician (1777–1855) said, "Mathematics is the queen of the sciences—and number ...

algorithms, and faster computing technology require these solutions to be continually adapted. There exist information-theoretically secureInformation-theoretic security is security of a cryptosystem which derives purely from information theory; the system cannot be broken even if the adversary (cryptography), adversary has unlimited computing power. The cryptosystem is considered crypt ...

schemes that cannot be broken even with unlimited computing power—an example is the one-time pad
In cryptography
Cryptography, or cryptology (from grc, , translit=kryptós "hidden, secret"; and ''graphein'', "to write", or ''-logia
''-logy'' is a suffix in the English language, used with words originally adapted from Ancient G ...

—but these schemes are more difficult to implement than the best theoretically breakable but computationally secure mechanisms.
Line coding

Aline code
In telecommunication, a line code is a pattern of voltage, current, or photons used to represent digital data transmission (telecommunications), transmitted down a transmission line. This repertoire of signals is usually called a constrained cod ...

(also called digital baseband modulation or digital baseband transmission method) is a code
In communication
Communication (from Latin
Latin (, or , ) is a classical language
A classical language is a language
A language is a structured system of communication
Communication (from Latin ''communicare'', mean ...

chosen for use within a communications system
400px, Communication system
A communications system or communication system is a collection of individual telecommunications networks, transmission systems, relay stations, tributary stations, and terminal equipment usually capable of inte ...

for baseband
Baseband is a signal
In signal processing
Signal processing is an electrical engineering subfield that focuses on analysing, modifying, and synthesizing signals such as audio signal processing, sound, image processing, images, and scie ...

transmission
Transmission may refer to:
Science and technology
* Power transmission
** Electric power transmission
* Transmission (mechanics), a machine which provides controlled application of power
* Transmission (telecommunications), the process of sending ...

purposes. Line coding is often used for digital data transport.
Line coding consists of representing the digital signal
A digital signal is a signal
In signal processing
Signal processing is an electrical engineering subfield that focuses on analysing, modifying, and synthesizing signals such as audio signal processing, sound, image processing, image ...

to be transported by an amplitude- and time-discrete signal that is optimally tuned for the specific properties of the physical channel (and of the receiving equipment). The waveform
In electronics
The field of electronics is a branch of physics and electrical engineering that deals with the emission, behaviour and effects of electrons
The electron is a subatomic particle
In physical sciences, subatomic part ...

pattern of voltage or current used to represent the 1s and 0s of a digital data on a transmission link is called ''line encoding''. The common types of line encoding are unipolar, polar
Polar may refer to:
Geography
Polar may refer to:
* Geographical pole, either of two fixed points on the surface of a rotating body or planet, at 90 degrees from the equator, based on the axis around which a body rotates
*Polar climate, the cli ...

, bipolar, and Manchester encoding
In telecommunication
Telecommunication is the transmission of information by various types of technologies over wire
A wire is a single usually cylindrical
A cylinder (from Greek
Greek may refer to:
Greece
Anything of, from, or re ...

.
Other applications of coding theory

Another concern of coding theory is designing codes that helpsynchronization
Synchronization is the coordination of events to operate a system
A system is a group of Interaction, interacting or interrelated elements that act according to a set of rules to form a unified whole.
A system, surrounded and influenced ...

. A code may be designed so that a phase shift
In physics
Physics is the that studies , its , its and behavior through , and the related entities of and . "Physical science is that department of knowledge which relates to the order of nature, or, in other words, to the regular succ ...

can be easily detected and corrected and that multiple signals can be sent on the same channel.
Another application of codes, used in some mobile phone systems, is code-division multiple access
Code-division multiple access (CDMA) is a channel access method
In telecommunications and computer networks, a channel access method or multiple access method allows more than two terminal (telecommunication), terminals connected to the same t ...

(CDMA). Each phone is assigned a code sequence that is approximately uncorrelated with the codes of other phones. When transmitting, the code word is used to modulate the data bits representing the voice message. At the receiver, a demodulation process is performed to recover the data. The properties of this class of codes allow many users (with different codes) to use the same radio channel at the same time. To the receiver, the signals of other users will appear to the demodulator only as a low-level noise.
Another general class of codes are the automatic repeat-request (ARQ) codes. In these codes the sender adds redundancy to each message for error checking, usually by adding check bits. If the check bits are not consistent with the rest of the message when it arrives, the receiver will ask the sender to retransmit the message. All but the simplest wide area network
A wide area network (WAN) is a telecommunications network that extends over a large geographic area for the primary purpose of computer networking. Wide area networks are often established with Leased line, leased telecommunication circuits.
...

protocols use ARQ. Common protocols include SDLC (IBM), TCP (Internet), X.25
X.25 is an ITU-T
The ITU Telecommunication Standardization Sector (ITU-T) coordinates standards for telecommunications and Information Communication Technology such as X.509 for cybersecurity, Y.3172 and Y.3173 for machine learning, and H.264/MPE ...

(International) and many others. There is an extensive field of research on this topic because of the problem of matching a rejected packet against a new packet. Is it a new one or is it a retransmission? Typically numbering schemes are used, as in TCP.
Group testing

Group testing
In statistics and combinatorics, combinatorial mathematics, group testing is any procedure that breaks up the task of identifying certain objects into tests on groups of items, rather than on individual ones. First studied by Robert Dorfman in 1 ...

uses codes in a different way. Consider a large group of items in which a very few are different in a particular way (e.g., defective products or infected test subjects). The idea of group testing is to determine which items are "different" by using as few tests as possible. The origin of the problem has its roots in the Second World War
World War II or the Second World War, often abbreviated as WWII or WW2, was a global war
A world war is "a war
War is an intense armed conflict between states
State may refer to:
Arts, entertainment, and media Literatur ...

when the United States Army Air Forces
The United States Army Air Forces (USAAF or AAF) was the major land-based aerial warfare
Aerial warfare is the use of military aircraft
A military aircraft is any fixed-wing
A fixed-wing aircraft is a heavier-than-air flying machi ...

needed to test its soldiers for syphilis
Syphilis is a sexually transmitted infection
Sexually transmitted infections (STIs), also referred to as sexually transmitted diseases (STDs) and the older term venereal disease, are infection
An infection is the invasion of an orga ...

.
Analog coding

Information is encoded analogously in theneural network
A neural network is a network or circuit of neurons, or in a modern sense, an artificial neural network
Artificial neural networks (ANNs), usually simply called neural networks (NNs), are computing systems vaguely inspired by the biolog ...

s of brain
A brain is an organ
Organ may refer to:
Biology
* Organ (anatomy)
An organ is a group of Tissue (biology), tissues with similar functions. Plant life and animal life rely on many organs that co-exist in organ systems.
A given organ's tis ...

s, in analog signal processingAnalog signal processing is a type of signal processing
Signal processing is an electrical engineering subfield that focuses on analysing, modifying, and synthesizing signals such as audio signal processing, sound, image processing, images, and ...

, and analog electronics
Analogue electronics ( en-US, analog electronics) are electronic systems with a continuously variable signal, in contrast to digital electronics where signals usually take only two levels. The term "analogue" describes the proportional relations ...

. Aspects of analog coding include analog error correction,
analog data compression and analog encryption.
Neural coding

Neural coding
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mecha ...

is a neuroscience
Neuroscience is the scientific study of the nervous system
In biology, the classical doctrine of the nervous system determines that it is a Complex system, highly complex part of an animal that coordinates its Behavior, actions and Sens ...

-related field concerned with how sensory and other information is represented in the brain
A brain is an organ
Organ may refer to:
Biology
* Organ (anatomy)
An organ is a group of Tissue (biology), tissues with similar functions. Plant life and animal life rely on many organs that co-exist in organ systems.
A given organ's tis ...

by networks
Network and networking may refer to:
Arts, entertainment, and media
* Network (1976 film), ''Network'' (1976 film), a 1976 American film
* Network (2019 film), ''Network'' (2019 film), an Indian film
* Network (album), ''Network'' (album), a 2004 ...

of neurons
A neuron or nerve cell is an electrically excitable cell that communicates with other cells via specialized connections called synapse
In the nervous system
In biology
Biology is the natural science that studies life and living ...

. The main goal of studying neural coding is to characterize the relationship between the stimulus
A stimulus is something that causes a physiological response. It may refer to:
*Stimulation
**Stimulus (physiology), something external that influences an activity
**Stimulus (psychology), a concept in behaviorism and perception
*Stimulus (economi ...

and the individual or ensemble neuronal responses and the relationship among electrical activity of the neurons in the ensemble. It is thought that neurons can encode both digital
Digital usually refers to something using discrete digits, often binary digits.
Technology and computing Hardware
*Digital electronics
Digital electronics is a field of electronics
The field of electronics is a branch of physics and electr ...

and information, and that neurons follow the principles of information theory and compress information, and detect and correct
errors in the signals that are sent throughout the brain and wider nervous system.
See also

*Coding gain
In coding theory and related engineering problems, coding gain is the measure in the difference between the signal-to-noise ratio (SNR) levels between the uncoded system and coded system required to reach the same bit error rate (BER) levels when ...

* Covering code
* Error correction code
In computing
Computing is any goal-oriented activity requiring, benefiting from, or creating computing machinery. It includes the study and experimentation of algorithmic processes and development of both computer hardware , hardware and soft ...

* Folded Reed–Solomon code
* Group testing
In statistics and combinatorics, combinatorial mathematics, group testing is any procedure that breaks up the task of identifying certain objects into tests on groups of items, rather than on individual ones. First studied by Robert Dorfman in 1 ...

* Hamming distance
In information theory
Information theory is the scientific study of the quantification, storage, and communication
Communication (from Latin ''communicare'', meaning "to share" or "to be in relation with") is "an apparent answer to ...

, Hamming weight
The Hamming weight of a string
String or strings may refer to:
*String (structure), a long flexible structure made from threads twisted together, which is used to tie, bind, or hang other objects
Arts, entertainment, and media Films
* Strings ( ...

* Lee distance In coding theory
Coding theory is the study of the properties of codes and their respective fitness for specific applications. Codes are used for data compression, cryptography, error detection and correction, data transmission and data storage. ...

* List of algebraic coding theory topics
* Spatial coding and MIMO
Miami Modernist architecture, or MiMo, is a regional style of architecture that developed in South Florida
South Florida is the southernmost region of the U.S. state
In the United States
The United States of America (USA), common ...

in multiple antenna research
** Spatial diversity coding is spatial coding that transmits replicas of the information signal along different spatial paths, so as to increase the reliability of the data transmission.
** Spatial interference cancellation coding
**
* Timeline of information theory, data compression, and error correcting codes
Notes

References

* Elwyn R. Berlekamp (2014), ''Algebraic Coding Theory'', World Scientific Publishing (revised edition), . * David J. C. MacKay, MacKay, David J. C..Information Theory, Inference, and Learning Algorithms

' Cambridge: Cambridge University Press, 2003. * Vera Pless (1982), ''Introduction to the Theory of Error-Correcting Codes'', John Wiley & Sons, Inc., . * Randy Yates,

A Coding Theory Tutorial

'. {{Authority control Coding theory, Error detection and correction