A video coding format (or sometimes video compression format) is a
content representation format for storage or transmission of
digital video content (such as in a data file or
bitstream
A bitstream (or bit stream), also known as binary sequence, is a sequence of bits.
A bytestream is a sequence of bytes. Typically, each byte is an 8-bit quantity, and so the term octet stream is sometimes used interchangeably. An octet may ...
). It typically uses a standardized
video compression
In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression ...
algorithm, most commonly based on
discrete cosine transform (DCT) coding and
motion compensation
Motion compensation in computing, is an algorithmic technique used to predict a frame in a video, given the previous and/or future frames by accounting for motion of the camera and/or objects in the video. It is employed in the encoding of video d ...
. A specific software,
firmware
In computing, firmware is a specific class of computer software that provides the low-level control for a device's specific hardware. Firmware, such as the BIOS of a personal computer, may contain basic functions of a device, and may provide h ...
, or hardware implementation capable of compression or decompression to/from a specific video coding format is called a
video codec
A video codec is software or hardware that compresses and decompresses digital video. In the context of video compression, ''codec'' is a portmanteau of ''encoder'' and ''decoder'', while a device that only compresses is typically called an '' ...
.
Some video coding formats are documented by a detailed
technical specification
A specification often refers to a set of documented requirements to be satisfied by a material, design, product, or service. A specification is often a type of technical standard.
There are different types of technical or engineering specificati ...
document known as a video coding specification. Some such specifications are written and approved by
standardization organization
A standards organization, standards body, standards developing organization (SDO), or standards setting organization (SSO) is an organization whose primary function is developing, coordinating, promulgating, revising, amending, reissuing, interpr ...
s as
technical standard
A technical standard is an established norm or requirement for a repeatable technical task which is applied to a common and repeated use of rules, conditions, guidelines or characteristics for products or related processes and production methods, ...
s, and are thus known as a video coding standard. The term 'standard' is also sometimes used for
''de facto'' standards as well as formal standards.
Video content encoded using a particular video coding format is normally bundled with an audio stream (encoded using an
audio coding format
An audio coding format (or sometimes audio compression format) is a content representation format for storage or transmission of digital audio (such as in digital television, digital radio and in audio and video files). Examples of audio coding ...
) inside a
multimedia container format
A container format (informally, sometimes called a wrapper) or metafile is a file format that allows multiple data streams to be embedded into a single file, usually along with metadata for identifying and further detailing those streams. Not ...
such as
AVI,
MP4,
FLV
Flash Video is a container file format used to deliver digital video content (e.g., TV shows, movies, etc.) over the Internet using Adobe Flash Player version 6 and newer. Flash Video content may also be embedded within SWF files. There are t ...
,
RealMedia, or
Matroska
Matroska is a project to create a container format that can hold an unlimited number of video, audio, picture, or subtitle tracks in one file. The Matroska Multimedia Container is similar in concept to other containers like AVI, MP4, or Adva ...
. As such, the user normally doesn't have a
H.264
Advanced Video Coding (AVC), also referred to as H.264 or MPEG-4 Part 10, is a video compression standard based on block-oriented, motion-compensated coding. It is by far the most commonly used format for the recording, compression, and distr ...
file, but instead has a .mp4
video file
A video file format is a type of file format for storing digital video data on a computer system. Video is almost always stored using lossy compression to reduce the file size.
A video file normally consists of a container (e.g. in the Matroska ...
, which is an MP4 container containing H.264-encoded video, normally alongside
AAC-encoded audio. Multimedia container formats can contain any one of a number of different video coding formats; for example the MP4 container format can contain video in either the
MPEG-2 Part 2 or the H.264 video coding format, among others. Another example is the initial specification for the file type
WebM
WebM is an audiovisual media file format. It is primarily intended to offer a royalty-free alternative to use in the HTML5 video and the HTML5 audio elements. It has a sister project, WebP, for images. The development of the format is sponsored ...
, which specified the container format (Matroska), but also exactly which video (
VP8
VP8 is an open and royalty-free video compression format released by On2 Technologies in 2008.
Initially released as a proprietary successor to On2's previous VP7 format, VP8 was released as an open and royalty-free format in May 2010 after Goo ...
) and audio (
Vorbis
Vorbis is a free and open-source software project headed by the Xiph.Org Foundation. The project produces an audio coding format and software reference encoder/decoder (codec) for lossy audio compression. Vorbis is most commonly used in conj ...
) compression format is used inside the Matroska container, even though the Matroska container format itself is capable of containing other video coding formats (
VP9
VP9 is an open and royalty-free video coding format developed by Google.
VP9 is the successor to VP8 and competes mainly with MPEG's High Efficiency Video Coding (HEVC/H.265).
At first, VP9 was mainly used on Google's video platform YouTube. ...
video and
Opus
''Opus'' (pl. ''opera'') is a Latin word meaning "work". Italian equivalents are ''opera'' (singular) and ''opere'' (pl.).
Opus or OPUS may refer to:
Arts and entertainment Music
* Opus number, (abbr. Op.) specifying order of (usually) publicatio ...
audio support was later added to the
WebM
WebM is an audiovisual media file format. It is primarily intended to offer a royalty-free alternative to use in the HTML5 video and the HTML5 audio elements. It has a sister project, WebP, for images. The development of the format is sponsored ...
specification).
Distinction between ''format'' and ''codec''
A ''format'' is the layout plan for data produced or consumed by a ''codec''.
Although video coding formats such as H.264 are sometimes referred to as ''codecs'', there is a clear conceptual difference between a specification and its implementations. Video coding formats are described in specifications, and software,
firmware
In computing, firmware is a specific class of computer software that provides the low-level control for a device's specific hardware. Firmware, such as the BIOS of a personal computer, may contain basic functions of a device, and may provide h ...
, or hardware to encode/decode data in a given video coding format from/to uncompressed video are implementations of those specifications. As an analogy, the video coding format
H.264
Advanced Video Coding (AVC), also referred to as H.264 or MPEG-4 Part 10, is a video compression standard based on block-oriented, motion-compensated coding. It is by far the most commonly used format for the recording, compression, and distr ...
(specification) is to the
codec
A codec is a device or computer program that encodes or decodes a data stream or signal. ''Codec'' is a portmanteau of coder/decoder.
In electronic communications, an endec is a device that acts as both an encoder and a decoder on a signal or da ...
OpenH264
OpenH264 is a free software library for real-time encoding and decoding video streams in the H.264/MPEG-4 AVC format. It is released under the terms of the Simplified BSD License."
History Move to free-to-use binaries
On October 30, 2013, Rowan T ...
(specific implementation) what the
C Programming Language
''The C Programming Language'' (sometimes termed ''K&R'', after its authors' initials) is a computer programming book written by Brian Kernighan and Dennis Ritchie, the latter of whom originally designed and implemented the language, as well as ...
(specification) is to the compiler
GCC (specific implementation). Note that for each specification (e.g.
H.264
Advanced Video Coding (AVC), also referred to as H.264 or MPEG-4 Part 10, is a video compression standard based on block-oriented, motion-compensated coding. It is by far the most commonly used format for the recording, compression, and distr ...
), there can be many codecs implementing that specification (e.g.
x264
x264 is a free and open-source software library and a command-line utility developed by VideoLAN for encoding video streams into the H.264/MPEG-4 AVC video coding format. It is released under the terms of the GNU General Public License.
Histor ...
, OpenH264,
H.264/MPEG-4 AVC products and implementations).
This distinction is not consistently reflected terminologically in the literature. The H.264 specification calls
H.261
H.261 is an ITU-T video compression standard, first ratified in November 1988. It is the first member of the H.26x family of video coding standards in the domain of the ITU-T Study Group 16 Video Coding Experts Group (VCEG, then Specialists Gro ...
,
H.262
H.262 or MPEG-2 Part 2 (formally known as ITU-T Recommendation H.262 and ISO/IEC 13818-2, also known as MPEG-2 Video) is a video coding format standardised and jointly maintained by ITU-T Study Group 16 Video Coding Experts Group (VCEG) and Inte ...
,
H.263, and
H.264
Advanced Video Coding (AVC), also referred to as H.264 or MPEG-4 Part 10, is a video compression standard based on block-oriented, motion-compensated coding. It is by far the most commonly used format for the recording, compression, and distr ...
''video coding standards'' and does not contain the word ''codec''.
The
Alliance for Open Media
The Alliance for Open Media (AOMedia) is a non-profit industry consortium that develops open, royalty-free technology for multimedia delivery headquartered in Wakefield, Massachusetts. It uses the ideas and principles of open web standard develo ...
clearly distinguishes between the
AV1
AOMedia Video 1 (AV1) is an open, royalty-free video coding format initially designed for video transmissions over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia), a consortium founded in 2015 th ...
video coding format and the accompanying codec they are developing, but calls the video coding format itself a ''
video codec
A video codec is software or hardware that compresses and decompresses digital video. In the context of video compression, ''codec'' is a portmanteau of ''encoder'' and ''decoder'', while a device that only compresses is typically called an '' ...
specification''. The
VP9
VP9 is an open and royalty-free video coding format developed by Google.
VP9 is the successor to VP8 and competes mainly with MPEG's High Efficiency Video Coding (HEVC/H.265).
At first, VP9 was mainly used on Google's video platform YouTube. ...
specification calls the video coding format VP9 itself a ''codec''.
As an example of conflation, Chromium's and Mozilla's pages listing their video format support both call video coding formats such as H.264 ''codecs''. As another example, in Cisco's announcement of a free-as-in-beer video codec, the press release refers to the H.264 video coding format as a ''codec'' ("choice of a common video codec"), but calls Cisco's implementation of a H.264 encoder/decoder a ''codec'' shortly thereafter ("open-source our H.264 codec").
A video coding format does not dictate all
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific Computational problem, problems or to perform a computation. Algorithms are used as specificat ...
s used by a
codec
A codec is a device or computer program that encodes or decodes a data stream or signal. ''Codec'' is a portmanteau of coder/decoder.
In electronic communications, an endec is a device that acts as both an encoder and a decoder on a signal or da ...
implementing the format. For example, a large part of how video compression typically works is by finding
similarities between video frames (block-matching), and then achieving compression by copying previously-coded similar subimages (e.g.,
macroblocks) and adding small differences when necessary. Finding optimal combinations of such predictors and differences is an
NP-hard
In computational complexity theory, NP-hardness ( non-deterministic polynomial-time hardness) is the defining property of a class of problems that are informally "at least as hard as the hardest problems in NP". A simple example of an NP-hard pr ...
problem, meaning that it is practically impossible to find an optimal solution. While the video coding format must support such compression across frames in the bitstream format, by not needlessly mandating specific algorithms for finding such block-matches and other encoding steps, the codecs implementing the video coding specification have some freedom to optimize and innovate in their choice of algorithms. For example, section 0.5 of the H.264 specification says that encoding algorithms are not part of the specification.
Free choice of algorithm also allows different
space–time complexity trade-offs for the same video coding format, so a live feed can use a fast but space-inefficient algorithm, while a one-time
DVD
The DVD (common abbreviation for Digital Video Disc or Digital Versatile Disc) is a digital optical disc data storage format. It was invented and developed in 1995 and first released on November 1, 1996, in Japan. The medium can store any kind ...
encoding for later mass production can trade long encoding-time for space-efficient encoding.
History
The concept of
analog video
Video is an electronic medium for the recording, copying
Copying is the duplication of information or an artifact based on an instance of that information or artifact, and not using the process that originally generated it. With analog f ...
compression dates back to
1929
This year marked the end of a period known in American history as the Roaring Twenties after the Wall Street Crash of 1929 ushered in a worldwide Great Depression. In the Americas, an agreement was brokered to end the Cristero War, a Catholic ...
, when R.D. Kell in
Britain
Britain most often refers to:
* The United Kingdom, a sovereign state in Europe comprising the island of Great Britain, the north-eastern part of the island of Ireland and many smaller islands
* Great Britain, the largest island in the United King ...
proposed the concept of transmitting only the portions of the scene that changed from frame-to-frame. The concept of
digital video compression dates back to 1952, when
Bell Labs
Nokia Bell Labs, originally named Bell Telephone Laboratories (1925–1984),
then AT&T Bell Laboratories (1984–1996)
and Bell Labs Innovations (1996–2007),
is an American industrial research and scientific development company owned by mult ...
researchers B.M. Oliver and
C.W. Harrison proposed the use of
differential pulse-code modulation
Differential pulse-code modulation (DPCM) is a signal encoder that uses the baseline of pulse-code modulation (PCM) but adds some functionalities based on the prediction of the samples of the signal. The input can be an analog signal or a digital ...
(DPCM) in video coding. In 1959, the concept of
inter-frame
An inter frame is a frame in a video compression stream which is expressed in terms of one or more neighboring frames. The "inter" part of the term refers to the use of ''Inter frame prediction''. This kind of prediction tries to take advantage fr ...
motion compensation
Motion compensation in computing, is an algorithmic technique used to predict a frame in a video, given the previous and/or future frames by accounting for motion of the camera and/or objects in the video. It is employed in the encoding of video d ...
was proposed by
NHK
, also known as NHK, is a Japanese public broadcaster. NHK, which has always been known by this romanized initialism in Japanese, is a statutory corporation funded by viewers' payments of a television license fee.
NHK operates two terrestr ...
researchers Y. Taki, M. Hatori and S. Tanaka, who proposed predictive inter-frame video coding in the
temporal dimension
In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coordi ...
.
In 1967,
University of London
The University of London (UoL; abbreviated as Lond or more rarely Londin in post-nominals) is a federal public research university located in London, England, United Kingdom. The university was established by royal charter in 1836 as a degree ...
researchers A.H. Robinson and C. Cherry proposed
run-length encoding
Run-length encoding (RLE) is a form of lossless data compression in which ''runs'' of data (sequences in which the same data value occurs in many consecutive data elements) are stored as a single data value and count, rather than as the original ...
(RLE), a
lossless compression scheme, to reduce the transmission bandwidth of
analog television
Analog television is the original television technology that uses analog signals to transmit video and audio. In an analog television broadcast, the brightness, colors and sound are represented by amplitude, instantaneous phase and frequency, ...
signals.
The earliest digital video coding algorithms were either for
uncompressed video
Uncompressed video is digital video that either has never been compressed or was generated by decompressing previously compressed digital video. It is commonly used by video cameras, video monitors, video recording devices (including general-purp ...
or used
lossless compression, both methods inefficient and impractical for digital video coding.
Digital video was introduced in the 1970s,
initially using uncompressed
pulse-code modulation
Pulse-code modulation (PCM) is a method used to digitally represent sampled analog signals. It is the standard form of digital audio in computers, compact discs, digital telephony and other digital audio applications. In a PCM stream, the ...
(PCM) requiring high
bitrate
In telecommunications and computing, bit rate (bitrate or as a variable ''R'') is the number of bits that are conveyed or processed per unit of time.
The bit rate is expressed in the unit bit per second (symbol: bit/s), often in conjunction w ...
s around 45200
Mbit/s
In telecommunications, data-transfer rate is the average number of bits (bitrate), characters or symbols (baudrate), or data blocks per unit time passing through a communication link in a data-transmission system. Common data rate units are multi ...
for
standard-definition
Standard-definition television (SDTV, SD, often shortened to standard definition) is a television system which uses a resolution that is not considered to be either high or enhanced definition. "Standard" refers to it being the prevailing sp ...
(SD) video,
which was up to 2,000 times greater than the
telecommunication
Telecommunication is the transmission of information by various types of technologies over wire, radio, optical, or other electromagnetic systems. It has its origin in the desire of humans for communication over a distance greater than that fe ...
bandwidth
Bandwidth commonly refers to:
* Bandwidth (signal processing) or ''analog bandwidth'', ''frequency bandwidth'', or ''radio bandwidth'', a measure of the width of a frequency range
* Bandwidth (computing), the rate of data transfer, bit rate or thr ...
(up to 100
kbit/s
In telecommunications, data-transfer rate is the average number of bits (bitrate), characters or symbols (baudrate), or data blocks per unit time passing through a communication link in a data-transmission system. Common data rate units are multi ...
) available until the 1990s.
Similarly, uncompressed
high-definition (HD)
1080p
1080p (1920×1080 progressively displayed pixels; also known as Full HD or FHD, and BT.709) is a set of HDTV high-definition video modes characterized by 1,920 pixels displayed across the screen horizontally and 1,080 pixels down the screen vert ...
video requires bitrates exceeding 1
Gbit/s
In telecommunications, data-transfer rate is the average number of bits ( bitrate), characters or symbols ( baudrate), or data blocks per unit time passing through a communication link in a data-transmission system. Common data rate units are mu ...
, significantly greater than the bandwidth available in the
2000s.
Motion-compensated DCT
Practical
video compression
In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression ...
emerged with the development of
motion-compensated DCT (MC DCT) coding,
also called block motion compensation (BMC)
or DCT motion compensation. This is a hybrid coding algorithm,
which combines two key
data compression
In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either lossy or lossless. Lossless compression ...
techniques:
discrete cosine transform (DCT) coding
in the
spatial dimension
In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coordin ...
, and predictive
motion compensation
Motion compensation in computing, is an algorithmic technique used to predict a frame in a video, given the previous and/or future frames by accounting for motion of the camera and/or objects in the video. It is employed in the encoding of video d ...
in the
temporal dimension
In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coordi ...
.
DCT coding is a
lossy
In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size ...
block compression
transform coding
Transform coding is a type of data compression for "natural" data like audio signals or photographic images. The transformation is typically lossless (perfectly reversible) on its own but is used to enable better (more targeted) quantization, ...
technique that was first proposed by
Nasir Ahmed, who initially intended it for
image compression
Image compression is a type of data compression applied to digital images, to reduce their cost for storage or transmission. Algorithms may take advantage of visual perception and the statistical properties of image data to provide superior r ...
, while he was working at
Kansas State University
Kansas State University (KSU, Kansas State, or K-State) is a public land-grant research university with its main campus in Manhattan, Kansas, United States. It was opened as the state's land-grant college in 1863 and was the first public instit ...
in 1972. It was then developed into a practical image compression algorithm by Ahmed with T. Natarajan and
K. R. Rao at the
University of Texas
The University of Texas at Austin (UT Austin, UT, or Texas) is a public research university in Austin, Texas. It was founded in 1883 and is the oldest institution in the University of Texas System. With 40,916 undergraduate students, 11,075 ...
in 1973, and was published in 1974.
The other key development was motion-compensated hybrid coding.
In 1974, Ali Habibi at the
University of Southern California
The University of Southern California (USC, SC, or Southern Cal) is a Private university, private research university in Los Angeles, California, United States. Founded in 1880 by Robert M. Widney, it is the oldest private research university in C ...
introduced hybrid coding,
which combines predictive coding with transform coding.
He examined several transform coding techniques, including the DCT,
Hadamard transform
The Hadamard transform (also known as the Walsh–Hadamard transform, Hadamard–Rademacher–Walsh transform, Walsh transform, or Walsh–Fourier transform) is an example of a generalized class of Fourier transforms. It performs an orthogonal ...
,
Fourier transform
A Fourier transform (FT) is a mathematical transform that decomposes functions into frequency components, which are represented by the output of the transform as a function of frequency. Most commonly functions of time or space are transformed, ...
, slant transform, and
Karhunen-Loeve transform.
However, his algorithm was initially limited to
intra-frame
Intra-frame coding is a data compression technique used within a video frame, enabling smaller file sizes and lower bitrates, with little or no loss in quality. Since neighboring pixels within an image are often very similar, rather than storing ...
coding in the spatial dimension. In 1975, John A. Roese and Guner S. Robinson extended Habibi's hybrid coding algorithm to the temporal dimension, using transform coding in the spatial dimension and predictive coding in the temporal dimension, developing
inter-frame
An inter frame is a frame in a video compression stream which is expressed in terms of one or more neighboring frames. The "inter" part of the term refers to the use of ''Inter frame prediction''. This kind of prediction tries to take advantage fr ...
motion-compensated hybrid coding.
For the spatial transform coding, they experimented with different transforms, including the DCT and the
fast Fourier transform (FFT), developing inter-frame hybrid coders for them, and found that the DCT is the most efficient due to its reduced complexity, capable of compressing image data down to 0.25-
bit
The bit is the most basic unit of information in computing and digital communications. The name is a portmanteau of binary digit. The bit represents a logical state with one of two possible values. These values are most commonly represente ...
per
pixel
In digital imaging, a pixel (abbreviated px), pel, or picture element is the smallest addressable element in a raster image, or the smallest point in an all points addressable display device.
In most digital display devices, pixels are the smal ...
for a
videotelephone
Videotelephony, also known as videoconferencing and video teleconferencing, is the two-way or multipoint reception and transmission of audio and video signals by people in different locations for real time communication.McGraw-Hill Concise Ency ...
scene with image quality comparable to a typical intra-frame coder requiring 2-bit per pixel.
The DCT was applied to video encoding by Wen-Hsiung Chen,
who developed a fast DCT algorithm with C.H. Smith and S.C. Fralick in 1977,
and founded
Compression Labs to commercialize DCT technology.
In 1979,
Anil K. Jain and Jaswant R. Jain further developed motion-compensated DCT video compression.
This led to Chen developing a practical video compression algorithm, called motion-compensated DCT or adaptive scene coding, in 1981.
Motion-compensated DCT later became the standard coding technique for video compression from the late 1980s onwards.
Video coding standards
The first digital video coding standard was
H.120, developed by the
CCITT (now ITU-T) in 1984.
H.120 was not usable in practice, as its performance was too poor.
H.120 used motion-compensated DPCM coding,
a lossless compression algorithm that was inefficient for video coding.
During the late 1980s, a number of companies began experimenting with
discrete cosine transform (DCT) coding, a much more efficient form of compression for video coding. The CCITT received 14 proposals for DCT-based video compression formats, in contrast to a single proposal based on
vector quantization
Vector quantization (VQ) is a classical quantization technique from signal processing that allows the modeling of probability density functions by the distribution of prototype vectors. It was originally used for data compression. It works by di ...
(VQ) compression. The
H.261
H.261 is an ITU-T video compression standard, first ratified in November 1988. It is the first member of the H.26x family of video coding standards in the domain of the ITU-T Study Group 16 Video Coding Experts Group (VCEG, then Specialists Gro ...
standard was developed based on motion-compensated DCT compression.
H.261 was the first practical video coding standard,
and uses
patents licensed from a number of companies, including
Hitachi
() is a Japanese multinational corporation, multinational Conglomerate (company), conglomerate corporation headquartered in Chiyoda, Tokyo, Japan. It is the parent company of the Hitachi Group (''Hitachi Gurūpu'') and had formed part of the Ni ...
,
PictureTel
PictureTel Corporation, often shortened to PictureTel Corp., was one of the first commercial videoconferencing product companies. It achieved peak revenues of over $490 million in 1996 and 1997 and was eventually acquired by Polycom in October ...
,
NTT,
BT, and
Toshiba
, commonly known as Toshiba and stylized as TOSHIBA, is a Japanese multinational conglomerate corporation headquartered in Minato, Tokyo, Japan. Its diversified products and services include power, industrial and social infrastructure system ...
, among others.
Since H.261, motion-compensated DCT compression has been adopted by all the major video coding standards (including the
H.26x
The Video Coding Experts Group or Visual Coding Experts Group (VCEG, also known as Question 6) is a working group of the ITU Telecommunication Standardization Sector (ITU-T) concerned with standards for compression coding of video, images, audio, ...
and
MPEG
The Moving Picture Experts Group (MPEG) is an alliance of working groups established jointly by International Organization for Standardization, ISO and International Electrotechnical Commission, IEC that sets standards for media coding, includ ...
formats) that followed.
MPEG-1
MPEG-1 is a standard for lossy compression of video and audio. It is designed to compress VHS-quality raw digital video and CD audio down to about 1.5 Mbit/s (26:1 and 6:1 compression ratios respectively) without excessive quality loss, making ...
, developed by the
Motion Picture Experts Group
The Moving Picture Experts Group (MPEG) is an alliance of working groups established jointly by ISO and IEC that sets standards for media coding, including compression coding of audio, video, graphics, and genomic data; and transmission and f ...
(MPEG), followed in 1991, and it was designed to compress
VHS-quality video.
It was succeeded in 1994 by
MPEG-2
MPEG-2 (a.k.a. H.222/H.262 as was defined by the ITU) is a standard for "the generic video coding format, coding of moving pictures and associated audio information". It describes a combination of Lossy compression, lossy video compression and ...
/
H.262
H.262 or MPEG-2 Part 2 (formally known as ITU-T Recommendation H.262 and ISO/IEC 13818-2, also known as MPEG-2 Video) is a video coding format standardised and jointly maintained by ITU-T Study Group 16 Video Coding Experts Group (VCEG) and Inte ...
,
which was developed with patents licensed from a number of companies, primarily
Sony
, commonly stylized as SONY, is a Japanese multinational conglomerate corporation headquartered in Minato, Tokyo, Japan. As a major technology company, it operates as one of the world's largest manufacturers of consumer and professional ...
,
Thomson Thomson may refer to:
Names
* Thomson (surname), a list of people with this name and a description of its origin
* Thomson baronets, four baronetcies created for persons with the surname Thomson
Businesses and organizations
* SGS-Thomson Mic ...
and
Mitsubishi Electric
, established on 15 January 1921, is a Japanese multinational electronics and electrical equipment manufacturing company headquartered in Tokyo, Japan. It is one of the core companies of Mitsubishi. The products from MELCO include elevators an ...
.
MPEG-2 became the standard video format for
DVD
The DVD (common abbreviation for Digital Video Disc or Digital Versatile Disc) is a digital optical disc data storage format. It was invented and developed in 1995 and first released on November 1, 1996, in Japan. The medium can store any kind ...
and
SD digital television.
Its motion-compensated DCT algorithm was able to achieve a
compression ratio
The compression ratio is the ratio between the volume of the cylinder and combustion chamber in an internal combustion engine at their maximum and minimum values.
A fundamental specification for such engines, it is measured two ways: the stati ...
of up to 100:1, enabling the development of
digital media
Digital media is any communication media that operate in conjunction with various encoded machine-readable data formats. Digital media can be created, viewed, distributed, modified, listened to, and preserved on a digital electronics device. ' ...
technologies such as
video-on-demand
Video on demand (VOD) is a media distribution system that allows users to access videos without a traditional video playback device and the constraints of a typical static broadcasting schedule. In the 20th century, broadcasting in the form of o ...
(VOD)
and
high-definition television
High-definition television (HD or HDTV) describes a television system which provides a substantially higher image resolution than the previous generation of technologies. The term has been used since 1936; in more recent times, it refers to the g ...
(HDTV).
In 1999, it was followed by
MPEG-4/
H.263, which was a major leap forward for video compression technology.
It uses patents licensed from a number of companies, primarily Mitsubishi,
Hitachi
() is a Japanese multinational corporation, multinational Conglomerate (company), conglomerate corporation headquartered in Chiyoda, Tokyo, Japan. It is the parent company of the Hitachi Group (''Hitachi Gurūpu'') and had formed part of the Ni ...
and
Panasonic
formerly between 1935 and 2008 and the first incarnation of between 2008 and 2022, is a major Japanese multinational corporation, multinational Conglomerate (company), conglomerate corporation, headquartered in Kadoma, Osaka, Kadoma, Osaka P ...
.
The most widely used video coding format is
H.264/MPEG-4 AVC.
It was developed in 2003, and uses patents licensed from a number of organizations, primarily Panasonic,
Godo Kaisha IP Bridge and
LG Electronics
LG Electronics Inc. () is a South Korean multinational electronics company headquartered in Yeouido-dong, Seoul, South Korea. LG Electronics is a part of LG Corporation, the fourth largest '' chaebol'' in South Korea, and often considered a ...
.
In contrast to the standard DCT used by its predecessors, AVC uses the
integer DCT.
H.264 is one of the video encoding standards for
Blu-ray Disc
The Blu-ray Disc (BD), often known simply as Blu-ray, is a Digital media, digital optical disc data storage format. It was invented and developed in 2005 and released on June 20, 2006 worldwide. It is designed to supersede the DVD format, and c ...
s; all Blu-ray Disc players must be able to decode H.264. It is also widely used by streaming internet sources, such as videos from
YouTube
YouTube is a global online video platform, online video sharing and social media, social media platform headquartered in San Bruno, California. It was launched on February 14, 2005, by Steve Chen, Chad Hurley, and Jawed Karim. It is owned by ...
,
Netflix
Netflix, Inc. is an American subscription video on-demand over-the-top streaming service and production company based in Los Gatos, California. Founded in 1997 by Reed Hastings and Marc Randolph in Scotts Valley, California, it offers a fil ...
,
Vimeo
Vimeo, Inc. () is an American video hosting, sharing, and services platform provider headquartered in New York City. Vimeo focuses on the delivery of high-definition video across a range of devices. Vimeo's business model is through software as ...
, and the
iTunes Store
The iTunes Store is a digital media store operated by Apple Inc. It opened on April 28, 2003, as a result of Steve Jobs' push to open a digital marketplace for music. As of April 2020, iTunes offered 60 million songs, 2.2 million apps, 25,00 ...
, web software such as the
Adobe Flash Player
Adobe Flash Player (known in Internet Explorer, Firefox, and Google Chrome as Shockwave Flash) is Software, computer software for viewing multimedia contents, executing rich Internet applications, and streaming media, streaming audio and vide ...
and
Microsoft Silverlight
Microsoft Silverlight is a discontinued application framework designed for writing and running rich web applications, similar to Adobe Inc., Adobe's Run time environment, runtime, Adobe Flash. A plugin for Silverlight is still available for a v ...
, and also various
HDTV
High-definition television (HD or HDTV) describes a television system which provides a substantially higher image resolution than the previous generation of technologies. The term has been used since 1936; in more recent times, it refers to the g ...
broadcasts over terrestrial (
Advanced Television Systems Committee standards
Advanced Television Systems Committee (ATSC) standards are an American set of standards for digital television transmission over terrestrial, cable and satellite networks. It is largely a replacement for the analog NTSC standard and, like th ...
,
ISDB-T,
DVB-T
DVB-T, short for Digital Video Broadcasting – Terrestrial, is the DVB European-based consortium standard for the broadcast transmission of digital terrestrial television that was first published in 1997 and first broadcast in Singapore in Febr ...
or
DVB-T2
DVB-T2 is an abbreviation for "Digital Video Broadcasting – Second Generation Terrestrial"; it is the extension of the television standard DVB-T, issued by the consortium DVB, devised for the broadcast transmission of digital terrestrial tele ...
), cable (
DVB-C), and satellite (
DVB-S2).
A main problem for many video coding formats has been
patent
A patent is a type of intellectual property that gives its owner the legal right to exclude others from making, using, or selling an invention for a limited period of time in exchange for publishing an enabling disclosure of the invention."A p ...
s, making it expensive to use or potentially risking a patent lawsuit due to
submarine patents. The motivation behind many recently designed video coding formats such as
Theora
Theora is a free lossy video compression format. It is developed by the Xiph.Org Foundation and distributed without licensing fees alongside their other free and open media projects, including the Vorbis audio format and the Ogg contai ...
,
VP8
VP8 is an open and royalty-free video compression format released by On2 Technologies in 2008.
Initially released as a proprietary successor to On2's previous VP7 format, VP8 was released as an open and royalty-free format in May 2010 after Goo ...
and
VP9
VP9 is an open and royalty-free video coding format developed by Google.
VP9 is the successor to VP8 and competes mainly with MPEG's High Efficiency Video Coding (HEVC/H.265).
At first, VP9 was mainly used on Google's video platform YouTube. ...
have been to create a (
libre
Libre may refer to:
Computing
* Libre software, free software
* Libre Computer Project, developer of open-hardware single-board computers
Medicine
*FreeStyle Libre, a glucose monitoring device
Media
* Libre Times, news site which people can free ...
) video coding standard covered only by royalty-free patents. Patent status has also been a major point of contention for the choice of which video formats the mainstream
web browser
A web browser is application software for accessing websites. When a user requests a web page from a particular website, the browser retrieves its files from a web server and then displays the page on the user's screen. Browsers are used on ...
s will support inside the
HTML5 video
The HTML5 specification introduced the video element for the purpose of playing videos, partially replacing the object element. HTML5 video is intended by its creators to become the new standard way to show video on the web, instead of the previo ...
tag.
The current-generation video coding format is
HEVC
High Efficiency Video Coding (HEVC), also known as H.265 and MPEG-H Part 2, is a video compression standard designed as part of the MPEG-H project as a successor to the widely used Advanced Video Coding (AVC, H.264, or MPEG-4 Part 10). In compar ...
(H.265), introduced in 2013. While AVC uses the integer DCT with 4x4 and 8x8 block sizes, HEVC uses integer DCT and
DST
Daylight saving time (DST), also referred to as daylight savings time or simply daylight time (United States, Canada, and Australia), and summer time (United Kingdom, European Union, and others), is the practice of advancing clocks (typicall ...
transforms with varied block sizes between 4x4 and 32x32.
HEVC is heavily patented, with the majority of patents belonging to
Samsung Electronics
Samsung Electronics Co., Ltd. (, sometimes shortened to SEC and stylized as SΛMSUNG) is a South Korean multinational corporation, multinational electronics corporation headquartered in Yeongtong-gu, Suwon, South Korea. It is the pinnacle of ...
,
GE,
NTT and
JVC Kenwood
, stylized as JVCKENWOOD, is a Japanese multinational electronics company headquartered in Yokohama, Japan. It was formed from the merger of Victor Company of Japan, Ltd (JVC) and Kenwood Corporation on October 1, 2008. Upon creation, Haruo Kaw ...
.
It is currently being challenged by the aiming-to-be-freely-licensed
AV1
AOMedia Video 1 (AV1) is an open, royalty-free video coding format initially designed for video transmissions over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia), a consortium founded in 2015 th ...
format. , AVC is by far the most commonly used format for the recording, compression and distribution of video content, used by 91% of video developers, followed by HEVC which is used by 43% of developers.
List of video coding standards
Lossless, lossy, and uncompressed video coding formats
Consumer video is generally compressed using
lossy
In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size ...
video codecs
The following is a list of compression formats and related codecs.
Audio compression formats
Non-compression
* Linear pulse-code modulation (LPCM, generally only described as PCM) is the format for uncompressed audio in media files and it is al ...
, since that results in significantly smaller files than
lossless
Lossless compression is a class of data compression that allows the original data to be perfectly reconstructed from the compressed data with no loss of information. Lossless compression is possible because most real-world data exhibits statistic ...
compression. While there are video coding formats designed explicitly for either lossy or lossless compression, some video coding formats such as
Dirac
Distributed Research using Advanced Computing (DiRAC) is an integrated supercomputing facility used for research in particle physics, astronomy and cosmology in the United Kingdom. DiRAC makes use of multi-core processors and provides a variety o ...
and
H.264
Advanced Video Coding (AVC), also referred to as H.264 or MPEG-4 Part 10, is a video compression standard based on block-oriented, motion-compensated coding. It is by far the most commonly used format for the recording, compression, and distr ...
support both.
Uncompressed video
Uncompressed video is digital video that either has never been compressed or was generated by decompressing previously compressed digital video. It is commonly used by video cameras, video monitors, video recording devices (including general-purp ...
formats, such as ''Clean HDMI'', is a form of lossless video used in some circumstances such as when sending video to a display over a
HDMI
High-Definition Multimedia Interface (HDMI) is a proprietary audio/video interface for transmitting uncompressed video data and compressed or uncompressed digital audio data from an HDMI-compliant source device, such as a display controller, ...
connection. Some high-end cameras can also capture video directly in this format.
Intra-frame video coding formats
Interframe compression complicates editing of an encoded video sequence.
One subclass of relatively simple video coding formats are the
intra-frame
Intra-frame coding is a data compression technique used within a video frame, enabling smaller file sizes and lower bitrates, with little or no loss in quality. Since neighboring pixels within an image are often very similar, rather than storing ...
video formats, such as
DV, in which each frame of the video stream is compressed independently without referring to other frames in the stream, and no attempt is made to take advantage of correlations between successive pictures over time for better compression. One example is
Motion JPEG
Motion JPEG (M-JPEG or MJPEG) is a video compression format in which each video frame or interlaced field of a digital video sequence is compressed separately as a JPEG image.
Originally developed for multimedia PC applications, Motion JPEG ...
, which is simply a sequence of individually
JPEG
JPEG ( ) is a commonly used method of lossy compression for digital images, particularly for those images produced by digital photography. The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and imag ...
-compressed images. This approach is quick and simple, at the expense the encoded video being much larger than a video coding format supporting
Inter frame
An inter frame is a frame in a video compression stream which is expressed in terms of one or more neighboring frames. The "inter" part of the term refers to the use of ''Inter frame prediction''. This kind of prediction tries to take advantage fro ...
coding.
Because interframe compression copies data from one frame to another, if the original frame is simply cut out (or lost in transmission), the following frames cannot be reconstructed properly. Making 'cuts' in intraframe-compressed video while
video editing
Video editing is the manipulation and arrangement of video shots. Video editing is used to structure and present all video information, including films and television shows, video advertisements and video essays. Video editing has been dramatical ...
is almost as easy as editing uncompressed video: one finds the beginning and ending of each frame, and simply copies bit-for-bit each frame that one wants to keep, and discards the frames one doesn't want. Another difference between intraframe and interframe compression is that, with intraframe systems, each frame uses a similar amount of data. In most interframe systems, certain frames (such as "
I frames" in
MPEG-2
MPEG-2 (a.k.a. H.222/H.262 as was defined by the ITU) is a standard for "the generic video coding format, coding of moving pictures and associated audio information". It describes a combination of Lossy compression, lossy video compression and ...
) aren't allowed to copy data from other frames, so they require much more data than other frames nearby.
It is possible to build a computer-based video editor that spots problems caused when I frames are edited out while other frames need them. This has allowed newer formats like
HDV
HDV is a format for recording of high-definition video on DV cassette tape. The format was originally developed by JVC and supported by Sony, Canon, and Sharp. The four companies formed the HDV Consortium in September 2003.
Conceived as an af ...
to be used for editing. However, this process demands a lot more computing power than editing intraframe compressed video with the same picture quality. But, this compression is not very effective to use for any audio format.
Profiles and levels
A video coding format can define optional restrictions to encoded video, called
profile
Profile or profiles may refer to:
Art, entertainment and media Music
* ''Profile'' (Jan Akkerman album), 1973
* ''Profile'' (Githead album), 2005
* ''Profile'' (Pat Donohue album), 2005
* ''Profile'' (Duke Pearson album), 1959
* '' ''Profi ...
s and levels. It is possible to have a decoder which only supports decoding a subset of profiles and levels of a given video format, for example to make the decoder program/hardware smaller, simpler, or faster.
A ''profile'' restricts which encoding techniques are allowed. For example, the H.264 format includes the profiles ''baseline'', ''main'' and ''high'' (and others). While
P-slices (which can be predicted based on preceding slices) are supported in all profiles,
B-slices (which can be predicted based on both preceding and following slices) are supported in the ''main'' and ''high'' profiles but not in ''baseline''.
A ''level'' is a restriction on parameters such as maximum resolution and data rates.
See also
*
Comparison of video container formats
*
Data compression#Video
*
Display resolution
The display resolution or display modes of a digital television, computer monitor or display device is the number of distinct pixels in each dimension that can be displayed. It can be an ambiguous term especially as the displayed resolution is ...
*
List of video compression formats
*
Video file format
A video file format is a type of file format for storing digital video data on a computer system. Video is almost always stored using lossy compression to reduce the file size.
A video file normally consists of a container (e.g. in the Matroska ...
Notes
References
{{Reflist
Video formats