scientific
Science is a systematic endeavor that builds and organizes knowledge in the form of testable explanations and predictions about the universe.
Science may be as old as the human species, and some of the earliest archeological evidence for ...
lossy
In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size ...
audio
Audio most commonly refers to sound, as it is transmitted in signal form. It may also refer to:
Sound
* Audio signal, an electrical representation of sound
*Audio frequency, a frequency in the audio spectrum
* Digital audio, representation of sou ...
codec
A codec is a device or computer program that encodes or decodes a data stream or signal. ''Codec'' is a portmanteau of coder/decoder.
In electronic communications, an endec is a device that acts as both an encoder and a decoder on a signal or ...
s, usually with respect to perceived
fidelity
Fidelity is the quality of faithfulness or loyalty. Its original meaning regarded duty in a broader sense than the related concept of ''fealty''. Both derive from the Latin word ''fidēlis'', meaning "faithful or loyal". In the City of London fin ...
or compression efficiency.
Most tests take the form of a
double-blind
In a blind or blinded experiment, information which may influence the participants of the experiment is withheld until after the experiment is complete. Good blinding can reduce or eliminate experimental biases that arise from a participants' expec ...
comparison. Commonly used methods are known as " ABX" or "ABC/HR" or "
MUSHRA
MUSHRA stands for Multiple Stimuli with Hidden Reference and Anchor and is a methodology for conducting a codec listening test to evaluate the perceived quality of the output from lossy audio compression algorithms. It is defined by ITU-R recommen ...
". There are various software packages available for individuals to perform this type of testing themselves with minimal assistance.
Testing methods
ABX test
In an ABX test, the listener has to identify an unknown sample X as being A or B, with A (usually the original) and B (usually the encoded version) available for reference. The outcome of a test must be statistically significant. This setup ensures that the listener is not biased by his/her expectations, and that the outcome is not likely to be the result of chance. If sample X cannot be determined reliably with a low p-value in a predetermined number of trials, then the
null hypothesis
In scientific research, the null hypothesis (often denoted ''H''0) is the claim that no difference or relationship exists between two sets of data or variables being analyzed. The null hypothesis is that any experimentally observed difference is d ...
cannot be rejected and it cannot be proved that there is a perceptible difference between samples A and B. This usually indicates that the encoded version will actually be ''
transparent
Transparency, transparence or transparent most often refer to:
* Transparency (optics), the physical property of allowing the transmission of light through a material
They may also refer to:
Literal uses
* Transparency (photography), a still, ...
'' to the listener.
ABC/HR test
In an ABC/HR test, C is the original which is always available for reference. A and B are the original and the encoded version in randomized order. The listener must first distinguish the encoded version from the original (which is the Hidden Reference that the "HR" in ABC/HR stands for), prior to assigning a score as a subjective judgment of the quality. Different encoded versions can be compared against each other using these scores.
MUSHRA
In MUSHRA (MUltiple Stimuli with Hidden Reference and Anchor), the listener is presented with the reference (labeled as such), a certain number of test samples, a hidden version of the reference and one or more anchors. The purpose of the anchor(s) is to make the scale be closer to an "absolute scale", making sure that minor artifacts are not rated as having very bad quality.
Results
Many double-blind music listening tests have been carried out. The following table lists the results of several listening tests that have been published online. To obtain meaningful results, listening tests must compare codecs' performance at similar or identical bitrates, since the audio quality produced by any
lossy
In information technology, lossy compression or irreversible compression is the class of data compression methods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size ...
encoder will be trivially improved by increasing the bitrate. If listeners cannot consistently distinguish a lossy encoder's output from the uncompressed original audio, then it may be concluded that the codec has achieved transparency.
Popular formats compared in these tests include
MP3
MP3 (formally MPEG-1 Audio Layer III or MPEG-2 Audio Layer III) is a coding format for digital audio developed largely by the Fraunhofer Society in Germany, with support from other digital scientists in the United States and elsewhere. Origin ...
extensions
Extension, extend or extended may refer to:
Mathematics
Logic or set theory
* Axiom of extensionality
* Extensible cardinal
* Extension (model theory)
* Extension (predicate logic), the set of tuples of values that satisfy the predicate
* E ...
),
Vorbis
Vorbis is a free and open-source software project headed by the Xiph.Org Foundation. The project produces an audio coding format and software reference encoder/decoder (codec) for lossy audio compression. Vorbis is most commonly used in conj ...
,
Musepack
Musepack or MPC is an open source lossy audio codec, specifically optimized for transparent compression of stereo audio at bitrates of 160–180 (manual set allows bitrates up to 320) kbit/s. It was formerly known as MPEGplus, MPEG+ or MP+ ...
mp3PRO
mp3PRO is an unmaintained proprietary audio compression codec that combines the MP3 audio format with the spectral band replication (SBR) compression method. At the time it was developed it could reduce the size of a stereo MP3 by as much as 50% ...
formats appear in some tests, despite much lower adoption . Many encoder and decoder implementations (both
Comparison of audio coding formats
The following tables compare general and technical information for a variety of audio coding formats.
For listening tests comparing the perceived audio quality of audio formats and codecs, see the article Codec listening test.
General informati ...
Audible artifact
In sound and music production, sonic artifact, or simply artifact, refers to sonic material that is accidental or unwanted, resulting from the editing or manipulation of a sound.
Types
Because there are always technical restrictions in the way a ...
s
*
Noise reduction system
Noise reduction is the process of removing noise from a signal. Noise reduction techniques exist for audio and images. Noise reduction algorithms may distort the signal to some degree. Noise rejection is the ability of a circuit to isolate an u ...
audiophile
An audiophile is a person who is enthusiastic about high-fidelity sound reproduction. An audiophile seeks to reproduce the sound of a piece of recorded music or a live musical performance, typically inside closed headphones, In-ear monitors, open ...
site, host of most non-commercial ABX testing
SoundExpert Continuous blind listening tests of codecs over the internet
Digital audio