Speaker recognition is the identification of a person from characteristics of voices. It is used to answer the question "Who is speaking?" The term voice recognition can refer to ''speaker recognition'' or
speech recognition
Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
. Speaker verification (also called speaker authentication) contrasts with identification, and ''speaker recognition'' differs from '' speaker diarisation'' (recognizing when the same speaker is speaking).
Recognizing the speaker can simplify the task of translating speech in systems that have been trained on specific voices or it can be used to authenticate or verify the identity of a speaker as part of a security process. Speaker recognition has a history dating back some four decades as of 2019 and uses the acoustic features of speech that have been found to differ between individuals. These acoustic patterns reflect both
anatomy
Anatomy () is the branch of biology concerned with the study of the structure of organisms and their parts. Anatomy is a branch of natural science that deals with the structural organization of living things. It is an old science, having its ...
and learned behavioral patterns.
Verification versus identification
There are two major applications of speaker recognition technologies and methodologies. If the speaker claims to be of a certain identity and the voice is used to verify this claim, this is called ''verification'' or ''authentication''. On the other hand, identification is the task of determining an unknown speaker's identity. In a sense, speaker verification is a 1:1 match where one speaker's voice is matched to a particular template whereas speaker identification is a 1:N match where the voice is compared against multiple templates.
From a security perspective, identification is different from verification. Speaker verification is usually employed as a "gatekeeper" in order to provide access to a secure system. These systems operate with the users' knowledge and typically require their cooperation. Speaker identification systems can also be implemented covertly without the user's knowledge to identify talkers in a discussion, alert automated systems of speaker changes, check if a user is already enrolled in a system, etc.
In forensic applications, it is common to first perform a speaker identification process to create a list of "best matches" and then perform a series of verification processes to determine a conclusive match. Working to match the samples from the speaker to the list of best matches helps figure out if they are the same person based on the amount of similarities or differences. The prosecution and defense use this as evidence to determine if the suspect is actually the offender.
Training
One of the earliest training technologies to commercialize was implemented in Worlds of Wonder's 1987 Julie doll. At that point, speaker independence was an intended breakthrough, and systems required a training period. A 1987 ad for the doll carried the tagline "Finally, the doll that understands you." - despite the fact that it was described as a product "which children could train to respond to their voice." The term voice recognition, even a decade later, referred to speaker independence.
Variants of speaker recognition
Each speaker recognition system has two phases: enrollment and verification. During enrollment, the speaker's voice is recorded and typically a number of features are extracted to form a voice print, template, or model. In the verification phase, a speech sample or "utterance" is compared against a previously created voice print. For identification systems, the utterance is compared against multiple voice prints in order to determine the best match(es) while verification systems compare an utterance against a single voice print. Because of the process involved, verification is faster than identification.
Speaker recognition systems fall into two categories: text-dependent and text-independent. Text-dependent recognition requires the text to be the same for both enrollment and verification. In a text-dependent system, prompts can either be common across all speakers (e.g. a common pass phrase) or unique. In addition, the use of shared-secrets (e.g.: passwords and PINs) or knowledge-based information can be employed in order to create a
multi-factor authentication
Multi-factor authentication (MFA; encompassing two-factor authentication, or 2FA, along with similar terms) is an electronic authentication method in which a user is granted access to a website or application only after successfully presenting ...
scenario. Conversely, text-independent systems do not require the use of a specific text. They are most often used for speaker identification as they require very little if any cooperation by the speaker. In this case the text during enrollment and test is different. In fact, the enrollment may happen without the user's knowledge, as in the case for many forensic applications. As text-independent technologies do not compare what was said at enrollment and verification, verification applications tend to also employ
speech recognition
Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
to determine what the user is saying at the point of authentication. In text independent systems both
acoustics
Acoustics is a branch of physics that deals with the study of mechanical waves in gases, liquids, and solids including topics such as vibration, sound, ultrasound and infrasound. A scientist who works in the field of acoustics is an acoustician ...
and
speech analysis
Voice analysis is the study of speech sounds for purposes other than linguistic content, such as in speech recognition. Such studies include mostly medical analysis of the voice (phoniatrics), but also speaker identification. More controversially ...
techniques are used.
Technology
Speaker recognition is a
pattern recognition
Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphi ...
problem. The various technologies used to process and store voice prints include
frequency estimation
In statistical signal processing, the goal of spectral density estimation (SDE) or simply spectral estimation is to estimate the spectral density (also known as the power spectral density) of a signal from a sequence of time samples of the signa ...
,
hidden Markov model
A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process — call it X — with unobservable ("''hidden''") states. As part of the definition, HMM requires that there be an ob ...
s,
Gaussian mixture model
In statistics, a mixture model is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed data set should identify the sub-population to which an individual observatio ...
s,
pattern matching
In computer science, pattern matching is the act of checking a given sequence of tokens for the presence of the constituents of some pattern. In contrast to pattern recognition, the match usually has to be exact: "either it will or will not be ...
algorithms,
neural networks
A neural network is a network or circuit of biological neurons, or, in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus, a neural network is either a biological neural network, made up of biological ...
,
matrix representation
Matrix representation is a method used by a computer language to store matrix (mathematics), matrices of more than one dimension in computer storage, memory.
Fortran and C (programming language), C use different schemes for their native arrays. Fo ...
, vector quantization and
decision trees
A decision tree is a decision support tool that uses a tree-like model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm that only contains condit ...
. For comparing utterances against voice prints, more basic methods like
cosine similarity
In data analysis, cosine similarity is a measure of similarity between two sequences of numbers. For defining it, the sequences are viewed as vectors in an inner product space, and the cosine similarity is defined as the cosine of the angle betwe ...
are traditionally used for their simplicity and performance. Some systems also use "anti-speaker" techniques such as
cohort model The cohort model in psycholinguistics and neurolinguistics is a model of lexical retrieval first proposed by William Marslen-Wilson in the late 1970s. It attempts to describe how visual or auditory input (i.e., hearing or reading a word) is mapped ...
s and world models. Spectral features are predominantly used in representing speaker characteristics.
Linear predictive coding
Linear predictive coding (LPC) is a method used mostly in audio signal processing and speech processing for representing the spectral envelope of a digital signal of speech in compressed form, using the information of a linear predictive model. ...
(LPC) is a
speech coding
Speech coding is an application of data compression of digital audio signals containing speech. Speech coding uses speech-specific parameter estimation using audio signal processing techniques to model the speech signal, combined with generic da ...
method used in speaker recognition and
speech verification
Speech verification uses speech recognition to verify the correctness of the pronounced speech. Speech verification does not try to decode unknown speech from a huge search space, but instead, knowing the expected speech to be pronounced, it attem ...
.
Ambient noise level
In atmospheric sounding and noise pollution, ambient noise level (sometimes called background noise level, reference sound level, or room noise level) is the background sound pressure level at a given location, normally specified as a reference l ...
s can impede both collections of the initial and subsequent voice samples. Noise reduction algorithms can be employed to improve accuracy, but incorrect application can have the opposite effect. Performance degradation can result from changes in behavioural attributes of the voice and from enrollment using one telephone and verification on another telephone. Integration with
two-factor authentication
Multi-factor authentication (MFA; encompassing two-factor authentication, or 2FA, along with similar terms) is an electronic authentication method in which a user is granted access to a website or application only after successfully presenting ...
products is expected to increase. Voice changes due to ageing may impact system performance over time. Some systems adapt the speaker models after each successful verification to capture such long-term changes in the voice, though there is debate regarding the overall security impact imposed by automated adaptation
Legal implications
Due to the introduction of legislation like the
General Data Protection Regulation
The General Data Protection Regulation (GDPR) is a European Union regulation on data protection and privacy in the EU and the European Economic Area (EEA). The GDPR is an important component of EU privacy law and of human rights law, in partic ...
in the
European Union
The European Union (EU) is a supranational political and economic union of member states that are located primarily in Europe. The union has a total area of and an estimated total population of about 447million. The EU has often been des ...
and the
California Consumer Privacy Act
The California Consumer Privacy Act (CCPA) is a state statute intended to enhance privacy rights and consumer protection for residents of California, United States. The bill was passed by the California State Legislature and signed into law by Jer ...
in the United States, there has been much discussion about the use of speaker recognition in the work place. In September 2019 Irish speech recognition developer Soapbox Labs warned about the legal implications that may be involved.
Applications
The first international patent was filed in 1983, coming from the telecommunication research in
CSELT
Centro Studi e Laboratori Telecomunicazioni (CSELT) was an Italian research center for telecommunication based in Torino, the biggest in Italy and one of the most important in Europe. It played a major role internationally especially in the stand ...
(Italy) by Michele Cavazza and
Alberto Ciaramella
Alberto Ciaramella (born 1947) is an Italian computer engineer and scientist. He is notable for extensive pioneering contributions in the field of speech technologies and applied natural language processing, most of them at CSELT and Loquendo, w ...
as a basis for both future telco services to final customers and to improve the noise-reduction techniques across the network.
Between 1996 and 1998, speaker recognition technology was used at the
Scobey–Coronach Border Crossing
The Scobey–Coronach Border Crossing connects the towns of Scobey, Montana and Coronach, Saskatchewan on the Canada–US border. Montana Highway 13 on the American side joins Saskatchewan Highway 36 on the Canadian side. An airport with a grass r ...
to enable enrolled local residents with nothing to declare to cross the
Canada–United States border
The border between Canada and the United States is the longest international border in the world. The terrestrial boundary (including boundaries in the Great Lakes, Atlantic, and Pacific coasts) is long. The land border has two sections: Can ...
when the inspection stations were closed for the night. The system was developed for the U.S.
Immigration and Naturalization Service
The United States Immigration and Naturalization Service (INS) was an agency of the U.S. Department of Labor from 1933 to 1940 and the U.S. Department of Justice from 1940 to 2003.
Referred to by some as former INS and by others as legacy INS, ...
by Voice Strategies of Warren, Michigan.
In May 2013 it was announced that
Barclays Wealth
Barclays Wealth Management serves affluent and high net worth clients through offices across the UK, offering personalised banking, credit, investment management and wealth planning services.
Reported client assets were £202.8 billion (as ...
was to use passive speaker recognition to verify the identity of telephone customers within 30 seconds of normal conversation. The system used had been developed by voice recognition company Nuance (that in 2011 acquired the company
Loquendo
Loquendo is a multinational computer software technology corporation, headquartered in Torino, Italy, that provides speech recognition, speech synthesis, speaker verification and identification applications. Loquendo, which was founded in 2001 u ...
, the spin-off from CSELT itself for speech technology), the company behind Apple's
Siri
Siri ( ) is a virtual assistant that is part of Apple Inc.'s iOS, iPadOS, watchOS, macOS, tvOS, and audioOS operating systems. It uses voice queries, gesture based control, focus-tracking and a natural-language user interface to answer questio ...
technology. A verified voiceprint was to be used to identify callers to the system and the system would in the future be rolled out across the company.
The private banking division of Barclays was the first financial services firm to deploy voice biometrics as the primary means to authenticate customers to their
call center
A call centre ( Commonwealth spelling) or call center (American spelling; see spelling differences) is a managed capability that can be centralised or remote that is used for receiving or transmitting a large volume of enquiries by telephone. ...
s. 93% of customer users had rated the system at "9 out of 10" for speed, ease of use and security.
Speaker recognition may also be used in criminal investigations, such as those of the 2014 executions of, amongst others, James Foley and
Steven Sotloff
Steven Joel Sotloff ( he, סטיבן סוטלוף; May 11, 1983 – September 2, 2014) was an American-Israeli journalist. In August 2013, he was kidnapped in Aleppo, Syria, and held captive by militants from the Islamic State of Iraq and the L ...
.
In February 2016 UK high-street bank
HSBC
HSBC Holdings plc is a British multinational universal bank and financial services holding company. It is the largest bank in Europe by total assets ahead of BNP Paribas, with US$2.953 trillion as of December 2021. In 2021, HSBC had $10.8 tri ...
and its internet-based retail bank
First Direct
First Direct (styled first direct) is a telephone and internet based retail bank division of HSBC Bank plc based in the United Kingdom. First Direct has headquarters in Leeds, England, and has 1.45 million customers. It was awarded Most T ...
announced that it would offer 15 million customers its biometric banking software to access online and phone accounts using their fingerprint or voice.
See also
*
AI effect
:''For the magnitude of effect of a pesticide, see Pesticide application. Of change in farming practices, see Agricultural intensification.''
The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by argui ...
*
Applications of artificial intelligence
Artificial intelligence (AI) has been used in applications to alleviate certain problems throughout industry and academia. AI, like electricity or computers, is a general purpose technology that has a multitude of applications. It has been used ...
*
Speaker diarisation
Speaker diarisation ( or diarization) is the process of partitioning an audio stream containing human speech into homogeneous segments according to the identity of each speaker. It can enhance the readability of an automatic speech transcription b ...
*
Speech recognition
Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the m ...
*
Voice changer
The term voice changer (also known as voice enhancer) refers to a device which can change the tone or pitch of or add distortion to the user's voice, or a combination and vary greatly in price and sophistication. A kazoo/didgeridoo can be used as ...
; Lists
*
List of emerging technologies
This is a list of emerging technologies, in-development technical innovations with significant potential in their applications. The criteria for this list is that the technology must:
# Exist in some way; purely hypothetical technologies can ...
*
Outline of artificial intelligence
The following outline is provided as an overview of and topical guide to artificial intelligence:
Artificial intelligence (AI) – intelligence exhibited by machines or software. It is also the name of the scientific field which studies how to c ...
Notes
References
*Homayoon Beigi (2011), Fundamentals of Speaker Recognition , Springer-Verlag, Berlin, 2011, . "Biometrics from the movies" –National Institute of Standards and Technology
*Elisabeth Zetterholm (2003), ''Voice Imitation. A Phonetic Study of Perceptual Illusions and Acoustic Success'', Phd thesis,
Lund University
, motto = Ad utrumque
, mottoeng = Prepared for both
, established =
, type = Public research university
, budget = SEK 9 billion Enhancement of Speaker Recognition Performance Using Block Level, Relative and Temporal Information of Subband Energies ', PhD thesis,
Indian Institute of Technology Kharagpur
Indian Institute of Technology Kharagpur (IIT Kharagpur) is a public institute of technology established by the Government of India in Kharagpur, West Bengal, India. Established in 1951, the institute is the first of the IITs to be established ...