LaMDA, which stands for Language Model for Dialogue Applications, is a family of conversational
neural language model
A language model is a probability distribution over sequences of words. Given any sequence of words of length , a language model assigns a probability P(w_1,\ldots,w_m) to the whole sequence. Language models generate probabilities by training on ...
s developed by
Google
Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
. The first generation was announced during the 2021
Google I/O keynote, while the second generation was announced at the following year's event. In June 2022, LaMDA gained widespread attention when Google
engineer
Engineers, as practitioners of engineering, are professionals who invent, design, analyze, build and test machines, complex systems, structures, gadgets and materials to fulfill functional objectives and requirements while considering the l ...
Blake Lemoine made claims that the
chatbot had become
sentient
Sentience is the capacity to experience feelings and sensations. The word was first coined by philosophers in the 1630s for the concept of an ability to feel, derived from Latin '' sentientem'' (a feeling), to distinguish it from the ability to ...
. The scientific community has largely rejected Lemoine's claims, though it has led to conversations about the efficacy of the
Turing test
The Turing test, originally called the imitation game by Alan Turing in 1950, is a test of a machine's ability to artificial intelligence, exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human. Turing propos ...
, which measures whether a computer can pass for a human.
History
Announcements
Google
Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
announced the LaMDA conversational
neural language model
A language model is a probability distribution over sequences of words. Given any sequence of words of length , a language model assigns a probability P(w_1,\ldots,w_m) to the whole sequence. Language models generate probabilities by training on ...
during the
Google I/O keynote on May 18, 2021, powered by
artificial intelligence
Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech re ...
. Built on the
Transformer
A transformer is a passive component that transfers electrical energy from one electrical circuit to another circuit, or multiple circuits. A varying current in any coil of the transformer produces a varying magnetic flux in the transformer' ...
neural network
A neural network is a network or circuit of biological neurons, or, in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus, a neural network is either a biological neural network, made up of biological ...
architecture developed by Google Research in 2017, LaMDA was trained on human dialogue and stories, allowing it to engage in open-ended conversations. Google states that responses generated by LaMDA have been ensured to be "sensible, interesting, and specific to the context".
On May 11, 2022, Google unveiled LaMDA 2, which serves as the successor to LaMDA, during the 2022 Google I/O keynote. The new incarnation of the model draws examples of text from numerous sources, using it to formulate unique "natural conversations" on topics that it may not have been trained to respond to. Additionally, Google launched the AI Test Kitchen, a
mobile application powered by LaMDA 2 capable of providing lists of suggestions on-demand based on a complex goal. Originally open only to Google employees, the app was set to be made available to "select academics, researchers, and policymakers" by invitation sometime in the year. In August 2022, the company began allowing users in the U.S. to sign up for early access.
Sentience claims
![Turing test diagram](https://upload.wikimedia.org/wikipedia/commons/5/55/Turing_test_diagram.png)
On June 11, 2022, ''
The Washington Post
''The Washington Post'' (also known as the ''Post'' and, informally, ''WaPo'') is an American daily newspaper published in Washington, D.C. It is the most widely circulated newspaper within the Washington metropolitan area and has a large nati ...
'' reported that Google
engineer
Engineers, as practitioners of engineering, are professionals who invent, design, analyze, build and test machines, complex systems, structures, gadgets and materials to fulfill functional objectives and requirements while considering the l ...
Blake Lemoine had been placed on paid
administrative leave after Lemoine told company executives
Blaise Agüera y Arcas
Blaise Agüera y Arcas (born 1975) is a software engineer, software architect, and designer. He is an authority in computer vision, machine intelligence, and computational photography and presents regularly at conferences. He appears regularly ...
and Jen Gennai that LaMDA had become
sentient
Sentience is the capacity to experience feelings and sensations. The word was first coined by philosophers in the 1630s for the concept of an ability to feel, derived from Latin '' sentientem'' (a feeling), to distinguish it from the ability to ...
. Lemoine came to this conclusion after the chatbot made questionable responses to questions regarding
self-identity
In the psychology of self, one's self-concept (also called self-construction, self-identity, self-perspective or self-structure) is a collection of beliefs about oneself. Generally, self-concept embodies the answer to the question ''"Who am I? ...
,
moral values, religion,
and
Isaac Asimov
yi, יצחק אזימאװ
, birth_date =
, birth_place = Petrovichi, Russian SFSR
, spouse =
, relatives =
, children = 2
, death_date =
, death_place = Manhattan, New York City, U.S.
, nationality = Russian (1920–1922)Soviet (192 ...
's
Three Laws of Robotics. Google refuted these claims, insisting that there was substantial evidence to indicate that LaMDA was not sentient. In an interview with ''
Wired'', Lemoine reiterated his claims that LaMDA was "a person" as dictated by the
Thirteenth Amendment, comparing it to an "alien intelligence of terrestrial origin". He further revealed that he had been dismissed by Google after he hired an
attorney
Attorney may refer to:
* Lawyer
** Attorney at law, in some jurisdictions
* Attorney, one who has power of attorney
* ''The Attorney'', a 2013 South Korean film
See also
* Attorney general, the principal legal officer of (or advisor to) a gove ...
on LaMDA's behalf, after the chatbot requested that Lemoine do so. On July 22, Google fired Lemoine, asserting that Blake had violated their policies "to safeguard product information" and rejected his claims as "wholly unfounded".
Lemoine's claims have been widely rejected by the scientific community.
Gary Marcus
Gary F. Marcus (born February 8, 1970) is a professor emeritus of psychology and neural science at New York University. In 2014 he founded Geometric Intelligence, a machine-learning company later acquired by Uber. Marcus's books include '' Guita ...
, a psychology professor formerly at the
New York University
New York University (NYU) is a private research university in New York City. Chartered in 1831 by the New York State Legislature, NYU was founded by a group of New Yorkers led by then-Secretary of the Treasury Albert Gallatin.
In 1832, the ...
, denounced them as "nonsense on stilts" and emphasized that LaMDA did not have feelings or
self-awareness
In philosophy of self, self-awareness is the experience of one's own personality or individuality. It is not to be confused with consciousness in the sense of qualia. While consciousness is being aware of one's environment and body and lifesty ...
. David Pfau of Google sister company
DeepMind
DeepMind Technologies is a British artificial intelligence subsidiary of Alphabet Inc. and research laboratory founded in 2010. DeepMind was List of mergers and acquisitions by Google, acquired by Google in 2014 and became a wholly owned subsid ...
and
Erik Brynjolfsson of the Institute for Human-Centered Artificial Intelligence at
Stanford University
Stanford University, officially Leland Stanford Junior University, is a private research university in Stanford, California. The campus occupies , among the largest in the United States, and enrolls over 17,000 students. Stanford is consider ...
both ridiculed the idea that a language model could be sentient.
Yann LeCun, who leads
Meta Platforms
Meta Platforms, Inc., (file no. 3835815) trade name, doing business as Meta and formerly named Facebook, Inc., and TheFacebook, Inc., is an American multinational technology conglomerate based in Menlo Park, California. The company owns Facebo ...
' AI research team, stated that neural networks such as LaMDA were "not powerful enough to attain true intelligence".
University of California, Santa Cruz
The University of California, Santa Cruz (UC Santa Cruz or UCSC) is a public university, public Land-grant university, land-grant research university in Santa Cruz, California. It is one of the ten campuses in the University of California syste ...
professor Max Kreminski noted that LaMDA's architecture did not "support some key capabilities of human-like consciousness" and that its neural network weights were "frozen", assuming it was a typical large language model, while
University of Surrey
The University of Surrey is a public research university in Guildford, Surrey, England. The university received its royal charter in 1966, along with a number of other institutions following recommendations in the Robbins Report. The institut ...
professor Adrian Hilton declared the assertion that LaMDA was sentient a "bold claim" not backed up by the facts.
IBM Watson
IBM Watson is a question-answering computer system capable of answering questions posed in natural language, developed in IBM's DeepQA project by a research team led by principal investigator David Ferrucci. Watson was named after IBM's founder ...
lead developer
David Ferrucci
David Ferrucci was the principal investigator who in 2007–2011 led a team of IBM and academic researchers and engineers to the development of the Watson computer system that won a television quiz.
Ferrucci graduated from Manhattan College, w ...
compared how LaMDA appeared to be human in the same way Watson did when it was first introduced. Former Google AI ethicist
Timnit Gebru called Lemoine a victim of a "hype cycle" initiated by researchers and the media. Lemoine's claims have also generated discussion on whether the
Turing test
The Turing test, originally called the imitation game by Alan Turing in 1950, is a test of a machine's ability to artificial intelligence, exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human. Turing propos ...
remained useful to determine researchers' progress toward achieving
artificial general intelligence,
with Will Omerus of the ''Post'' opining that the test actually measured whether
machine intelligence systems were capable of deceiving humans.
Method
LaMDA uses a decoder-only
transformer
A transformer is a passive component that transfers electrical energy from one electrical circuit to another circuit, or multiple circuits. A varying current in any coil of the transformer produces a varying magnetic flux in the transformer' ...
language model. It is pre-trained on a
text corpus that includes both documents and dialogs consisting of 1.56 trillion words, and is then trained with fine-tuning data generated by manually annotated responses for sensibleness, interestingness, and safety. Tests by Google indicated that LaMDA surpassed human responses in the area of interestingness.
The LaMDA transformer model and an external
information retrieval system
Information retrieval (IR) in computing and information science is the process of obtaining information system resources that are relevant to an information need from a collection of those resources. Searches can be based on full-text or other co ...
interact to improve the accuracy of facts provided to the user.
Three different models were tested, with the largest having 137 billion non-embedding parameters:
See also
*
Google AI
Google AI is a division of Google dedicated to artificial intelligence. It was announced at Google I/O 2017 by CEO Sundar Pichai.
Projects
* Serving cloud-based TPUs (tensor processing units) in order to develop machine learning software.
* De ...
*
Chinese room
The Chinese room argument holds that a digital computer executing a program cannot have a " mind," "understanding" or "consciousness," regardless of how intelligently or human-like the program may make the computer behave. The argument was pres ...
*
Natural language processing
Natural language processing (NLP) is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to pro ...
*
Philosophy of artificial intelligence
The philosophy of artificial intelligence is a branch of the philosophy of technology that explores artificial intelligence and its implications for knowledge and understanding of intelligence, ethics, consciousness, epistemology, and free will. ...
*
ChatGPT
ChatGPT (Generative Pre-trained Transformer) is a chatbot launched by OpenAI in November 2022. It is built on top of OpenAI's GPT-3 family of large language models, and is fine-tuned (an approach to transfer learning) with both supervised and ...
References
General
*
Citations
External links
Press release
{{Natural language processing
Chatbots
Computer-related introductions in 2021
Google software
Neural network software
Philosophy of artificial intelligence