Digital puppetry is the manipulation and performance of digitally animated 2D or 3D figures and objects in a virtual environment that are rendered in real time by computers. It is most commonly used in
filmmaking
Filmmaking (film production) is the process by which a motion picture is produced. Filmmaking involves a number of complex and discrete stages, starting with an initial story, idea, or commission. It then continues through screenwriting, cast ...
and
television production
A television show – or simply TV show – is any content produced for viewing on a television set which can be broadcast via over-the-air, satellite, or cable, excluding breaking news, advertisements, or trailers that are typically placed be ...
, but has also been used in interactive
theme park
An amusement park is a park that features various attractions, such as rides and games, as well as other events for entertainment purposes. A theme park is a type of amusement park that bases its structures and attractions around a central ...
attractions and live
theatre
Theatre or theater is a collaborative form of performing art that uses live performers, usually actors or actresses, to present the experience of a real or imagined event before a live audience in a specific place, often a stage. The perform ...
.
The exact definition of what is and is not digital puppetry is subject to debate among puppeteers and
computer graphics
Computer graphics deals with generating images with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, cell phone and computer displays, and many specialized applications. A great de ...
designers, but it is generally agreed that digital puppetry differs from conventional
computer animation
Computer animation is the process used for digitally generating animations. The more general term computer-generated imagery (CGI) encompasses both static scenes (still images) and dynamic images (moving images), while computer animation refe ...
in that it involves performing characters in real time, rather than animating them frame by frame.
Digital puppetry is closely associated with
character animation
Character animation is a specialized area of the animation process, which involves bringing animated s to life. The role of a character animator is analogous to that of a film or stage actor and character animators are often said to be "actors wit ...
,
motion capture
Motion capture (sometimes referred as mo-cap or mocap, for short) is the process of recording the movement of objects or people. It is used in military, entertainment, sports, medical applications, and for validation of computer vision and robo ...
technologies, and
3D animation
Animation is a method by which still figures are manipulated to appear as moving images. In traditional animation, images are drawn or painted by hand on transparent celluloid sheets to be photographed and exhibited on film. Today, most anima ...
, as well as
skeletal animation
Skeletal animation or rigging is a technique in computer animation in which a character (or other articulated object) is represented in two parts: a surface representation used to draw the character (called the ''mesh'' or ''skin'') and a hierarc ...
. Digital puppetry is also known as virtual puppetry, performance animation, living animation, aniforms, live animation and real-time animation (although the latter also refers to animation generated by computer game engines).
Machinima
Machinima, originally machinema () is the use of real-time computer graphics engines to create a cinematic production. Most often, video games are used to generate the computer animation. The word "machinima" is a portmanteau of the words ''ma ...
is another form of digital puppetry, and Machinima performers are increasingly being identified as
puppeteers
A puppeteer is a person who manipulates an inanimate object, called a puppet, to create the illusion that the puppet is alive. The puppet is often shaped like a human, animal, or legendary creature. The puppeteer may be visible to or hidden from ...
.
History and usage
Early experiments
One of the earliest pioneers of digital puppetry was
Lee Harrison III Lee Harrison III (1929–1998) was a pioneer in analog electronic animation. Harrison received two bachelor's degrees from Washington University in St. Louis (Bachelor of Fine Arts in 1952 and a Bachelor of Science in Mechanical Engineering in 1959) ...
. He conducted experiments in the early 1960s that animated figures using analog circuits and a cathode ray tube. Harrison rigged up a body suit with potentiometers and created the first working motion capture rig, animating 3D figures in real-time on his CRT screen. He made several short films with this system, which he called ANIMAC. Among the earliest examples of digital puppets produced with the system included a character called "Mr. Computer Image" who was controlled by a combination of the ANIMAC's body control rig and an early form of voice-controlled automatic lip sync.
Waldo C. Graphic
Perhaps the first truly commercially successful example of a digitally animated figure being performed and rendered in real-time is
Waldo C. Graphic, a character created in 1988 by
Jim Henson
James Maury Henson (September 24, 1936 – May 16, 1990) was an American puppeteer, animator, cartoonist, actor, inventor, and filmmaker who achieved worldwide notice as the creator of The Muppets and '' Fraggle Rock'' (1983–1987) and ...
and Pacific Data Images for the Muppet television series ''
The Jim Henson Hour
''The Jim Henson Hour'' is a television series that aired on NBC in 1989. It was developed as a showcase for The Jim Henson Company's various puppet creations, including the Muppet characters.
Nine of the twelve episodes produced aired on NBC bef ...
''. Henson had used the
Scanimate
{{about, the computer animation system, the animation effect, Barrier grid animation and stereography
Scanimate is an analog computer animation (video synthesizer) system developed from the late 1960s to the 1980s by Computer Image Corporation of ...
system to generate a digital version of his Nobody character in real-time for the television series
Sesame Street
''Sesame Street'' is an American educational children's television series that combines live-action, sketch comedy, animation and puppetry. It is produced by Sesame Workshop (known as the Children's Television Workshop until June 2000) a ...
as early as 1970 and Waldo grew out of experiments Henson conducted to create a computer generated version of his character
Kermit the Frog
Kermit the Frog is a Muppet character created and originally performed by Jim Henson. Introduced in 1955, Kermit serves as the everyman protagonist of numerous Muppet productions, most notably ''Sesame Street'' and ''The Muppet Show'', as well ...
in 1985.
Waldo's strength as a computer generated puppet was that he could be controlled by a single puppeteer (
Steve Whitmire
Steven Lawrence Whitmire (born September 24, 1959) is an American puppeteer, known primarily for his work on ''The Muppets'' and ''Sesame Street''. Beginning his involvement with the Muppets in 1978, Whitmire inherited the roles of Ernie and Kerm ...
) in real-time in concert with conventional puppets. The computer image of Waldo was mixed with the video feed of the camera focused on physical puppets so that all of the puppeteers in a scene could perform together. (It was already standard Muppeteering practice to use monitors while performing, so the use of a virtual puppet did not significantly increase the complexity of the system.) Afterwards, in post production, PDI re-rendered Waldo in full resolution, adding a few dynamic elements on top of the performed motion.
Waldo C. Graphic can be seen today in ''
Jim Henson's Muppet*Vision 3D'' at
Disney's Hollywood Studios
Disney's Hollywood Studios is a theme park at the Walt Disney World Resort in Bay Lake, Florida, near Orlando. It is owned and operated by The Walt Disney Company through its Parks, Experiences and Products division. Based on a concept by Ma ...
in
Lake Buena Vista, Florida
Lake Buena Vista () is a city in Orange County, Florida, United States. It is mostly known for being the mailing address for Walt Disney World—although almost all of the resort facilities, including all four theme parks, are physically located i ...
.
Mike Normal
Another significant development in digital puppetry in 1988 was Mike Normal, which Brad DeGraf and partner Michael Wahrman developed to show off the real-time capabilities of
Silicon Graphics
Silicon Graphics, Inc. (stylized as SiliconGraphics before 1999, later rebranded SGI, historically known as Silicon Graphics Computer Systems or SGCS) was an American high-performance computing manufacturer, producing computer hardware and soft ...
' then-new 4D series workstations. Unveiled at the 1988
SIGGRAPH
SIGGRAPH (Special Interest Group on Computer Graphics and Interactive Techniques) is an annual conference on computer graphics (CG) organized by the ACM SIGGRAPH, starting in 1974. The main conference is held in North America; SIGGRAPH Asia ...
convention, it was the first live performance of a
digital character
Character animation is a specialized area of the animation process, which involves bringing animated s to life. The role of a character animator is analogous to that of a film or stage actor and character animators are often said to be "actors wi ...
. Mike was a sophisticated talking head driven by a specially built controller that allowed a single puppeteer to control many parameters of the character's face, including mouth, eyes, expression, and head position.
The system developed by deGraf/Wahrman to perform Mike Normal was later used to create a representation of the villain Cain in the motion picture ''
RoboCop 2
''RoboCop 2'' is a 1990 American science fiction action film directed by Irvin Kershner and written by Frank Miller and Walon Green. It stars Peter Weller, Nancy Allen, Dan O'Herlihy, Belinda Bauer, Tom Noonan and Gabriel Damon. It is the ...
'', which is believed to be the first example of digital puppetry being used to create a character in a full-length motion picture.
Trey Stokes
Trey Stokes (born 1960 in Nashville, Tennessee) is an American filmmaker and puppeteer, best known for his '' Star Wars'' parody series '' Pink Five'', and his puppeteering work on various movie, TV, and motion-ride projects.
Early career
Stoke ...
was the puppeteer for both Mike Normal's SIGGRAPH debut and ''Robocop II''.
Sesame Street: Elmo's World
One of the most widely seen successful examples of digital puppetry in a TV series is Sesame Street's "Elmo's World" segment. A set of furniture characters were created with CGI, to perform simultaneously with Elmo and other real puppets. They were performed in real time on set, simultaneously with live puppet performances. As with the example of Henson's Waldo C. Graphic above, the digital puppets' video feed was seen live by both the digital and physical puppet performers, allowing the digital and physical characters to interact.
Disney theme parks
Walt Disney Imagineering
Walt Disney Imagineering Research & Development, Inc., commonly referred to as Imagineering, is the research and development arm of The Walt Disney Company, responsible for the creation, design, and construction of Disney theme parks and attra ...
has also been an important innovator in the field of digital puppetry, developing new technologies to enable visitors to Disney theme parks to interact with some of the company's famous animated characters. In 2004, they used digital puppetry techniques to create the
Turtle Talk with Crush
Turtle Talk with Crush is an interactive talk show type attraction that has appeared at several of the Disney theme parks. It first opened on November 16, 2004 at The Living Seas pavilion (later renamed The Seas with Nemo & Friends) at Epcot and ...
attractions at
Epcot
Epcot, stylized in all uppercase as EPCOT, is a theme park at the Walt Disney World Resort in Bay Lake, Florida. It is owned and operated by The Walt Disney Company through its Parks, Experiences and Products division. Inspired by an unreal ...
and
Disney California Adventure Park
Disney California Adventure Park, commonly referred to as California Adventure or by its acronym DCA, is a theme park located at the Disneyland Resort in Anaheim, California. It is owned and operated by The Walt Disney Company through its Park ...
. In the attraction, a hidden puppeteer performs and voices a digital puppet of Crush, the laid-back sea turtle from ''
Finding Nemo
''Finding Nemo'' is a 2003 American computer-animated comedy-drama adventure film produced by Pixar Animation Studios and released by Walt Disney Pictures. Directed by Andrew Stanton with co-direction by Lee Unkrich, the screenplay was writ ...
'', on a large rear-projection screen. To the audience Crush appears to be swimming inside an aquarium and engages in unscripted, real-time conversations with theme park guests.
Disney Imagineering continued its use of digital puppetry with the
Monsters Inc. Laugh Floor, a new attraction in
Tomorrowland
Tomorrowland is one of the many themed lands featured at all of the Magic Kingdom styled Disney theme parks around the world owned or licensed by The Walt Disney Company. Each version of the land is different and features numerous attractions t ...
at
Walt Disney World's Magic Kingdom
Magic Kingdom Park, previously known as Walt Disney World Magic Kingdom (1971–1994) and The Magic Kingdom (1994–2017), is a theme park at the Walt Disney World Resort in Bay Lake, Florida, near Orlando, Florida. Owned and operated by The W ...
, which opened in the spring of 2007. Guests temporarily enter the "monster world" introduced in Disney and
Pixar's 2001 film, ''
Monsters, Inc.
''Monsters, Inc.'' (also known as ''Monsters, Incorporated'') is a 2001 American computer-animated Monster movie, monster comedy film produced by Pixar, Pixar Animation Studios for Walt Disney Pictures. Featuring the voices of John Goodman, B ...
'', where they are entertained by
Mike Wazowski
Michael "Mike" Wazowski is a fictional character who appears in Disney/Pixar's Monsters Inc. franchise. He is a green one-eyed round monster with two arms, legs, and small horns. In the films, Mike is one of the two protagonists, alongside Jam ...
and other monster comedians who are attempting to capture laughter, which they convert to energy. Much like Turtle Talk, the puppeteers interact with guests in real-time, just as a real-life comedian would interact with his/her audience.
Disney also uses digital puppetry techniques in
Stitch Encounter, which opened in 2006 at the
Hong Kong Disneyland
Hong Kong Disneyland () (local nickname ''HKDL''; also known as HK Disneyland) is a theme park located on reclaimed land in Penny's Bay, Lantau Island. It is located inside the Hong Kong Disneyland Resort and it is owned and managed by Hong Ko ...
park. Disney has another version of the same attraction in
Disneyland Resort Paris
Disneyland Paris is an entertainment resort in Chessy, France, east of Paris. It encompasses two theme parks, resort hotels, Disney Nature Resorts, a shopping, dining and entertainment complex, and a golf course. Disneyland Park is the origin ...
called Stitch Live!
Military Simulation & Training
Since 2014, the
United States Army
The United States Army (USA) is the land service branch of the United States Armed Forces. It is one of the eight U.S. uniformed services, and is designated as the Army of the United States in the U.S. Constitution.Article II, section 2, cla ...
's Program Executive Office for
Simulation
A simulation is the imitation of the operation of a real-world process or system over time. Simulations require the use of Conceptual model, models; the model represents the key characteristics or behaviors of the selected system or proc ...
, Training, Research, and Instrumentation (PEO STRI), a division of
US Army Simulation and Training Technology Center (STTC), has been experimenting with digital puppetry as a method of teaching advanced
situational awareness
Situational awareness or situation awareness (SA) is the perception of environmental elements and events with respect to time or space, the comprehension of their meaning, and the projection of their future status. An alternative definition is tha ...
for infantry squads. A single
improvisor using motion capture technology from Organic Motion Inc interacted with squads through the medium of several different life-sized
avatars
Avatar (, ; ), is a concept within Hinduism that in Sanskrit literally means "descent". It signifies the material appearance or incarnation of a powerful deity, goddess or spirit on Earth. The relative verb to "alight, to make one's appearance ...
of varying ages and genders that were projected onto multiple walls throughout an
urban operations
Urban warfare is combat conducted in urban areas such as towns and cities. Urban combat differs from combat in the open at both the operational and the tactical levels. Complicating factors in urban warfare include the presence of civilians and ...
training facility. The motion capture technology was paired with real-time voice shifting in order to achieve the effect.
Types of digital puppetry
Waldo puppetry
A digital puppet is controlled onscreen in real time by a puppeteer who uses a telemetric input device known as a Waldo (after the short story "
Waldo
Waldo may refer to:
People
* Waldo (given name), a list of people and fictional characters
* Waldo (surname), a list of people
* Waldo (footballer) (1934-2019), full name Waldo Machado da Silva, Brazilian footballer
Places Canada
* Waldo, ...
" by
Robert A. Heinlein
Robert Anson Heinlein (; July 7, 1907 – May 8, 1988) was an American science fiction author, aeronautical engineer, and naval officer. Sometimes called the "dean of science fiction writers", he was among the first to emphasize scientific accu ...
which features a man who invents and uses such devices), connected to the computer. The X-Y-Z axis movement of the input device causes the digital puppet to move correspondingly.
Computer facial animation
Computer facial animation is primarily an area of
computer graphics
Computer graphics deals with generating images with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, cell phone and computer displays, and many specialized applications. A great de ...
that encapsulates methods and techniques for generating and animating images or models of a character face. The importance of
human faces
The face is the front of an animal's head that features the eyes, nose and mouth, and through which animals express many of their emotions. The face is crucial for human identity, and damage such as scarring or developmental deformities may aff ...
in
verbal and non-verbal communication and advances in
computer graphics hardware and
software
Software is a set of computer programs and associated documentation and data. This is in contrast to hardware, from which the system is built and which actually performs the work.
At the lowest programming level, executable code consists ...
have caused considerable scientific, technological, and artistic interests in computer facial animation.
Motion capture puppetry/performance animation
An object (puppet) or human body is used as a physical representation of a digital puppet and manipulated by a puppeteer. The movements of the object or body are matched correspondingly by the digital puppet in real-time. Motion capture puppetry is commonly used, for example, by
VTuber
A , or , is an online entertainer who uses a virtual avatar generated using computer graphics. Real-time motion capture software or technology are often—but not always—used to capture movement. The digital trend originated in Japan in the mid- ...
s, who rig digital avatars to correspond to the movements of their heads.
Aniforms
Aniforms is the name of a process for creating simple real-time animation for live audiences or visual media. It was invented by
Morey Bunin
Morey () is a commune in the Saône-et-Loire department in the region of Bourgogne-Franche-Comté in eastern France.
Geography
Morey is located in the hills surrounding the Dheune river and the Canal Du Centre. In addition to the main village, ...
, a
puppeteer
A puppeteer is a person who manipulates an inanimate object, called a puppet, to create the illusion that the puppet is alive. The puppet is often shaped like a human, animal, or legendary creature. The puppeteer may be visible to or hidden from ...
who worked with string
marionettes
A marionette (; french: marionnette, ) is a puppet controlled from above using wires or strings depending on regional variations. A marionette's puppeteer is called a marionettist. Marionettes are operated with the puppeteer hidden or revealed ...
and
hand puppets, and his spouse Charlotte. The distinctive feature of an Aniforms character is that it displays a physical form that appears "animated" on a real or simulated television screen. The technique was used in
television production
A television show – or simply TV show – is any content produced for viewing on a television set which can be broadcast via over-the-air, satellite, or cable, excluding breaking news, advertisements, or trailers that are typically placed be ...
.
Machinima
A production technique that can be used to perform digital puppets. Machinima involves creating
computer-generated imagery
Computer-generated imagery (CGI) is the use of computer graphics to create or contribute to images in art, printed media, video games, simulators, and visual effects in films, television programs, shorts, commercials, and videos. The images may ...
(CGI) using the low-end 3D engines in video games. Players act out scenes in real-time using characters and settings within a game and the resulting footage is recorded and later edited into a finished film.
References
{{Animation
External links
Animata- Free, open source real-time animation software commonly used to create digital puppets.
- Web page about Mike Normal, one of the earliest examples of digital puppetry.
Organic Motion LIVE- Commercial digital puppetry technology currently used for simulation & training purposes.
Puppetry
Computer animation
3D computer graphics
Machinima
User interface techniques