Skinput is an

input technology Input may refer to: Computing * Input (computer science), the act of entering data into a computer or data processing system * Information, any data entered into a computer or data processing system * Input device * Input method * Input port (disa ...

that uses bio-acoustic sensing to localize finger taps on the skin. When augmented with a pico-projector, the device can provide a direct manipulation,

graphical user interface A graphical user interface, or GUI, is a form of user interface that allows user (computing), users to human–computer interaction, interact with electronic devices through Graphics, graphical icon (computing), icons and visual indicators such ...

on the body. The technology was developed by Chris Harrison,

Desney Tan Desney Tan is vice president and managing director of Microsoft Health Futures, a cross-organizational incubation group that serves as Microsoft's Health and Life Science "moonshot factory". He also holds an affiliate faculty appointment in the D ...

and Dan Morris, at

Microsoft Research Microsoft Research (MSR) is the research subsidiary of Microsoft. It was created in 1991 by Richard Rashid, Bill Gates and Nathan Myhrvold with the intent to advance state-of-the-art computing and solve difficult world problems through technologi ...

's Computational User Experiences Group. Skinput represents one way to decouple input from electronic devices with the aim of allowing devices to become smaller without simultaneously shrinking the surface area on which input can be performed. While other systems, like

SixthSense SixthSense is a Gesture recognition, gesture-based wearable computer system developed at MIT Media Lab by Steve Mann in 1994 and 1997 (headworn gestural interface), and 1998 (neckworn version), and further developed by Pranav Mistry (also at MIT ...

have attempted this with computer vision, Skinput employs acoustics, which take advantage of the human body's natural sound conductive properties (e.g.,

bone conduction Bone conduction is the conduction of sound to the inner ear primarily through the bones of the skull, allowing the hearer to perceive audio content even if the ear canal is blocked. Bone conduction transmission occurs constantly as sound waves vi ...

). This allows the body to be annexed as an input surface without the need for the

skin Skin is the layer of usually soft, flexible outer tissue covering the body of a vertebrate animal, with three main functions: protection, regulation, and sensation. Other animal coverings, such as the arthropod exoskeleton, have different ...

to be invasively instrumented with sensors, tracking markers, or other items.

Microsoft Microsoft Corporation is an American multinational corporation and technology company, technology conglomerate headquartered in Redmond, Washington. Founded in 1975, the company became influential in the History of personal computers#The ear ...

has not commented on the future of the projects, other than it is under active development. In 2010, it was reported that this would not appear in commercial devices for at least 2 years.

Operation

Skinput has been publicly demonstrated as an armband, which sits on the

biceps The biceps or biceps brachii (, "two-headed muscle of the arm") is a large muscle that lies on the front of the upper arm between the shoulder and the elbow. Both heads of the muscle arise on the scapula and join to form a single muscle bel ...

. This prototype contains ten small cantilevered Piezo elements configured to be highly resonant, sensitive to frequencies between 25 and 78 Hz. This configuration acts like a mechanical

Fast Fourier transform A fast Fourier transform (FFT) is an algorithm that computes the discrete Fourier transform (DFT) of a sequence, or its inverse (IDFT). A Fourier transform converts a signal from its original domain (often time or space) to a representation in ...

and provides extreme out-of-band noise suppression, allowing the system to function even while the user is in motion. From the upper arm, the sensors can localize finger taps provided to any part of the arm, all the way down to the finger tips, with accuracies in excess of 90% (as high as 96% for five input locations). Classification is driven by a

support vector machine In machine learning, support vector machines (SVMs, also support vector networks) are supervised max-margin models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laborato ...

using a series of time-independent acoustic features that act like a fingerprint. Like

speech recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers. It is also ...

systems, the Skinput recognition engine must be trained on the "sound" of each input location before use. After training, locations can be bound to interactive functions, such as pause/play song, increase/decrease music volume,

speed dial Speed dial was a function available on many telephone systems allowing the user to place a call by pressing a reduced number of keys. This function was particularly useful for phone users who dial certain numbers on a regular basis. In most cas ...

, and menu navigation. With the addition of a pico-projector to the armband, Skinput allows users to interact with a graphical user interface displayed directly on the skin. This enables several interaction modalities, including button-based hierarchical navigation, list-based sliding navigation (similar to an iPod/SmartPhone/MID), text/number entry (e.g., telephone number keypad), and gaming (e.g.,

Tetris ''Tetris'' () is a puzzle video game created in 1985 by Alexey Pajitnov, a Soviet software engineer. In ''Tetris'', falling tetromino shapes must be neatly sorted into a pile; once a horizontal line of the game board is filled in, it disa ...

Frogger is a 1981 arcade action game developed by Konami and published by Sega. In North America, it was distributed by Sega/Gremlin. The object of the game is to direct five frogs to their homes by dodging traffic on a busy road, then crossing a ri ...

)

Demonstrations

Despite being a Microsoft Research internal project, Skinput has been demonstrated publicly several times. The first public appearance was at Microsoft's

TechFest Techfest is the annual science and technology festival of the Indian Institute of Technology Bombay, consisting of social initiatives and outreach programs throughout the year. Started in 1998 with the aim of providing a platform for the Indi ...

2010, where the recognition model was trained live on stage, during the presentation, followed by an interactive walkthrough of a simple mobile application with four modes: music player, email inbox,

, and

voice mail A voicemail system (also known as voice message or voice bank) is a computer-based system that allows callers to leave a Voice recording, recorded message when the recipient has been unable (or unwilling) to answer the Telephone, phone. Calls may ...

. A similar live demo was given at the ACM CHI 2010 conference, where the academic paper received a "Best Paper" award. Attendees were allowed to try the system. Numerous media outlets have covered the technology, with several featuring live demos.

References

{{reflist

External links

ACM Digital Library - Skinput: appropriating the body as an input surface
Computing input devices