![Apple iPad Event03](https://upload.wikimedia.org/wikipedia/commons/c/c9/Apple_iPad_Event03.jpg)
A virtual keyboard is a software component that allows the
input of characters without the need for physical keys. The interaction with the virtual
keyboard
Keyboard may refer to:
Text input
* Keyboard, part of a typewriter
* Computer keyboard
** Keyboard layout, the software control of computer keyboards and their mapping
** Keyboard technology, computer keyboard hardware and firmware
Music
* Musi ...
happens mostly via a
touchscreen interface
A touchscreen or touch screen is the assembly of both an input ('touch panel') and output ('display') device. The touch panel is normally layered on the top of an electronic visual display of an information processing system. The display is often ...
, but can also take place in a different form in
virtual or
augmented reality
Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be de ...
.
Types
On a desktop computer, a virtual keyboard might provide an alternative input mechanism for users with
disabilities
Disability is the experience of any condition that makes it more difficult for a person to do certain activities or have equitable access within a given society. Disabilities may be cognitive, developmental, intellectual, mental, physical, se ...
who cannot use a conventional keyboard, or for bi- or
multi-lingual
Multilingualism is the use of more than one language, either by an individual speaker or by a group of speakers. It is believed that multilingual speakers outnumber monolingual speakers in the world's population. More than half of all E ...
users who switch frequently between different character sets or alphabets, which may be confusing over time. Although hardware keyboards are available with dual
keyboard layouts
A keyboard layout is any specific physical, visual or functional arrangement of the keys, legends, or key-meaning associations (respectively) of a computer keyboard, mobile phone, or other computer-controlled typographic keyboard.
is the actua ...
(e.g.
Cyrillic
, bg, кирилица , mk, кирилица , russian: кириллица , sr, ћирилица, uk, кирилиця
, fam1 = Egyptian hieroglyphs
, fam2 = Proto-Sinaitic
, fam3 = Phoenician
, fam4 = G ...
/Latin letters in various national layouts), the on-screen keyboard provides a handy substitute while working at different stations or on laptops, which seldom come with dual layouts.
Virtual keyboards can be categorized by the following aspects:
* Virtual keyboards with touchscreen keyboard layouts or sensing areas
* Character variants, punctuation, and other special characters accessible through a menu and through holding buttons
* Keyboard software may include a number pad feature to facilitate typing numbers.
*
Optically projected keyboard layouts or similar arrangements of "keys" or sensing areas
* Optically detected human hand and finger motions
* Online virtual keyboards for multiple languages that don't require OS settings change
* Depending on which device the keyboard is used (
desktop
A desktop traditionally refers to:
* The surface of a desk (often to distinguish office appliances that fit on a desk, such as photocopiers and printers, from larger equipment covering its own area on the floor)
Desktop may refer to various compu ...
/
mobile
Mobile may refer to:
Places
* Mobile, Alabama, a U.S. port city
* Mobile County, Alabama
* Mobile, Arizona, a small town near Phoenix, U.S.
* Mobile, Newfoundland and Labrador
Arts, entertainment, and media Music Groups and labels
* Mobile ( ...
/
virtual reality
Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), educ ...
/
augmented reality
Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be de ...
)
On the Internet, various
JavaScript
JavaScript (), often abbreviated as JS, is a programming language that is one of the core technologies of the World Wide Web, alongside HTML and CSS. As of 2022, 98% of Website, websites use JavaScript on the Client (computing), client side ...
virtual keyboards have been created, allowing users to type their own languages on foreign keyboards, particularly in Internet cafes.
Multitouch
In computing, multi-touch is technology that enables a surface (a touchpad or touchscreen) to recognize the presence of more than one point of contact with the surface at the same time. The origins of multitouch began at CERN, MIT, University o ...
screens allow the possibility to create virtual
chorded keyboard
A keyset or chorded keyboard (also called a chorded keyset, ''chord keyboard'' or ''chording keyboard'') is a computer input device that allows the user to enter characters or commands formed by pressing several keys together, like playing a " c ...
s for
tablet computers
A tablet computer, commonly shortened to tablet, is a mobile device, typically with a mobile operating system and touchscreen display processing circuitry, and a rechargeable battery in a single, thin and flat package. Tablets, being comput ...
,
touchscreens,
touchpads and
wired glove
A wired glove (also called a dataglove or cyberglove) is an input device for human–computer interaction worn like a glove.
Various sensor technologies are used to capture physical data such as bending of fingers. Often a motion tracker, such a ...
s.
Mobile devices
Virtual keyboards are commonly used as an on-screen input method in devices with no physical keyboard, where there is no room for one, such as a
pocket computer
A pocket computer was a 1980s-era user programmable calculator-sized computer that had fewer screen lines,
Some had only one line and often fewer characters per line, than the Pocket-sized computers introduced beginning in 1989. Manufacturers in ...
,
personal digital assistant
A personal digital assistant (PDA), also known as a handheld PC, is a variety mobile device which functions as a personal information manager. PDAs have been mostly displaced by the widespread adoption of highly capable smartphones, in partic ...
(PDA),
tablet computer
A tablet computer, commonly shortened to tablet, is a mobile device, typically with a mobile operating system and touchscreen display processing circuitry, and a rechargeable battery in a single, thin and flat package. Tablets, being comput ...
or
touchscreen
A touchscreen or touch screen is the assembly of both an input ('touch panel') and output ('display') device. The touch panel is normally layered on the top of an electronic visual display of an information processing system. The display is often ...
-equipped
mobile phone
A mobile phone, cellular phone, cell phone, cellphone, handphone, hand phone or pocket phone, sometimes shortened to simply mobile, cell, or just phone, is a portable telephone that can make and receive calls over a radio frequency link whil ...
. Text is commonly inputted either by tapping a virtual keyboard or finger-tracing. Virtual keyboards are also used as features of
emulation software
In computing, an emulator is Computer hardware, hardware or software that enables one computer system (called the ''host'') to behave like another computer system (called the ''guest''). An emulator typically enables the host system to run so ...
for systems that have fewer buttons than a computer keyboard would have.
Historical development
PDA
![Keyboard on a pocket-PC](https://upload.wikimedia.org/wikipedia/commons/5/53/Keyboard_on_a_pocket-PC.jpg)
The four main approaches to enter text into a
PDA
PDA may refer to:
Science and technology
* Patron-driven acquisition, a mechanism for libraries to purchase books
*Personal digital assistant, a mobile device
* Photodiode array, a type of detector
* Polydiacetylenes, a family of conducting poly ...
were: virtual keyboards operated by a stylus, external USB keyboards, handwritten keyboards, and stroke recognition. Many early PDAs were not primarily focused on virtual keyboards. Microsoft's mobile operating system approach was to simulate a complete functional keyboard, which resulted in a slightly overloaded keyboard layout. The main problem that early PDAs faced was support for multi-touch technology, and as a result, usability problems for the user.
First iPhone
When
Apple
An apple is an edible fruit produced by an apple tree (''Malus domestica''). Apple fruit tree, trees are agriculture, cultivated worldwide and are the most widely grown species in the genus ''Malus''. The tree originated in Central Asia, wh ...
presented the
first iPhone in 2007, the decision not to include a physical keyboard was seen as a detriment to the device. But Apple brought the
multi-touch
In computing, multi-touch is technology that enables a surface (a touchpad or touchscreen) to recognize the presence of more than one somatosensory system, point of contact with the surface at the same time. The origins of multitouch began at CER ...
technology into their new device, which enabled them to overcome the usability problems of PDAs. Apple's virtual keyboard design pattern has become a standard on mobile devices today.
Implementation and use
Both most common mobile operating systems,
Android and
iOS
iOS (formerly iPhone OS) is a mobile operating system created and developed by Apple Inc. exclusively for its hardware. It is the operating system that powers many of the company's mobile devices, including the iPhone; the term also includes ...
, give the developer community the possibility to individually develop custom virtual keyboards.
Android
![Android Keyboard Settings Menue](https://upload.wikimedia.org/wikipedia/commons/4/44/Android_Keyboard_Settings_Menue.png)
The
Android SDK provides a so-called InputMethodService. This service provides a standard implementation of an input method, which final implementations can derive from and customize, enabling the Android development community to implement their own keyboard layouts. The InputMethodService ships with it on Keyboard View. While the InputMethod Service can be used to customize key and gesture inputs, the Keyboard Class loads an
XML
Extensible Markup Language (XML) is a markup language and file format for storing, transmitting, and reconstructing arbitrary data. It defines a set of rules for encoding documents in a format that is both human-readable and machine-readable. T ...
description of a keyboard and stores the attributes of the keys.
As a result, it is possible to install different keyboard versions on an
Android device, and that the keyboard is only an application; a simple activation over the Android settings menu is possible.
iOS
Apple
An apple is an edible fruit produced by an apple tree (''Malus domestica''). Apple fruit tree, trees are agriculture, cultivated worldwide and are the most widely grown species in the genus ''Malus''. The tree originated in Central Asia, wh ...
also provides the possibility for the community to develop custom keyboards, but does not give any access to the
dictionary
A dictionary is a listing of lexemes from the lexicon of one or more specific languages, often arranged alphabetically (or by radical and stroke for ideographic languages), which may include information on definitions, usage, etymologies ...
or general keyboard settings. Further
iOS
iOS (formerly iPhone OS) is a mobile operating system created and developed by Apple Inc. exclusively for its hardware. It is the operating system that powers many of the company's mobile devices, including the iPhone; the term also includes ...
is automatically switching between system and custom keyboards, if the user enters text into the text input field.
The UIInputViewController is the primary view controller for a custom keyboard app extension. This controller provides different methods for the implementation of a custom keyboard, such as a user interface for a custom keyboard, obtaining a supplementary lexicon or changing the primary language of a custom keyboard.
Text entry performance
Next to the classic virtual keyboard implementation
Android,
iOS
iOS (formerly iPhone OS) is a mobile operating system created and developed by Apple Inc. exclusively for its hardware. It is the operating system that powers many of the company's mobile devices, including the iPhone; the term also includes ...
and custom keyboards, such as
SwiftKey
Microsoft SwiftKey is a virtual keyboard app originally developed by TouchType for Android and iOS devices. It was first released for Android in July 2010, followed by an iOS release in September 2014 after Apple implemented third-party keyb ...
for example, are providing different features to improve the usability and the efficiency of their keyboards.
Autocorrection and spelling checker
The Android platform offers a spelling checker framework that offers the possibility to implement and access spell checking in the application itself. The framework is one of the Text Service
API
An application programming interface (API) is a way for two or more computer programs to communicate with each other. It is a type of software interface, offering a service to other pieces of software. A document or standard that describes how ...
s offered by the Android platform. Based on provided text, the session object returns spelling suggestions generated by the spelling checker.
iOS is using the class UITextChecker, an object used to check a string (usually the text of a document) for misspelled words, commonly known as Apple's autocorrection. UITextChecker spell-checks are using a lexicon for a given language. It can be told to ignore specific words when spell-checking a particular document and it can learn new words, which adds those words to the lexicon.
Users may be able to add a custom dictionary of
whitelist
A whitelist, allowlist, or passlist is a mechanism which explicitly allows some identified entities to access a particular privilege, service, mobility, or recognition i.e. it is a list of things allowed when everything is denied by default. It is ...
ed terms that are treated by auto correction as usual words, and specify "aliases" or "text shortcuts", where entering a specified text string causes it to get replaced with a target text string, or the target text string appears as suggestion. The former means they are not replaced with other terms but may be corrected ''to'' from other terms. It may be possible to exclude unwanted existing suggestions.
Word suggestions
Diverse scientific papers at the beginning of the 2000s showed even before the invention of smart phones, that predicting words, based on what the user is typing, is very helpful to increase the typing speed. At the beginning of development of this keyboard feature, prediction was mainly based on static dictionaries.
Google
Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
implemented the predicting method in 2013 in Android 4.4. This development was mainly driven by third party keyboard providers, such as
SwiftKey
Microsoft SwiftKey is a virtual keyboard app originally developed by TouchType for Android and iOS devices. It was first released for Android in July 2010, followed by an iOS release in September 2014 after Apple implemented third-party keyb ...
and
Swype
Swype was a virtual keyboard for touchscreen smartphones and tablets originally developed by Swype Inc., founded in 2002, where the user enters words by sliding a finger or stylus from the first letter of a word to its last letter, lifting only be ...
. Both provide powerful word search engine with corresponding databases. In 2014 Apple presented
iOS 8
iOS 8 is the eighth major release of the iOS mobile operating system developed by Apple Inc., being the successor to iOS 7. It was announced at the company's Worldwide Developers Conference on June 2, 2014, and was released on September 17, ...
which includes a new predictive typing feature called QuickType, which displays word predictions above the keyboard as the user types.
Gesture typing
iOS
iOS (formerly iPhone OS) is a mobile operating system created and developed by Apple Inc. exclusively for its hardware. It is the operating system that powers many of the company's mobile devices, including the iPhone; the term also includes ...
and
Android allow developers to replace its keyboard with their own keyboard apps. This has led to experimentation and new features, like the gesture-typing feature that's made its way into Android's official keyboard after proving itself in third-party keyboards. Research by
Google
Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
itself confirmed that gesture-typing is increasing the typing rate by 22% and is decreasing the error rate near to 0%. Google further showed that the gesture-typing method is also useful on smart watches. Their scientific research is primarily based on research made by I. Scott MacKenzie and papers about modeling finger touch with fitts’ law.
Haptic feedback
Haptic feedback provides for tactile confirmation that a key has been successfully triggered i.e. the user hears and feels a "click" as a key is pressed. Utilising
hysteresis
Hysteresis is the dependence of the state of a system on its history. For example, a magnet may have more than one possible magnetic moment in a given magnetic field, depending on how the field changed in the past. Plots of a single component of ...
, the feel of a physical key can be emulated to an even greater degree. In this case, there is an initial "click" that is heard and felt as the virtual key is pressed down, but then as finger pressure is reduced once the key is triggered, there is a further "unclick" sound and sensation as if a physical key is respringing back to its original unclicked state. This behaviour is explained in Aleks Oniszczak & Scott Mackenzie's 2004 paper "A Comparison of Two Input Methods for Keypads on Mobile Devices" which first introduced haptic feedback with hysteresis on a virtual keyboard.
Special keyboard types
Keyboards are needed in different digital areas. Not only
smartphone
A smartphone is a portable computer device that combines mobile telephone and computing functions into one unit. They are distinguished from feature phones by their stronger hardware capabilities and extensive mobile operating systems, whic ...
s need a virtual keyboards, also devices which create virtual worlds, for example
virtual reality
Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), educ ...
or
augmented reality
Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be de ...
glasses, need to provide text input possibilities.
Optical virtual keyboard
An
optical
Optics is the branch of physics that studies the behaviour and properties of light, including its interactions with matter and the construction of instruments that use or detect it. Optics usually describes the behaviour of visible, ultraviole ...
virtual keyboard was invented and patented by
IBM engineers in 1992.
It optically detects and analyses human hand and finger motions and interprets them as operations on a physically non-existent input device like a surface having painted keys. In that way it allows to emulate unlimited types of manually operated input devices such as a mouse or keyboard. All mechanical input units can be replaced by such virtual devices, optimized for the current application and for the user's physiology maintaining speed, simplicity and unambiguity of manual data input.
Augmented reality keyboards
The basic idea of a virtual keyboard in an
augmented reality
Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be de ...
environment is to give the user a text input possibility. A common approach is to render a flat keyboard into the augmented reality, e.g. using the
Unity
Unity may refer to:
Buildings
* Unity Building, Oregon, Illinois, US; a historic building
* Unity Building (Chicago), Illinois, US; a skyscraper
* Unity Buildings, Liverpool, UK; two buildings in England
* Unity Chapel, Wyoming, Wisconsin, US; a h ...
TouchScreenKeyboard. The
Microsoft
Microsoft Corporation is an American multinational technology corporation producing computer software, consumer electronics, personal computers, and related services headquartered at the Microsoft Redmond campus located in Redmond, Washing ...
HoloLens
Microsoft HoloLens is an augmented reality (AR)/ mixed reality (MR) headset developed and manufactured by Microsoft. HoloLens runs the Windows Mixed Reality platform under the Windows 10 operating system. Some of the positional tracking tech ...
enables the user to point at letters on the keyboard by moving his head.
Another approach was researched by the Korean KJIST U-VR Lab in 2003. Their suggestion was to use wearables to track the finger motion to replace a physical keyboards with virtual ones. They also tried to give an audiovisual feedback to the user, when a key got hit. The basic idea was to give the user a more natural way to enter text, based on what he is used to.
The Magic Leap 1 from
Magic Leap
Magic Leap, Inc. is an American technology company that released a head-mounted virtual retinal display, called ''Magic Leap One'', which superimposes 3D computer-generated imagery over real world objects, by "projecting a digital light field ...
implements a virtual keyboard with augmented reality.
Virtual reality keyboards
The challenges, as in
augmented reality
Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be de ...
, is to give the user the possibility to enter text in a completely virtual environment. One big issue is that most augmented reality systems on the market are not
tracking the hands of the user. So many available system provide the possibility to point at letters.
In September 2016,
Google
Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...
released a virtual keyboard app for their
Daydream
Daydreaming is the stream of consciousness that detaches from current, external tasks when attention drifts to a more personal and internal direction. This phenomenon is common in people's daily life shown by a large-scale study in which partici ...
virtual reality headset. To enter text, the user can point at specific letters with the Daydream controller.
In February 2017,
Logitech
Logitech International S.A. ( ; often shortened to Logi) is a Swiss multinational manufacturer of computer peripherals and software, with headquarters in Lausanne, Switzerland, and Newark, California. The company has offices throughout Europe, ...
presented experimental approach to bring their keyboards into the virtual environment. With the
Vive
Vive may refer to:
* Vive, Viva, a Romance language expression
* ''Vive'' (José José album), 1974
* ''Vive'' (Lucía Méndez album), 2004
*Vive (a cappella group), a cappella group from England, United Kingdom
*ViVe Televisión, a state-owned Ve ...
Tracker and the Logitech G gaming keyboard it is possible to exactly track every finger movement, without wearing any type of glove. 50 of such packages were send to exclusive developers, enabling them, in combination of Logitech's BRIDGE developers kit, to test and experiment with the new technology.
Security considerations
Virtual keyboards may be used in some cases to reduce the risk of
keystroke logging
Keystroke logging, often referred to as keylogging or keyboard capturing, is the action of recording (logging) the keys struck on a keyboard, typically covertly, so that a person using the keyboard is unaware that their actions are being monitored ...
. For example,
Westpac
Westpac Banking Corporation, known simply as Westpac, is an Australian multinational banking and financial services company headquartered at Westpac Place in Sydney, New South Wales.
Established in 1817 as the Bank of New South Wales, it ...
’s online banking service uses a virtual keyboard for the password entry, as does
TreasuryDirect
TreasuryDirect is a website run by the Bureau of the Fiscal Service under the United States Department of the Treasury that allows US individual investors to purchase treasury securities, such as savings bonds, directly from the US government ...
(see picture). It is more difficult for
malware
Malware (a portmanteau for ''malicious software'') is any software intentionally designed to cause disruption to a computer, server, client, or computer network, leak private information, gain unauthorized access to information or systems, depri ...
to monitor the display and mouse to obtain the data entered via the virtual keyboard, than it is to monitor real keystrokes. However it is possible, for example by recording
screenshots
screenshot (also known as screen capture or screen grab) is a digital image that shows the contents of a computer display. A screenshot is created by the operating system or software running on the device powering the display.
Additionally, s ...
at regular intervals or upon each mouse click.
The use of an on-screen keyboard on which the user "types" with mouse clicks can increase the risk of password disclosure by
shoulder surfing, because:
* An observer can typically watch the screen more easily (and less suspiciously) than the keyboard, and see which characters the mouse moves to.
* Some implementations of the on-screen keyboard may give visual feedback of the "key" clicked, e.g. by changing its colour briefly. This makes it much easier for an observer to read the data from the screen. In the worst case, the implementation may leave the
focus
Focus, or its plural form foci may refer to:
Arts
* Focus or Focus Festival, former name of the Adelaide Fringe arts festival in South Australia Film
*''Focus'', a 1962 TV film starring James Whitmore
* ''Focus'' (2001 film), a 2001 film based ...
on the most recently clicked "key" until the next virtual key is clicked, thus allowing the observer time to read each character even after the mouse starts moving to the next character.
* A user may not be able to "point and click" as fast as they could type on a keyboard, thus making it easier for the observer.
See also
*
Caldera SoftKeyboards
DR-WebSpyder is a DOS web browser, mail client and operating system runtime environment that was developed by Caldera UK in 1997. It was based on the DR-DOS operating system and networking components from Novell as well as the Arachne web bro ...
(1997)
*
Ease of Access
Ease of Access Center, formerly Utility Manager, is a component of the Windows NT family of operating systems that enables use of assistive technologies. Utility Manager is included with Windows 2000 and Windows XP. In Windows Vista, Utility Manag ...
*
Finger Touching Cell Phone
The Finger Touching Cell Phone was a concept Mobile phone, cell-phone developed by Samsung Electronics, Samsung and Sunman Kwon at Hong-ik University, South Korea.
Concept
The phone was designed to be worn as a wristband. The phone would project a ...
*
Input method
An input method (or input method editor, commonly abbreviated IME) is an operating system component or program that enables users to generate characters not natively available on their input devices by using sequences of characters (or mouse o ...
*
Mouse keys Mouse keys is a feature of some graphical user interfaces that uses the keyboard (especially numeric keypad)
as a pointing device (usually replacing a mouse). Its roots lie in the earliest days of visual editors when line and column navigation was ...
*
Multi-touch
In computing, multi-touch is technology that enables a surface (a touchpad or touchscreen) to recognize the presence of more than one somatosensory system, point of contact with the surface at the same time. The origins of multitouch began at CER ...
Notes
External links
{{Commons category, Virtual keyboards
Assistive technology
Computer keyboard types
Pointing-device text input
Touch user interfaces