Human–robot Interaction

Human–robot interaction is the study of interactions between humans and robots. It is often referred as HRI by researchers. Human–robot interaction is a multidisciplinary field with contributions from human–computer interaction, artificial intelligence, robotics, natural-language understanding, design, and psychology.

Origins

Human–robot interaction has been a topic of both science fiction and academic speculation even before any robots existed. Because much of active HRI development depends on natural-language processing, many aspects of HRI are continuations of human communications, a field of research which is much older than robotics.

The origin of HRI as a discrete problem was stated by 20th-century author Isaac Asimov in 1941, in his novel ''I, Robot''. He states the Three Laws of Robotics as:
# A robot may not injure a human being or, through inaction, allow a human being to come to harm.
# A robot must obey the orders by human beings except where such orders would conflict with the First Law.
# A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

These three laws provide an overview of the goals engineers and researchers hold for safety in the HRI field, although the fields of robot ethics and machine ethics are more complex than these three principles. However, generally human–robot interaction prioritizes the safety of humans that interact with potentially dangerous robotics equipment. Solutions to this problem range from the philosophical approach of treating robots as ethical agents (individuals with moral agency), to the practical approach of creating safety zones. These safety zones use technologies such as lidar to detect human presence or physical barriers to protect humans by preventing any contact between machine and operator.

Although initially robots in the human–robot interaction field required some human intervention to function, research has expanded this to the extent that fully autonomous systems are now far more common than in the early 2000s. Autonomous systems include from simultaneous localization and mapping systems which provide intelligent robot movement to natural-language processing and natural-language generation systems which allow for natural, human-esque interaction which meet well-defined psychological benchmarks.

Anthropomorphic robots (machines which imitate human body structure) are better described by the biomimetics field, but overlap with HRI in many research applications. Examples of robots which demonstrate this trend include Willow Garage's PR2 robot, the NASA Robonaut, and Honda ASIMO. However, robots in the human–robot interaction field are not limited to human-like robots: Paro and Kismet are both robots designed to elicit emotional response from humans, and so fall into the category of human–robot interaction.

Goals in HRI range from industrial manufacturing through Cobots, medical technology through rehabilitation, autism intervention, and elder care devices, entertainment, human augmentation, and human convenience. Future research therefore covers a wide range of fields, much of which focuses on assistive robotics, robot-assisted search-and-rescue, and space exploration.

The goal of friendly human–robot interactions

Robots are artificial agents with capacities of perception and action in the physical world often referred by researchers as workspace. Their use has been generalized in factories but nowadays they tend to be found in the most technologically advanced societies in such critical domains as search and rescue, military battle, mine and bomb detection, scientific exploration, law enforcement, entertainment and hospital care.

These new domains of applications imply a closer interaction with the user. The concept of closeness is to be taken in its full meaning, robots and humans share the workspace but also share goals in terms of task achievement. This close interaction needs new theoretical models, on one hand for the robotics scientists who work to improve the robots utility and on the other hand to evaluate the risks and benefits of this new "friend" for our modern society.

With the advance in AI, the research is focusing on one part towards the safest physical interaction but also on a socially correct interaction, dependent on cultural criteria. The goal is to build an intuitive, and easy communication with the robot through speech, gestures, and facial expressions.

Kerstin Dautenhahn refers to friendly Human–robot interaction as "Robotiquette" defining it as the "social rules for robot behaviour (a 'robotiquette') that is comfortable and acceptable to humans" The robot has to adapt itself to our way of expressing desires and orders and not the contrary. But every day environments such as homes have much more complex social rules than those implied by factories or even military environments. Thus, the robot needs perceiving and understanding capacities to build dynamic models of its surroundings. It needs to categorize objects, recognize and locate humans and further recognize their emotions. The need for dynamic capacities pushes forward every sub-field of robotics.

Furthermore, by understanding and perceiving social cues, robots can enable collaborative scenarios with humans. For example, with the rapid rise of personal fabrication machines such as desktop 3d printers, laser cutters, etc., entering our homes, scenarios may arise where robots can collaboratively share control, co-ordinate and achieve tasks together. Industrial robots have already been integrated into industrial assembly lines and are collaboratively working with humans. The social impact of such robots have been studied and has indicated that workers still treat robots and social entities, rely on social cues to understand and work together.

On the other end of HRI research the cognitive modelling of the "relationship" between human and the robots benefits the psychologists and robotic researchers the user study are often of interests on both sides. This research endeavours part of human society. For effective ''human – humanoid robot'' interaction numerous communication skills and related features should be implemented in the design of such artificial agents/systems.

General HRI research

HRI research spans a wide range of fields, some general to the nature of HRI.

Methods for perceiving humans

Methods for perceiving humans in the environment are based on sensor information. Research on sensing components and software led by Microsoft provide useful results for extracting the human kinematics (see Kinect). An example of older technique is to use colour information for example the fact that for light skinned people the hands are lighter than the clothes worn. In any case a human modelled a priori can then be fitted to the sensor data. The robot builds or has (depending on the level of autonomy the robot has) a 3D mapping of its surroundings to which is assigned the humans locations.

Most methods intend to build a 3D model through vision of the environment. The proprioception sensors permit the robot to have information over its own state. This information is relative to a reference.

A speech recognition system is used to interpret human desires or commands. By combining the information inferred by proprioception, sensor and speech the human position and state (standing, seated). In this matter, natural-language processing is concerned with the interactions between computers and human (natural) languages, in particular how to program computers to process and analyze large amounts of natural-language data. For instance, neural-network architectures and learning algorithms that can be applied to various natural-language processing tasks including part-of-speech tagging, chunking, named-entity recognition, and semantic role labeling.

Methods for motion planning

Motion planning in dynamic environments is a challenge that can at the moment only be achieved for robots with 3 to 10 degrees of freedom. Humanoid robots or even 2 armed robots, which can have up to 40 degrees of freedom, are unsuited for dynamic environments with today's technology. However lower-dimensional robots can use the potential field method to compute trajectories which avoid collisions with humans.

Cognitive models and theory of mind

Humans exhibit negative social and emotional responses as well as decreased trust toward some robots that closely, but imperfectly, resemble humans; this phenomenon has been termed the "Uncanny Valley." However recent research in telepresence robots has established that mimicking human body postures and expressive gestures has made the robots likeable and engaging in a remote setting. Further, the presence of a human operator was felt more strongly when tested with an android or humanoid telepresence robot than with normal video communication through a monitor.

While there is a growing body of research about users' perceptions and emotions towards robots, we are still far from a complete understanding. Only additional experiments will determine a more precise model.

Based on past research, we have some indications about current user sentiment and behavior around robots:

* During initial interactions, people are more uncertain, anticipate less social presence, and have fewer positive feelings when thinking about interacting with robots, and prefer to communicate with a human. This finding has been called the human-to-human interaction script.
* It has been observed that when the robot performs a proactive behaviour and does not respect a "safety distance" (by penetrating the user space) the user sometimes expresses fear. This fear response is person-dependent.
* It has also been shown that when a robot has no particular use, negative feelings are often expressed. The robot is perceived as useless and its presence becomes annoying.
* People have also been shown to attribute personality characteristics to the robot that were not implemented in software.
* People similarly infer the mental states of both humans and robots, except for when robots and humans use non-literal language (such as sarcasm or white lies).
* In line with the contact hypothesis, supervised exposure to a social robot can decrease uncertainty and increase willingness to interact with the robot, compared to pre-exposure attitudes toward robots as a class of agents.
* Interacting with a robot by looking at or touching the robot can reduce negative feelings that some people have about robots before interacting with them. Even imagined interaction can reduce negative feelings. However, in some cases, interacting with a robot can increase negative feelings for people with strong pre-existing negative sentiments towards robots.

Methods for human–robot coordination

A large body of work in the field of human–robot interaction has looked at how humans and robots may better collaborate. The primary social cue for humans while collaborating is the shared perception of an activity, to this end researchers have investigated anticipatory robot control through various methods including: monitoring the behaviors of human partners using eye tracking, making inferences about human task intent, and proactive action on the part of the robot. The studies revealed that the anticipatory control helped users perform tasks faster than with reactive control alone.

A common approach to program social cues into robots is to first study human–human behaviors and then transfer the learning. For example, coordination mechanisms in human–robot collaboration are based on work in neuroscience which examined how to enable joint action in human–human configuration by studying perception and action in a social context rather than in isolation. These studies have revealed that maintaining a shared representation of the task is crucial for accomplishing tasks in groups. For example, the authors have examined the task of driving together by separating responsibilities of acceleration and braking i.e., one person is responsible for accelerating and the other for braking; the study revealed that pairs reached the same level of performance as individuals only when they received feedback about the timing of each other's actions. Similarly, researchers have studied the aspect of human–human handovers with household scenarios like passing dining plates in order to enable an adaptive control of the same in human–robot handovers. Another study in the domain of Human Factors and Ergonomics of human–human handovers in warehouses and supermarkets reveal that Givers and Receivers perceive handover tasks differently which has significant implications for designing user-centric human–robot collaborative systems. Most recently, researchers have studied a system that automatically distributes assembly tasks among co-located workers to improve co-ordination.

Application Areas

The application areas of human–robot interaction include robotic technologies that are used by humans for industry, medicine, and companionship, among other purposes.

Industrial Robots

Industrial robots have been implemented to collaborate with humans to perform industrial manufacturing tasks. While humans have the flexibility and the intelligence to consider different approaches to solve the problem, choose the best option among all choices, and then command robots to perform assigned tasks, robots are able to be more precise and more consistent in performing repetitive and dangerous work. Together, the collaboration of industrial robots and humans demonstrates that robots have the capabilities to ensure efficiency of manufacturing and assembling. However, there are persistent concerns about the safety of human–robot collaboration, since industrial robots have the ability to move heavy objects and operate often dangerous and sharp tools, quickly and with force. As a result, this presents a potential threat to the people who work in the same workspace. Therefore, the planning of safe and effective layouts for collaborative workplaces is one of the most challenging topics that research faces.

Medical Robots

Rehabilitation

A rehabilitation robot is an example of a robot-aided system implemented in health care. This type of robot would aid stroke survivors or individuals with neurological impairment to recover their hand and finger movements. In the past few decades, the idea of how human and robot interact with each other is one factor that has been widely considered in the design of rehabilitation robots. For instance, human–robot interaction plays an important role in designing exoskeleton rehabilitation robots since the exoskeleton system makes direct contact with humans' body.

Elder Care and Companion Robot

Nursing robots are aimed to provide assistance to elderly people who may have faced a decline in physical and cognitive function, and, consequently, developed psychosocial issues. By assisting in daily physical activities, physical assistance from the robots would allow the elderly to have a sense of autonomy and feel that they are still able to take care of themselves and stay in their own homes.

Long-term research on human-robot interaction could show that residents of care home are willing to interact with humanoid robots and benefit from cognitive and physical activation that is led by the robot Pepper. Another long-term study in a care home could show that people working in the care sector are willing to use robots in their daily work with the residents. But it also revealed that even though that the robots are ready to be used, they do need human assistants, they cannot replace the human work force but they can assist them and give them new possibilities.

Social Robots

Autism Intervention

Over the past decade, human–robot interaction has shown promising outcomes in autism intervention. Children with autism spectrum disorders (ASD) are more likely to connect with robots than humans, and using social robots is considered to be a beneficial approach to help these children with ASD.

However, social robots that are used to intervene in children's ASD are not viewed as viable treatment by clinical communities because the study of using social robots in ASD intervention, often, does not follow standard research protocol. In addition, the outcome of the research could not demonstrate a consistent positive effect that could be considered as evidence-based practice (EBP) based on the clinical systematic evaluation. As a result, the researchers have started to establish guidelines which suggest how to conduct studies with robot-mediated intervention and hence produce reliable data that could be treated as EBP that would allow clinicians to choose to use robots in ASD intervention.

Rehabilitation

Robots can be configured as collaborative robot and can be used for rehabilitation of users with motor impairment. Using various interactive technologies like automatic speech recognition, eye gaze tracking and so on, users with motor impairment can control robotic agents and use it for rehabilitation activities like powered wheelchair control, object manipulation and so on.

Automatic Driving

A specific example of human–robot interaction is the human-vehicle interaction in automated driving. The goal of human-vehicle cooperation is to ensure safety, security, and comfort in automated driving systems. The continued improvement in this system and the progress in advancements towards highly and fully automated vehicles aim to make the driving experience safer and more efficient in which humans do not need to intervene in the driving process when there is an unexpected driving condition such as a pedestrian walking across the street when it is not supposed to.

Search and Rescue

Unmanned Aerial Vehicles (UAV) and Unmanned Underwater Vehicles (UUV) have the potential to assist search and rescue work in wilderness areas, such as locating a missing person remotely from the evidence that they left in surrounding areas. The system integrates autonomy and information, such as coverage maps, GPS information and quality search video, to support humans performing the search and rescue work efficiently in the given limited time.

Space Exploration

Humans have been working on achieving the next breakthrough in space exploration, such as a crewed mission to Mars. This challenge identified the need for developing planetary rovers that are able to assist astronauts and support their operations during their mission. The collaboration between rovers, UAVs, and humans enables leveraging capabilities from all sides and optimizes task performance.

Agricultural robots

Human labor has been greatly used in agriculture but Agricultural robots like milking robots have been adopted in large-scale farming. Hygiene is the main issue in the agri-food sector and the invention of this technology has widely impacted agriculture. Robots can also be used in tasks that might be hazardous to human health like in the application of chemicals to plants.

See also

Robotics

* Autonomous robots
* Cobots
* Gesture recognition
* Humanoid robots
* Human–robot collaboration
* Mobile robots
* Motion planning
* Personal robot
* Robot simulations
* Robot teams
* Social robot

Technology

* Artificial intelligence
* CAPTCHA
* Computer supported collaborative work
* Dialog management
* Face detection
* Haptic technology
* Human–computer interaction
* Interactive Systems Engineering
* Multimodal interaction
* Natural-language understanding
* Telematics
* Face recognition
* Human sensing

Psychology

* Anthropomorphism and the uncanny valley

Properties

Bartneck and Okada suggest that a robotic user interface can be described by the following four properties:

; Tool – toy scale

* Is the system designed to solve a problem effectively or is it just for entertainment?

; Remote control – autonomous scale

* Does the robot require remote control or is it capable of action without direct human influence?

; Reactive – dialogue scale

* Does the robot rely on a fixed interaction pattern or is it able to have dialogue — exchange of information — with a human?

; Anthropomorphism scale

* Does it have the shape or properties of a human?

Conferences

ACE – International Conference on Future Applications of AI, Sensors, and Robotics in Society

The International Conference on Future Applications of AI, Sensors, and Robotics in Society explore the state of the art research, highlighting the future challenges as well as the hidden potential behind the technologies. The accepted contributions to this conference will be published annually in the special edition of the Journal of Future Robot Life.

International Conference on Social Robotics

The International Conference on Social Robotics is a conference for scientists, researchers, and practitioners to report and discuss the latest progress of their forefront research and findings in social robotics, as well as interactions with human beings and integration into our society.

* ICSR2009, Incheon, Korea in collaboration with the FIRA RoboWorld Congress
* ICSR2010, Singapore
* ICSR2011, Amsterdam, Netherlands

International Conference on Human–Robot Personal Relationships

* HRPR2008, Maastricht
* HRPR 2009, Tilburg. Keynote speaker was Hiroshi Ishiguro.
* HRPR2010, Leiden. Keynote speaker was Kerstin Dautenhahn.

International Congress on Love and Sex with Robots

The International Congress on Love and Sex with Robots is an annual congress that invites and encourages a broad range of topics, such as AI, Philosophy, Ethics, Sociology, Engineering, Computer Science, Bioethics.

The earliest academic papers on the subject were presented at the 2006 E.C. Euron Roboethics Atelier, organized by the School of Robotics in Genoa, followed a year later by the first book – "Love and Sex with Robots" – published by Harper Collins in New York. Since that initial flurry of academic activity in this field the subject has grown significantly in breadth and worldwide interest. Three conferences on Human–Robot Personal Relationships were held in the Netherlands during the period 2008–2010, in each case the proceedings were published by respected academic publishers, including Springer-Verlag. After a gap until 2014 the conferences were renamed as the "International Congress on Love and Sex with Robots", which have previously taken place at the University of Madeira in 2014; in London in 2016 and 2017; and in Brussels in 2019. Additionally, the Springer-Verlag "International Journal of Social Robotics", had, by 2016, published articles mentioning the subject, and an open access journal called "Lovotics" was launched in 2012, devoted entirely to the subject. The past few years have also witnessed a strong upsurge of interest by way of increased coverage of the subject in the print media, TV documentaries and feature films, as well as within the academic community.

The International Congress on Love and Sex with Robots provides an excellent opportunity for academics and industry professionals to present and discuss their innovative work and ideas in an academic symposium.

* 2020, Berlin, Germany
* 2019, Brussels, Belgium
* 2017, London, United Kingdom
* 2016, London, United Kingdom
* 2014, Madeira, Portugal

International Symposium on New Frontiers in Human–Robot Interaction

This symposium is organized in collaboration with the Annual Convention of the Society for the Study of Artificial Intelligence and Simulation of Behaviour.

* 2015, Canterbury, United Kingdom
* 2014, London, United Kingdom
* 2010, Leicester, United Kingdom
* 2009, Edinburgh, United Kingdom

IEEE International Symposium in Robot and Human Interactive Communication

The IEEE International Symposium on Robot and Human Interactive Communication ( RO-MAN ) was founded in 1992 by Profs. Toshio Fukuda, Hisato Kobayashi, Hiroshi Harashima and Fumio Hara. Early workshop participants were mostly Japanese, and the first seven workshops were held in Japan. Since 1999, workshops have been held in Europe and the United States as well as Japan, and participation has been of international scope.

ACM/IEEE International Conference on Human–Robot Interaction

This conference is amongst the best conferences in the field of HRI and has a very selective reviewing process. The average acceptance rate is 26% and the average attendance is 187. Around 65% of the contributions to the conference come from the US and the high level of quality of the submissions to the conference becomes visible by the average of 10 citations that the HRI papers attracted so far.
* HRI 2006 in Salt Lake City, Utah, USA, Acceptance Rate: 0.29
* HRI 2007 in Washington, D.C., USA, Acceptance Rate: 0.23
* HRI 2008 in Amsterdam, Netherlands, Acceptance Rate: 0.36 (0.18 for oral presentations)
* HRI 2009 in San Diego, CA, USA, Acceptance Rate: 0.19
* HRI 2010 in Osaka, Japan, Acceptance Rate: 0.21
* HRI 2011 in Lausanne, Switzerland, Acceptance Rate: 0.22 for full papers
* HRI 2012 in Boston, Massachusetts, USA, Acceptance Rate: 0.25 for full papers
* HRI 2013 in Tokyo, Japan, Acceptance Rate: 0.24 for full papers
* HRI 2014 in Bielefeld, Germany, Acceptance Rate: 0.24 for full papers
* HRI 2015 in Portland, Oregon, USA, Acceptance Rate: 0.25 for full papers
* HRI 2016 in Christchurch, New Zealand, Acceptance Rate: 0.25 for full papers
* HRI 2017 in Vienna, Austria, Acceptance Rate: 0.24 for full papers
* HRI 2018 in Chicago, USA, Acceptance Rate: 0.24 for full papers
* HRI 2021 in Boulder, Colorado, Boulder, USA, Acceptance Rate: 0.23 for full papers

International Conference on Human–Agent Interaction

* HAI 2013 in Sapporo, Japan
* HAI 2014 in Tsukuba, Japan
* HAI 2015 in Daegu, Korea
* HAI 2016 in Singapore
* HAI 2017 in Bielefeld, Germany

Related conferences

There are many conferences that are not exclusively HRI, but deal with broad aspects of HRI, and often have HRI papers presented.

* IEEE-RAS/RSJ International Conference on Humanoid Robots (Humanoids)
* Ubiquitous Computing (UbiComp)
* IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
* Intelligent User Interfaces (IUI)
* Computer Human Interaction (CHI)
* American Association for Artificial Intelligence (AAAI)
* INTERACT

Journals

There are currently two dedicated HRI Journals

* ''ACM Transactions on Human–Robot Interaction'' (Originally Journal of Human–Robot Interaction)
* ''International Journal of Social Robotics''

and there are several more general journals in which one will find HRI articles.

* ''International Journal of Humanoid Robotics''
* ‘Entertainment robotics’ section of the ''Entertainment Computing Journal''
* ''Interaction Studies Journal''
* Artificial Intelligence (journal), ''Artificial Intelligence''
* ''Systems, Man and Cybernetics''

Books

There are several books available that specialise on Human–Robot Interaction. While there are several edited books, only a few dedicated texts are available:
* – free PDF available online
*
* – chapter in an extensive handbook.

Courses

Many universities offer courses in Human–Robot Interaction.

University Courses and Degrees

* Tufts University, Medford, MA, USA, MS and PhD programs in Human–Robot Interaction
* University of Waterloo, Canada, Kerstin Dautenhahn, Social Robotics – Foundations, Technology and Applications of Human-Centered Robotics
* National Taipei University in Taiwan, Taiwan, Hooman Samani, M5226 Advanced Robotics
* Ontario Tech University, Canada, Patrick C. K. Hung, BUSI4590U Topics in Technology Management & INFR 4599U Service Robots Innovation for Commerce
* The Colorado School of Mines, USA, Tom Williams, CSCI 436 / 536: Human–Robot Interaction
* Heriot-Watt University, UK, Lynne Baillie, F21HR Human Robot Interaction
* Uppsala University, Sweden, Filip Malmberg, UU-61611 Social Robotics and Human–Robot Interaction
* Skövde University, Sweden, MSc Human–Robot Interaction program
* Indiana University, Bloomington, USA, Selma Sabanovic, INFO-I 440 Human–Robot Interaction
* Ghent University, Belgium, Tony Belpaeme, E019370A Robotics module
* Bielefeld University, Germany, Frederike Eyssel, 270037 Sozialpsychologische Aspekte der Mensch-Maschine Interaktion
* Kyoto University, Japan, Takayuki Kanda, 3218000 Human–Robot Interaction (ヒューマンロボットインタラクション)

Online Courses and Degrees

There are also online courses available such as Mooc:
* University of Canterbury (UCx) – edX program
** Professional Certificate in Human–Robot Interaction
** Introduction to Human–Robot Interaction
** Methods and Application in Human–Robot Interaction

Footnotes

References

52. Chen, H., Leu, M. C., and Yin, Z. (June 10, 2022). "Real-Time Multi-Modal Human–Robot Collaboration Using Gestures and Speech." ASME. J. Manuf. Sci. Eng. October 2022; 144(10): 101007. https://doi.org/10.1115/1.4054297

External resources

*
*

{{DEFAULTSORT:Human-Robot Interaction
Human communication
Multimodal interaction
Robotics