Tools, devices or software (as diverse as a TV remote control, the interface of an oven, or a word processor) must be evaluated before their release on the market from different points of view such as their technical properties or their
usability
Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a soft ...
.
Usability evaluation allows assessing whether the product under evaluation is efficient enough (Are the users able to carry out their task while expending reasonable resources such as time, cognitive or physical demand), effective enough (Can the user complete the tasks they are supposed to perform with the tool? Is their performance complete and accurate?) and sufficiently satisfactory for the users (What is the users’ attitude towards the system? Do they experience discomfort?).
[International Organization for Standardization. Ergonomics of human system interaction - Part 210 -: Human centred design for interactive systems (Rep N°9241-210). 2010, International Organization for Standardization][Nielsen, Usability Engineering, 1994] For this assessment to be objective, there is a need for measurable goals
(for instance in terms of easiness of use or of learning) that the system must achieve.
That kind of goal is called a usability goal (or also usability requirement
). They are objective criteria against which the results of the usability evaluation are compared to assess the usability of the product under evaluation.[
]
In product design
Usability goals must be included in every product design process that intends to follow a Human Factors approach (for instance, User-centered design
User-centered design (UCD) or user-driven development (UDD) is a framework of process (not restricted to interfaces or technologies) in which usability goals, user characteristics, environment, tasks and workflow of a product, service or proce ...
[ process or Usability Engineering Lifecycle][Mayhew. The usability engineering lifecycle: a practitioner's handbook for user interface design. London, Academic press; 1999]). They have to be clearly stated from the onset of the process, as soon as the end-users needs, risk of use, contexts and aims of use are identified (cf. “definition of usability goals” part).
Then, usability goals are used at each usability evaluation phase of the design process. Whatever the type of evaluation phase (i.e. formative or summative evaluation[Brender J. Handbook of evaluation methods for health informatics. Burlington, MA: Elsevier; 2006.]), they are used to assess the performance of the users against the result of the evaluation process:
* During formative/constructive evaluations (i.e. evaluations that occur during the design process to contribute to further improvement of the object under evaluation[), the comparison of the evaluation results against usability goals allows verifying whether those goals are met or not: as long as they are not met, the product under evaluation must be re-engineered to improve its usability. In this frame, usability goals allow also identifying usability flaws and therefore supporting this re-engineering process. They can also be used all along the iterations of the ]User-centered design
User-centered design (UCD) or user-driven development (UDD) is a framework of process (not restricted to interfaces or technologies) in which usability goals, user characteristics, environment, tasks and workflow of a product, service or proce ...
process as indicators to follow up the evolution of the system in terms of usability.
* During summative evaluations (i.e. evaluations that try to give a definitive statement on the quality properties of a system under evaluation[), the meeting of usability goals means that the system is usable enough to go out the ]User-centered design
User-centered design (UCD) or user-driven development (UDD) is a framework of process (not restricted to interfaces or technologies) in which usability goals, user characteristics, environment, tasks and workflow of a product, service or proce ...
[ process and to be released.
]
Formulation
Usability goals must address the three usability components, i.e. effectiveness, efficiency and satisfaction.[ Their definition, for each of those components, must rest on the characteristics of the tasks that the tested system is supposed to support.] More practically, Mayhew proposes that their definition should refer to:
* The identified end-users profiles
* The tasks that the different categories of identified end-users are supposed to perform with the tested system in a given context of use (results from a Contextual Task Analysis).
* Business goals
Moreover, for certain types of products that are used for sensitive purposes (for instance, medical devices or nuclear plant control interface), usability goals must be defined in close relation to the Risk assessment
Broadly speaking, a risk assessment is the combined effort of:
# identifying and analyzing potential (future) events that may negatively impact individuals, assets, and/or the environment (i.e. hazard analysis); and
# making judgments "on the to ...
process of those products.[Marcilly et al., Patient Safety Oriented Usability Goals: a pilot study. MIE 2013.] This kind of “safety-oriented usability goal” is used to prevent a tool being released on the market while identifying deficiencies in its interface design that could induce Use errors. Thus, risks that may result in use errors must be identified; and then, for each of them, usability goals must be defined, taking into account the severity of the potential consequences of the risk[Association for the Advancement of Medical Instrumentation. Human factors engineering-design of medical devices (ANSI/AAMI HE75). Arlington, VA: AAMI; 2009.](for instance, in terms of operator, patient or environment safety).
Prioritization
For a given tool under evaluation, several usability goals are defined. If some goals are related to safety issues while others are more “comfort of use usability goals", they will not all require the same level of achievement.
: For instance, a “comfort of use usability goal” dealing with the easiness of browsing on the Internet that does not endanger users' safety could require a partial achievement (e.g. 80% of users must achieve using a function that make easier the browsing on the Internet, as a short-cut) while a usability goal concerning a major risk for users' or environment' safety would require a total achievement (no error tolerated; e.g.100% of the users must succeed in using a defibrillator at their first trial). For this kind of “safety-oriented usability goal”, a non-achievement reveals that the use of the tool may lead to dramatic consequences. Those goals should be satisfied before any release of the system (for instance, a patient safety sensitive Health Information Technology cannot be released if it has been shown to induce errors of use ).
Therefore, the achievement level of the defined usability goals should be prioritized.
Measurement
The goals are defined either in a qualitative or a quantitative way. Nonetheless, whatever their nature, they have to be operationally defined.
The achievement of qualitative usability goals can be assessed through verbal protocols analysis. Then, the goal will be formulated in terms related to the coding scheme used for the analysis. Those qualitative goals can be turned into quantitative goals to support an objective quantifiable assessment. This kind of goal can take the shape of:
*''"U% of a sample of the intended user population should express positive comments about a specific function while using the tool"''
*or ''“less than U% of the sample misinterprets the information provided by a display”.''
As for qualitative usability goals assessed through questionnaires, they can be formulated as:
*''“The average score of the sample of the intended user population for the scale S must be over N”''
As for quantitative goal, they can be assessed by various methods such as time measurement (instance in ), keystroke analysis or error rate quantification. They may look like (following[Smith E. Siochi A. Software usability: requirements by evaluation. In: Human factors perspectives on human-computer interaction. Santa Monica, CA: Human factors and Ergonomics Society, 1995.]):
*''“U% of a sample of the intended user population should accomplish T% of the benchmark tasks within M minutes and with no more than E errors”''
See also
* Accessibility
Accessibility is the design of products, devices, services, vehicles, or environments so as to be usable by people with disabilities. The concept of accessible design and practice of accessible development ensures both "direct access" (i. ...
* Component-Based Usability Testing
* GUI
The GUI ( "UI" by itself is still usually pronounced . or ), graphical user interface, is a form of user interface that allows users to interact with electronic devices through graphical icons and audio indicator such as primary notation, inste ...
* Human factors
Human factors and ergonomics (commonly referred to as human factors) is the application of psychological and physiological principles to the engineering and design of products, processes, and systems. Four primary goals of human factors learnin ...
* Interactive systems engineering
Systems engineering is an interdisciplinary field of engineering and engineering management that focuses on how to design, integrate, and manage complex systems over their life cycles. At its core, systems engineering utilizes systems thinking p ...
* ISO 9241
ISO 9241 is a multi-part standard from the International Organization for Standardization (ISO) covering ergonomics of human-computer interaction. It is managed by the ISO Technical Committee 159. It was originally titled ''Ergonomic requirement ...
* List of human-computer interaction topics
A ''list'' is any set of items in a row. List or lists may also refer to:
People
* List (surname)
Organizations
* List College, an undergraduate division of the Jewish Theological Seminary of America
* SC Germania List, German rugby uni ...
* Software performance testing
In software quality assurance, performance testing is in general a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, valid ...
* Software testing
Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to apprecia ...
* Test method
A test method is a method for a test in science or engineering, such as a physical test, chemical test, or statistical test. It is a definitive procedure that produces a test result. In order to ensure accurate and relevant test results, a test met ...
* Tree testing
Tree testing is a usability technique for evaluating the findability of topics in a website. It is also known as reverse card sorting or card-based classification.
A large website is typically organized into a hierarchy (a "tree") of topics and ...
* Usability
Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a soft ...
* Usability testing
Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. It is m ...
* Usability engineering
Usability engineering is a field that is concerned generally with human–computer interaction and specifically with devising human–computer interfaces that have high usability or user friendliness. It provides structured methods for achieving e ...
* Use error
* User-centered design
User-centered design (UCD) or user-driven development (UDD) is a framework of process (not restricted to interfaces or technologies) in which usability goals, user characteristics, environment, tasks and workflow of a product, service or proce ...
* User experience design
User experience design (UX design, UXD, UED, or XD) is the process of defining the experience a user would go through when interacting with a digital product or website. Design decisions in UX design are often driven by research, data analysis, an ...
* Web usability
Web usability of a website are broad goals of usability and presentation of information and choices in a clear and concise way, a lack of ambiguity and the placement of important items in appropriate areas as well as ensuring that the content work ...
References
{{DEFAULTSORT:Usability Testing
Usability
Software testing