NASA-TLX on:
[Wikipedia]
[Google]
[Amazon]
The
NASA-TLX originally consisted of two parts: the total workload is divided into six subjective subscales that are represented on a single page, serving as one part of the questionnaire:
*Mental Demand
*Physical Demand
*Temporal Demand
*Performance
*Effort
*Frustration
There is a description for each of these subscales that the subject should read before rating. They are rated ''for each task'' within a 100-points range with 5-point steps. These ratings are then combined to the task load index. Providing descriptions for each measurement can be found to help participants answer accurately. These descriptions are as follows:
; Mental Demand
: How much mental and perceptual activity was required? Was the task easy or demanding, simple or complex?
; Physical Demand
: How much physical activity was required? Was the task easy or demanding, slack or strenuous?
; Temporal Demand
: How much time pressure did you feel due to the pace at which the tasks or task elements occurred? Was the pace slow or rapid?
; Own Performance
: How successful were you in performing the task? How satisfied were you with your
paper and pencil version
or using th
Official NASA TLX for Apple iOS App
There are also numerous unofficial computerized implementations of the NASA TLX. These unofficial versions may collect Personally Identifiable Information (PII), which is a violation of NASA Human Subject Research Guidelines for the Collection of PII as set down by the NASA Independent Review Board (IRB). If a participant is required to answer the TLX questions multiple times, they only need to answer the 15 pairwise comparisons once per task type. If a participant's workload needs to be measured for intrinsically different tasks, then revisiting the pairwise comparisons may be required. In every case, the subject should answer all 6 subjective rating subscales. It is these successive ratings that are then scored using the original pairwise questions as weighting factors, that leads to an understanding of the overall workload change. While there are multiple ways to administer the NASA-TLX, some may change the results of the test. One study showed that a paper-and-pencil version led to less cognitive workload than processing the information on a computer screen. However, other studies found that computer screen versions, as well as on wearables, can nonetheless stably capture relative changes in workload. To overcome the delay in administrating the test, the Official NASA TLX Apple iOS App can be used to capture both the pairwise question answers and a subjects subjective subscale input, as well as calculating the final weighted and unweighted results. A feature found in the Official NASA TLX App is a new computer interface response rating scale, termed a Subjective Analogue Equivalent Rating (SAER) scale, that provides the closest possible user experience to that found in the paper and pencil version of NASA TLX. No other computerized version of the NASA TLX has successfully implemented this critical element for properly capturing a user subjective input. This can be seen in many unofficial computerized (both web and software application) versions that use an anchored or locking scale. This defeats the subjective purpose of the original paper and pencil implementation of the NASA TLX.
{{cite journal , last1=Hart, first1=Sandra G. , date=October 2006 , title=NASA-Task Load Index (NASA-TLX); 20 Years Later , journal=Proceedings of the Human Factors and Ergonomics Society Annual Meeting , volume=50 , issue=9 , pages=904–908 , doi=10.1177/154193120605000909 , s2cid=6292200 , url=http://www.stavelandhfe.com/images/nasa-tlx_paper.pdf NASA (1986)
Nasa Task Load Index (TLX) v. 1.0 Manual
/ref>
Official NASA TLX Task Load Index WebsiteOfficial NASA TLX Task Load Index iOS AppUnofficial NASA-TLX implemented in HTML + JavaScriptUnofficial NASA-TLX implemented in Java (Swing)Unofficial Flash Only Version
NASA Neuropsychological tests Cognitive tests
NASA
The National Aeronautics and Space Administration (NASA ) is an independent agency of the US federal government responsible for the civil space program, aeronautics research, and space research.
NASA was established in 1958, succeeding t ...
Task Load Index (NASA-TLX) is a widely used, subjective, multidimensional assessment tool that rates perceived workload
The term workload can refer to a number of different yet related entities.
An amount of labor
Workload is the amount of work an individual has to do.Jex, S. M. (1998). Stress and job performance: Theory, research, and implications for managerial ...
in order to assess a task, system, or team's effectiveness or other aspects of performance (task loading
In underwater diving, task load indicates the degree of difficulty experienced when performing a task, and task loading describes the accumulation of tasks that are necessary to perform an operation. A light task loading can be managed by the opera ...
). It was developed by the Human Performance Group at NASA's Ames Research Center over a three-year development cycle that included more than 40 laboratory simulations. It has been cited in over 4,400 studies, highlighting the influence the NASA-TLX has had in human factors research. It has been used in a variety of domains, including aviation
Aviation includes the activities surrounding mechanical flight and the aircraft industry. ''Aircraft'' includes fixed-wing and rotary-wing types, morphable wings, wing-less lifting bodies, as well as lighter-than-air craft such as hot air ...
, healthcare
Health care or healthcare is the improvement of health via the prevention, diagnosis, treatment, amelioration or cure of disease, illness, injury, and other physical and mental impairments in people. Health care is delivered by health profe ...
and other complex socio-technical domains.
It is a subjective self-reporting set of scores, and is not an objective measure of the Task Load that should be measured using objective metrics that examine the product of the speed and accuracy of users performing a task.
Scales
NASA-TLX originally consisted of two parts: the total workload is divided into six subjective subscales that are represented on a single page, serving as one part of the questionnaire:
*Mental Demand
*Physical Demand
*Temporal Demand
*Performance
*Effort
*Frustration
There is a description for each of these subscales that the subject should read before rating. They are rated ''for each task'' within a 100-points range with 5-point steps. These ratings are then combined to the task load index. Providing descriptions for each measurement can be found to help participants answer accurately. These descriptions are as follows:
; Mental Demand
: How much mental and perceptual activity was required? Was the task easy or demanding, simple or complex?
; Physical Demand
: How much physical activity was required? Was the task easy or demanding, slack or strenuous?
; Temporal Demand
: How much time pressure did you feel due to the pace at which the tasks or task elements occurred? Was the pace slow or rapid?
; Own Performance
: How successful were you in performing the task? How satisfied were you with your performance
A performance is an act of staging or presenting a play, concert, or other form of entertainment. It is also defined as the action or process of carrying out or accomplishing an action, task, or function.
Management science
In the work place ...
?
; Effort
: How hard did you have to work (mentally and physically) to accomplish your level of performance?
; Frustration Level
: How irritated, stressed, and annoyed versus content, relaxed, and complacent did you feel during the task?
Analysis
The second part of TLX intends to create an individual weighting of these subscales by letting the subjects compare them pairwise based on their perceived importance. This requires the user to choose whichmeasurement
Measurement is the quantification of attributes of an object or event, which can be used to compare with other objects or events.
In other words, measurement is a process of determining how large or small a physical quantity is as compared ...
is more relevant to workload
The term workload can refer to a number of different yet related entities.
An amount of labor
Workload is the amount of work an individual has to do.Jex, S. M. (1998). Stress and job performance: Theory, research, and implications for managerial ...
. The number of times each is chosen is the weighted score. This is multiplied by the scale score for each dimension and then divided by 15 to get a workload score from 0 to 100, the overall task load index. Many researchers eliminate these pairwise comparisons, though, and refer to the test as "Raw TLX" then. There has been evidence evaluating and supporting this shortened version over the full one since it might increase experimental validity.
When using the "raw TLX", individual subscales may be dropped if less relevant to the task.
Administration
The Official NASA-TLX can be administered usingpaper and pencil version
or using th
Official NASA TLX for Apple iOS App
There are also numerous unofficial computerized implementations of the NASA TLX. These unofficial versions may collect Personally Identifiable Information (PII), which is a violation of NASA Human Subject Research Guidelines for the Collection of PII as set down by the NASA Independent Review Board (IRB). If a participant is required to answer the TLX questions multiple times, they only need to answer the 15 pairwise comparisons once per task type. If a participant's workload needs to be measured for intrinsically different tasks, then revisiting the pairwise comparisons may be required. In every case, the subject should answer all 6 subjective rating subscales. It is these successive ratings that are then scored using the original pairwise questions as weighting factors, that leads to an understanding of the overall workload change. While there are multiple ways to administer the NASA-TLX, some may change the results of the test. One study showed that a paper-and-pencil version led to less cognitive workload than processing the information on a computer screen. However, other studies found that computer screen versions, as well as on wearables, can nonetheless stably capture relative changes in workload. To overcome the delay in administrating the test, the Official NASA TLX Apple iOS App can be used to capture both the pairwise question answers and a subjects subjective subscale input, as well as calculating the final weighted and unweighted results. A feature found in the Official NASA TLX App is a new computer interface response rating scale, termed a Subjective Analogue Equivalent Rating (SAER) scale, that provides the closest possible user experience to that found in the paper and pencil version of NASA TLX. No other computerized version of the NASA TLX has successfully implemented this critical element for properly capturing a user subjective input. This can be seen in many unofficial computerized (both web and software application) versions that use an anchored or locking scale. This defeats the subjective purpose of the original paper and pencil implementation of the NASA TLX.
See also
* Cognitive loadReferences
{{reflist, refs= {{cite journal , last1=Bustamante, first1=EA , last2=Spain, first2=RD , title=Measurement Invariance of the NASA TLX , journal=Proceedings of the Human Factors and Ergonomics Society Annual Meeting , date=September 2008 , volume=52 , issue=19 , pages=1522–1526 , doi=10.1177/154193120805201946 , s2cid=143607921 {{Subscription required {{cite book , last1=Hart , first1=Sandra G. , last2=Staveland, first2=Lowell E. , date=1988 , chapter=Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , pages=139–183 , editor1-last=Hancock, editor1-first=Peter A. , editor2-last=Meshkati, editor2-first=Najmedin , title=Human Mental Workload , series=Advances in Psychology, volume=52 , publisher=North Holland
North Holland ( nl, Noord-Holland, ) is a province of the Netherlands in the northwestern part of the country. It is located on the North Sea, north of South Holland and Utrecht, and west of Friesland and Flevoland. In November 2019, it had a ...
, location=Amsterdam , doi=10.1016/S0166-4115(08)62386-9 , isbn=978-0-444-70388-0 , chapter-url=http://humansystems.arc.nasa.gov/groups/TLX/downloads/NASA-TLXChapter.pdf
External link to Google Scholar{{cite journal , last1=Hart, first1=Sandra G. , date=October 2006 , title=NASA-Task Load Index (NASA-TLX); 20 Years Later , journal=Proceedings of the Human Factors and Ergonomics Society Annual Meeting , volume=50 , issue=9 , pages=904–908 , doi=10.1177/154193120605000909 , s2cid=6292200 , url=http://www.stavelandhfe.com/images/nasa-tlx_paper.pdf NASA (1986)
Nasa Task Load Index (TLX) v. 1.0 Manual
/ref>
External links
Official NASA TLX Task Load Index Website
NASA Neuropsychological tests Cognitive tests