Dunning–Kruger effect

The Dunning–Kruger effect is a cognitive bias whereby people with low ability, expertise, or experience regarding a certain type of task or area of knowledge tend to overestimate their ability or knowledge. Some researchers also include in their definition the opposite effect for high performers: their tendency to underestimate their skills.

The Dunning–Kruger effect is usually measured by comparing self-assessment with objective performance. For example, the participants in a study may be asked to complete a quiz and then estimate how well they performed. This subjective assessment is then compared with how well they actually performed. This can happen either in relative or in absolute terms, i.e., in comparison with one's peer group as the percentage of peers outperformed or in comparison with objective standards as the number of questions answered correctly. The Dunning–Kruger effect appears in both cases, but is more pronounced in relative terms; the bottom quartile of performers tend to see themselves as being part of the top two quartiles.
The initial study was published by David Dunning and Justin Kruger in 1999. It focused on logical reasoning, grammar, and social skills. Since then, various other studies have been conducted across a wide range of tasks, including skills from fields such as business, politics, medicine, driving, aviation, spatial memory, examinations in school, and literacy.

The Dunning–Kruger effect is usually explained in terms of metacognitive abilities. This approach is based on the idea that poor performers have not yet acquired the ability to distinguish between good and bad performances. They tend to overrate themselves because they do not see the qualitative difference between their performances and the performances of others. This has also been termed the "dual-burden account", since the lack of skill is paired with the ignorance of this lack. Some researchers include the metacognitive component as part of the definition of the Dunning–Kruger effect and not just as an explanation distinct from it.

Many debates surrounding the Dunning–Kruger effect and criticisms of it focus on the metacognitive explanation without denying the empirical findings. The statistical explanation interprets these findings as statistical artifacts. Some theorists hold that the way low and high performers are distributed makes assessing their skill level more difficult for low performers, thereby explaining their erroneous self-assessments independent of their metacognitive abilities. Another account sees the lack of incentives to give accurate self-assessments as the source of error.

The Dunning–Kruger effect has been described as relevant for various practical matters, but disagreements exist about the magnitude of its influence. Inaccurate self-assessment can lead people to make bad decisions, such as choosing a career for which they are unfit or engaging in dangerous behavior. It may also inhibit the affected from addressing their shortcomings to improve themselves. In some cases, the associated overconfidence may have positive side effects, like increasing motivation and energy.

Definition

The Dunning–Kruger effect is defined as the tendency of people with low ability in a specific area to give overly positive assessments of this ability. This is often understood as a cognitive bias, i.e. as a systematic tendency to engage in erroneous forms of thinking and judging. Biases are systematic in the sense that they occur consistently in different situations. They are tendencies since they concern certain inclinations or dispositions that may be observed in groups of people, but are not manifested in every performance. In the case of the Dunning–Kruger effect, this applies mainly to people with low skill in a specific area trying to evaluate their competence within this area. The systematic error concerns their tendency to greatly overestimate their competence or to see themselves as more skilled than they are.

Some researchers emphasize the metacognitive component in their definition. In this view, the Dunning–Kruger effect is the thesis that those who are incompetent in a given area tend to be ignorant of their incompetence, i.e. they lack the metacognitive ability to become aware of their incompetence. This definition lends itself to a simple explanation of the effect; incompetence often includes being unable to tell the difference between competence and incompetence, which is why it is difficult for the incompetent to recognize their incompetence. This is sometimes termed the "dual-burden" account since two burdens come paired: the lack of skill and the ignorance of this lack. But most definitions focus on the tendency to overestimate one's ability and see the relation to metacognition as a possible explanation independent of one's definition. This distinction is relevant since the metacognitive explanation is controversial and various criticisms of the Dunning–Kruger effect target this explanation, but not the effect itself when defined in the narrow sense.

The Dunning–Kruger effect is usually defined specifically for the self-assessments of people with a low level of competence. Some definitions, though, do not restrict it to the bias of people with low skill, and instead see it as pertaining to false self-evaluations on different skill levels. So it is sometimes claimed to include the reverse effect for people with high skill. On this view, the Dunning–Kruger effect also concerns the tendency of highly skilled people to underestimate their abilities relative to the abilities of others. Arguably, the source of this error is not the self-assessment of one's skills, but an overly positive assessment of the skills of others. This phenomenon has been categorized as a form of the false-consensus effect.

Measurement and analysis

The most common approach to measuring the Dunning–Kruger effect is to compare self-assessment with objective performance. The self-assessment is sometimes called ''subjective ability'' in contrast to the ''objective ability'' corresponding to the actual performance. The self-assessment may be done before or after the performance. If done afterward, it is important that the participants receive no independent clues during the performance as to how well they did. So, if the activity involves answering quiz questions, no feedback is given as to whether a given answer was correct. The measurement of the subjective and the objective abilities can be in absolute or relative terms. When done in absolute terms, self-assessment and performance are measured according to absolute standards, e.g. concerning how many quiz questions were answered correctly. When done in relative terms, the results are compared with a peer group. In this case, participants are asked to assess their performances in relation to the other participants, for example in the form of estimating the percentage of peers they outperformed. The Dunning–Kruger effect is present in both cases, but tends to be significantly more pronounced when done in relative terms. So people are usually more accurate when predicting their raw score than when assessing how well they did relative to their peer group.

The main point of interest for researchers is usually the correlation between subjective and objective ability. To provide a simplified form of analysis of the measurements, objective performances are often divided into four groups, starting from the bottom quartile of low performers to the top quartile of high performers. The strongest effect is seen for the participants in the bottom quartile, who tend to see themselves as being part of the top two quartiles when measured in relative terms. Some researchers focus their analysis on the difference between the two abilities, i.e. on subjective ability minus objective ability, to highlight the negative correlation.

Studies

The Dunning–Kruger effect has been studied across a wide range of tasks. The initial study focused on logical reasoning, grammar skills, and social abilities, such as emotional intelligence and judging which jokes are funny. While many studies are conducted in laboratories, others take place in real-world settings. The latter include assessing the knowledge hunters have of firearms and safety or laboratory technicians' knowledge of medical lab procedures. More recent studies have also engaged in large-scale attempts to collect the relevant data online. Various studies focus on students—for example, to self-assess their performance just after completing an exam. In some cases, these studies gather and compare data from many different countries. Other fields of research include business, politics, medicine, driving skills, aviation, spatial memory, literacy, debating skills, and chess.

David Dunning and Justin Kruger published the initial study in 1999 under the title "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments". It examines the performance and self-assessment of undergraduate students of introductory courses in psychology in the fields of inductive, deductive, and abductive logical reasoning, English grammar, and personal sense of humor. Across four studies, the research indicates that the participants who scored in the bottom quartile overestimated their test performance and their abilities; despite test scores that placed them in the 12th percentile, the participants estimated they ranked in the 62nd percentile. It proposes the metacognitive explanation of the observed tendencies and points out that training in a task, such as solving a logic puzzle, increases people's ability to accurately evaluate how good they are at it. It does not yet contain the term "Dunning–Kruger effect", which was introduced later. Dunning was inspired to engage in this research after reading a newspaper report about incompetent bank robbers and set up a research program soon afterward together with Kruger, who was his graduate student at the time.

Dunning, Kruger, and various other researchers published many subsequent studies. In the 2003 paper "Why People Fail to Recognize Their Own Incompetence", the relation between incorrect self-assessment of competence and the person's ignorance of a given activity's standards of performance is discussed. The 2003 study "How Chronic Self-Views Influence (and Potentially Mislead) Estimates of Performance" examines how a person's self-view causes inaccurate self-assessments of their abilities and why such misperceptions are maintained. The 2004 study "Mind-Reading and Metacognition: Narcissism, not Actual Competence, Predicts Self-estimated Ability" extends the research to test subjects' emotional sensitivity to other people and their own perceptions of them. In the 2005 paper "Self-insight: Roadblocks and Detours on the Path to Knowing Thyself", Dunning calls the Dunning–Kruger effect "the anosognosia of everyday life", referring to a neurological condition in which disabled persons either deny or seem unaware of their disability. He writes: "If you're incompetent, you can't know you're incompetent ... The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is."

The 2006 study "Skilled or Unskilled, but Still Unaware of It: How Perceptions of Difficulty Drive Miscalibration in Relative Comparisons" tries to show that it is not true of all activities that poor performers give more inaccurate self-assessments than strong performers. The study investigates 13 different tasks and concludes that the Dunning–Kruger effect obtains only in tasks that feel easy. Nonetheless, the 2008 study "Why the Unskilled are Unaware: Further Explorations of (Absent) Self-insight Among the Incompetent" applies the research to many additional fields and confirms that the Dunning–Kruger effect is seen in a great variety of tasks. In his 2011 article "The Dunning–Kruger Effect: On Being Ignorant of One's Own Ignorance", Dunning summarizes many of the earlier studies and reasserts the metacognitive explanation of these findings. As he writes, " short, those who are incompetent, for lack of a better term, should have little insight into their incompetence—an assertion that has come to be known as the Dunning–Kruger effect". In 2014, Dunning and Helzer wrote that the Dunning–Kruger effect "suggests that poor performers are not in a position to recognize the shortcomings in their performance" but added that self-assessment can be improved by becoming a better performer. A 2022 study found, consistent with the Dunning–Kruger effect, that people who reject the scientific consensus on issues think they know the most about them but actually know the least. The study assessed this on climate change, genetically modified organisms, vaccines, nuclear power, homeopathy, evolution, the Big Bang theory, and COVID-19.

Explanations

Metacognitive

Various explanations have been proposed to account for the Dunning–Kruger effect. The initial and most common account is based on metacognitive abilities. It rests on the assumption that part of acquiring a skill consists in learning to distinguish between good and bad performances of this skill. Since people with low skill have not yet acquired this discriminatory ability, they are unable to properly assess their performance. This leads them to believe that they are better than they are because they do not see the qualitative difference between their performances and the performances by others. So they lack the metacognitive ability to recognize their incompetence. This account has also been called the "dual-burden account" or the "double-burden of incompetence" since the burden of regular incompetence is paired with the burden of meta-cognitive incompetence. It is usually combined with the thesis that the relevant meta-cognitive abilities are acquired as one's skill level increases. But the meta-cognitive lack may also hinder some people from becoming better by hiding their flaws from them. This can then be used to explain how self-confidence is sometimes higher for unskilled people than for people with an average skill: only the latter are aware of their flaws. Some attempts have been made to measure metacognitive abilities directly to confirm this hypothesis. The findings suggest that a reduced metacognitive sensitivity exists among poor performers, but it is not clear that its extent is sufficient to explain the Dunning–Kruger effect. An indirect argument for the metacognitive account is based on the observation that training people in logical reasoning helps them make more accurate self-assessments.

Criticism and alternatives

Not everyone agrees with the assumptions on which the metacognitive account is based. Many criticisms of the Dunning–Kruger effect have the metacognitive account as their main focus, but agree with the empirical findings themselves. This line of argument usually proceeds by providing an alternative approach that promises a better explanation of the observed tendencies. Some explanations focus only on one specific factor, while others see a combination of various factors as the source. One such account is based on the idea that both low and high performers have in general the same metacognitive ability to assess their skill level. But given the assumption that the skill levels of many low performers are very close to each other, i.e., that "many people repiled up at the bottom rungs of skill level", they find themselves in a more difficult position to assess their skills in relation to their peers. So, the reason for the increased tendency to give false self-assessments is not lack of metacognitive ability, but a more challenging situation in which this ability is applied. Thus, the increased error can be explained without a dual-burden account. One criticism of this approach is directed against the assumption that this type of distribution of skill levels can always be used as an explanation. While it can be found in various fields where the Dunning–Kruger effect has been researched, it is not present in all of them. Another criticism rests on the fact that this account can explain the Dunning–Kruger effect only when the self-assessment is measured relative to one's peer group, not when measured relative to absolute standards.

Another account, sometimes given by theorists with an economic background, focuses on the fact that participants in the corresponding studies usually lack incentive to give accurate self-assessments. In such cases, intellectual laziness or a desire to look good to the experimenter may motivate participants to give overly positive self-assessments. For this reason, some studies were conducted with additional incentives to be accurate. One study, for example, gave participants a monetary reward based on how accurate their self-assessments were. These studies failed to show any significant increase in accuracy for the incentive group in contrast to the control group.

A different approach, further removed from psychological explanations, sees the Dunning–Kruger effect as mainly a statistical artifact. It is based on the idea that the statistical effect known as regression toward the mean accounts for the empirical findings. In the case of the quality of performances, this effect rests on the idea that the quality of a given performance depends not just on the agent's skill level, but also on the good or bad luck involved on an occasion. So, even if participants with average skill give an accurate self-assessment of their skill, their performance may be unlucky on that occasion, putting them in the category of low performers who overestimated their skill. According to this approach, the randomness of luck explains the discrepancy between self-assessed ability and objective performance, especially in extreme cases.

Most researchers acknowledge that regression toward the mean is a relevant statistical effect that must be taken into account when interpreting the empirical findings. This can be achieved by various methods. But such adjustments do not eliminate the Dunning–Kruger effect, which is why the view that regression toward the mean is sufficient to explain it is usually rejected. However, it has been suggested that regression toward the mean in combination with other cognitive biases, like the better-than-average effect, can almost completely explain the empirical findings. This type of explanation is sometimes called "noise plus bias". According to the better-than-average effect, people have a general tendency to rate their abilities, attributes, and personality traits as better than average. For example, the average IQ is 100, but people on average think their IQ is 115. The better-than-average effect differs from the Dunning–Kruger effect since it does not track how this overly positive outlook relates to the skill of the people assessing themselves, while the Dunning–Kruger effect mainly focuses on how this type of misjudgment happens for poor performers. When the better-than-average effect is paired with regression toward the mean, it can explain both that unskilled people tend to greatly overestimate their competence and that the reverse effect for highly skilled people is much less pronounced. By choosing the right variables for the randomness due to luck and a positive offset to account for the better-than-average effect, it is possible to simulate experiments that show almost the same correlation between self-assessed ability. This means that the Dunning–Kruger effect may still have a role to play, if only a minor one.

Opponents of this approach have argued that this explanation can account for the Dunning–Kruger effect only when assessing one's ability relative to one's peer group, not when the self-assessment is relative to an objective standard. But even proponents of this explanation agree that this does not explain the empirical findings in full. Moreover, questions have been raised about whether the conditions under which the Dunning-Kruger effect is assessed meet the criteria for a regression to the mean explanation; in short, regression to the mean occurs under conditions of repeated assessment, which is not a feature of a Dunning-Kruger effect experiment. Another statistical-artifact-based challenge to the Dunning-Kruger effect is the demonstration that a form of the effect can emerge when the errors of the self-assessment are randomly created. But the feature of the Dunning-Kruger effect that is not present in analyses of random data is the finding that the magnitude of the errors of self-assessment is larger for those with a low score on the performance assessment than for high scorers on that assessment.

Practical significance

Various claims have been made about the Dunning–Kruger effect's practical significance or why it matters. They often focus on how it causes the affected people to make decisions that lead to dire consequences for them or others. This is especially relevant for decisions that have long-term effects. For example, it can lead poor performers into careers for which they are unfit. High performers underestimating their skills, though, may forego viable career opportunities matching their skills in favor of less promising ones that are below their skill level. In other cases, the wrong decisions can also have severe short-term effects, as when overconfidence leads pilots to operate a new aircraft for which they lack adequate training or to engage in flight maneuvers that exceed their proficiency. Emergency medicine is another area where the correct assessment of one's skills and the risks of treatment is of central importance. The tendencies of physicians in training to be overconfident must be considered to ensure the appropriate degree of supervision and feedback. The Dunning–Kruger effect can also have negative implications for the agent in various economic activities, in which the price of a good, such as a used car, is often lowered by the buyers' uncertainty about its quality. An overconfident agent unaware of their lack of knowledge may be willing to pay a much higher price because they are unaware of all the potential flaws and risks relevant to the price.

Another implication concerns fields in which self-assessments play an essential role in evaluating skills. They are commonly used, for example, in vocational counseling or to estimate the information literacy skills of students and professionals. The Dunning–Kruger effect indicates that such self-assessments often do not correspond to the underlying skills, thereby rendering them unreliable as a method for gathering this type of data. Independent of the field of the skill in question, the metacognitive ignorance often associated with the Dunning–Kruger effect may inhibit low performers from improving themselves. Since they are unaware of many of their flaws, they may have little motivation to address and overcome them.

Not all accounts of the Dunning–Kruger effect focus on its negative sides. Some also concentrate on its positive side, e.g., ignorance can sometimes be bliss. In this sense, optimism can lead people to experience their situation more positively, and overconfidence may help them achieve even unrealistic goals. To distinguish the negative from the positive sides, two important phases have been suggested to be relevant for realizing a goal: preparatory planning and the execution of the plan. Overconfidence may be beneficial in the execution phase by increasing motivation and energy, but it can be detrimental in the planning phase since the agent may ignore bad odds, take unnecessary risks, or fail to prepare for contingencies. For example, being overconfident may be advantageous for a general on the day of battle because of the additional inspiration passed on to his troops, but disadvantageous in the weeks before by ignoring the need for reserve troops or protective gear.

Popular recognition

In 2000, Kruger and Dunning were awarded a satiric Ig Nobel Prize in recognition of the scientific work recorded in "their modest report".
"The Dunning–Kruger Song" is part of ''The Incompetence Opera'', a mini-opera that premiered at the Ig Nobel Prize ceremony in 2017. The mini-opera is billed as "a musical encounter with the Peter principle and the Dunning–Kruger Effect".

See also

*
*
*
*
*
*
*
*
*
*
*
*
* – Passing judgment beyond one's expertise
*
*

References

Notes

Citations

Sources

*

Further reading

*

External links

{{DEFAULTSORT:Dunning-Kruger Effect
Cognitive biases
Cognitive inertia
Incompetence