Heuristic Algorithms

A heuristic (; ), or heuristic technique, is any approach to problem solving or self-discovery, perfect, or rational, but is nevertheless sufficient for reaching an immediate, short-term goal or approximation. Where finding an optimal solution is impossible or impractical, heuristic methods can be used to speed up the process of finding a satisfactory solution. Heuristics can be mental shortcuts that ease the cognitive load of making a decision.

Examples that employ heuristics include using trial and error, a rule of thumb or an educated guess.

Heuristics are the strategies derived from previous experiences with similar problems. These strategies depend on using readily accessible, though loosely applicable, information to control problem solving in human beings, machines and abstract issues. When an individual applies a heuristic in practice, it generally performs as expected. However it can alternatively create systematic errors.

The most fundamental heuristic is trial and error, which can be used in everything from matching nuts and bolts to finding the values of variables in algebra problems. In mathematics, some common heuristics involve the use of visual representations, additional assumptions, forward/backward reasoning and simplification. Here are a few commonly used heuristics from George Pólya's 1945 book, ''How to Solve It'': This is because only the general problem can provide to a specific problem—a context from which to draw meaning.

In psychology, heuristics are simple, efficient rules, either learned or inculcated by evolutionary processes. These psychological heuristics have been proposed to explain how people make decisions, come to judgements, and solve problems. These rules typically come into play when people face complex problems or incomplete information. Researchers employ various methods to test whether people use these rules. The rules have been shown to work well under most circumstances, but in certain cases can lead to systematic errors or cognitive biases.

History

The study of heuristics in human decision-making was developed in the 1970s and the 1980s by the psychologists Amos Tversky and Daniel Kahneman, although the concept had been originally introduced by the Nobel laureate Herbert A. Simon. Simon's original primary object of research was problem solving that showed that we operate within what he calls ''bounded rationality''. He coined the term ''satisficing'', which denotes a situation in which people seek solutions, or accept choices or judgements, that are "good enough" for their purposes although they could be optimised.

Rudolf Groner analysed the history of heuristics from its roots in ancient Greece up to contemporary work in cognitive psychology and artificial intelligence, proposing a cognitive style "heuristic versus algorithmic thinking", which can be assessed by means of a validated questionnaire.

Adaptive toolbox

Gerd Gigerenzer and his research group argued that models of heuristics need to be formal to allow for predictions of behavior that can be tested. They study the fast and frugal heuristics in the "adaptive toolbox" of individuals or institutions, and the ecological rationality of these heuristics; that is, the conditions under which a given heuristic is likely to be successful. The descriptive study of the "adaptive toolbox" is done by observation and experiment, the prescriptive study of the ecological rationality requires mathematical analysis and computer simulation. Heuristics – such as the recognition heuristic, the take-the-best heuristic and fast-and-frugal trees – have been shown to be effective in predictions, particularly in situations of uncertainty. It is often said that heuristics trade accuracy for effort but this is only the case in situations of risk. Risk refers to situations where all possible actions, their outcomes and probabilities are known. In the absence of this information, that is under uncertainty, heuristics can achieve higher accuracy with lower effort. This finding, known as a less-is-more effect, would not have been found without formal models. The valuable insight of this program is that heuristics are effective not despite their simplicity — but because of it. Furthermore, Gigerenzer and Wolfgang Gaissmaier found that both individuals and organisations rely on heuristics in an adaptive way.

Cognitive-experiential self-theory

Heuristics, through greater refinement and research, have begun to be applied to other theories, or be explained by them. For example, the cognitive-experiential self-theory (CEST) also is an adaptive view of heuristic processing. CEST breaks down two systems that process information. At some times, roughly speaking, individuals consider issues rationally, systematically, logically, deliberately, effortfully and verbally. On other occasions, individuals consider issues intuitively, effortlessly, globally, and emotionally. From this perspective, heuristics are part of a larger experiential processing system that is often adaptive, but vulnerable to error in situations that require logical analysis.

Attribute substitution

In 2002, Daniel Kahneman and Shane Frederick proposed that cognitive heuristics work by a process called ''attribute substitution'', which happens without conscious awareness. According to this theory, when somebody makes a judgement (of a "target attribute") that is computationally complex, a more easily calculated "heuristic attribute" is substituted. In effect, a cognitively difficult problem is dealt with by answering a rather simpler problem, without being aware of this happening. This theory explains cases where judgements fail to show regression toward the mean. Heuristics can be considered to reduce the complexity of clinical judgments in health care.

Psychology

Informal models of heuristics

* Affect heuristic: a mental shortcut which uses emotion to influence the decision. Emotion is the effect that plays the lead role that makes the decision or solves the problem quickly or efficiently. It is used while judging the risks and benefits of something, depending on the positive or negative feelings that people associate with a stimulus. It can also be considered the gut decision since if the gut feeling is right, then the benefits are high and the risks are low.
* Anchoring and adjustment: describes the common human tendency to rely more heavily on the first piece of information offered (the "anchor") when making decisions. For example, in a study done with children, the children were told to estimate the number of jellybeans in a jar. Groups of children were given either a high or low "base" number (anchor). Children estimated the number of jellybeans to be closer to the anchor number that they were given.
* Availability heuristic: a mental shortcut that occurs when people make judgements about the probability of events by the ease with which examples come to mind. For example, in a 1973 Tversky & Kahneman experiment, the majority of participants reported that there were more words in the English language that start with the letter K than for which K was the third letter. There are actually twice as many words in the English Language that have K as the third letter as those that start with K, but words that start with K are much easier to recall and bring to mind.
* Balance heuristic: applies to when an individual balances the negative and positive effects from a decision which makes the choice obvious.
* Base rate heuristic: when a decision involves probability this is a mental shortcut that uses relevant data to determine the probability of an outcome occurring. When using this Heuristic there is a common issue where individuals misjudge the likelihood of a situation. For example, if there is a test for a disease which has an accuracy of 90%, people may think it's a 90% they have the disease even though the disease only affects 1 in 500 people.
* Common sense heuristic: used frequently by individuals when the potential outcomes of a decision appear obvious. For example, when your television remote stops working, you would probably change the batteries.
* Contagion heuristic: follows the Law of Contagion or Similarity. This leads people to avoid others that are viewed as "contaminated" to the observer. This happens due to the fact of the observer viewing something that is seen as bad or to seek objects that have been associated with what seems good. Some things one can view as harmful can tend not to really be. This sometimes leads to irrational thinking on behalf of the observer.
* Default heuristic: in real world models, it is common for consumers to apply this heuristic when selecting the default option regardless of whether the option was their preference.
* Educated guess heuristic: when an individual responds to a decision using relevant information they have stored relating to the problem.
* Effort heuristic: the worth of an object is determined by the amount of effort put into the production of the object. Objects that took longer to produce are more valuable while the objects that took less time are deemed not as valuable. Also applies to how much effort is put into achieving the product. This can be seen as the difference of working and earning the object versus finding the object on the side of the street. It can be the same object but the one found will not be deemed as valuable as the one that we earned.
* Escalation of commitment: describes the phenomenon where people justify increased investment in a decision, based on the cumulative prior investment, despite new evidence suggesting that the cost, starting today, of continuing the decision outweighs the expected benefit. This is related to the sunk cost fallacy.
* Fairness heuristic: applies to the reaction of an individual to a decision from an authoritative figure. If the decision is enacted in a fair manner the likelihood of the individual to comply voluntarily is higher than if it is unfair.
* Familiarity heuristic: a mental shortcut applied to various situations in which individuals assume that the circumstances underlying the past behavior still hold true for the present situation and that the past behavior thus can be correctly applied to the new situation. Especially prevalent when the individual experiences a high cognitive load.
* Naïve diversification: when asked to make several choices at once, people tend to diversify more than when making the same type of decision sequentially.
*Peak–end rule: a person's subjective perceptions during the most intense and final moments of an event are averaged together into a single judgment. For example, a person might judge the difficulty of a workout by taking into consideration only the most demanding part of the workout (e.g., Tabata sprints) and what happens at the very end (e.g., a cool-down). In this way, a difficult workout such as the one described here could be perceived as "easier" than a more relaxed workout that did not vary in intensity (e.g., 45 minutes of cycling in aerobic zone 3, without cool-down).
* Representativeness heuristic: a mental shortcut used when making judgements about the probability of an event under uncertainty. Or, judging a situation based on how similar the prospects are to the prototypes the person holds in his or her mind. For example, in a 1982 Tversky and Kahneman experiment, participants were given a description of a woman named Linda. Based on the description, it was likely that Linda was a feminist. Eighty to ninety percent of participants, choosing from two options, chose that it was more likely for Linda to be a feminist ''and'' a bank teller than only a bank teller. The likelihood of two events cannot be greater than that of either of the two events individually. For this reason, the representativeness heuristic is exemplary of the conjunction fallacy.
* Scarcity heuristic: as in economics, the scarcer an object or event is, the more value is attributed to the object or event. The lack of abundance is an indicator of value and provides a mental shortcut that influences the subjective valuation based on how easily the thing might be replaced or lost to competitors. The scarcity heuristic is a cognitive rule that the more difficult it is to acquire an item, the more value that item must have. In many situations we use an item's availability, its perceived abundance, to quickly estimate quality and/or utility. This can lead to systematic judgement errors or cognitive bias.
* Simulation heuristic: a simplified mental strategy in which people determine the likelihood of an event happening based on how easy it is to mentally picture the event happening. People regret the events that are easier to imagine over the ones that would be harder to. It is also thought that people will use this heuristic to predict the likelihood of another's behavior happening. This shows that people are constantly simulating everything around them in order to be able to predict the likelihood of events around them. It is believed that people do this by mentally undoing events that they have experienced and then running mental simulations of the events with the corresponding input values of the altered model.
* Social proof: also known as the informational social influence which was named by Robert Cialdini in his 1984 book Influence. It is where people copy the actions of others. It is more prominent when people are uncertain how to behave, especially in ambiguous social situations.
*Working backward heuristic: when an individual assumes they have already solved a problem they work backwards in order to find how to achieve the solution they originally figured out.

Formal models of heuristics

* Elimination by aspects heuristic
* Fast-and-frugal trees
* Fluency heuristic
* Gaze heuristic
* Recognition heuristic
* Satisficing
* Similarity heuristic
* Take-the-best heuristic

Cognitive maps

Heuristics were also found to be used in the manipulation and creation of cognitive maps. ''Cognitive maps'' are internal representations of our physical environment, particularly associated with spatial relationships. These internal representations are used by our memory as a guide in our external environment. It was found that when questioned about maps imaging, distancing, etc., people commonly made distortions to images. These distortions took shape in the regularisation of images (i.e., images are represented as more like pure abstract geometric images, though they are irregular in shape).

There are several ways that humans form and use cognitive maps, with visual intake being an especially key part of mapping: the first is by using landmarks, whereby a person uses a mental image to estimate a relationship, usually distance, between two objects. The second is route-road knowledge, and is generally developed after a person has performed a task and is relaying the information of that task to another person. The third is a survey, whereby a person estimates a distance based on a mental image that, to them, might appear like an actual map. This image is generally created when a person's brain begins making image corrections. These are presented in five ways:

# Right-angle bias: when a person straightens out an image, like mapping an intersection, and begins to give everything 90-degree angles, when in reality it may not be that way.
# Symmetry heuristic: when people tend to think of shapes, or buildings, as being more symmetrical than they really are.
# Rotation heuristic: when a person takes a naturally (realistically) distorted image and straightens it out for their mental image.
# Alignment heuristic: similar to the previous, where people align objects mentally to make them straighter than they really are.
# Relative-position heuristic: people do not accurately distance landmarks in their mental image based on how well they remember them.

Another method of creating cognitive maps is by means of auditory intake based on verbal descriptions. Using the mapping based from a person's visual intake, another person can create a mental image, such as directions to a certain location.

Philosophy

A heuristic device is used when an entity ''X'' exists to enable understanding of, or knowledge concerning, some other entity ''Y''.

A good example is a model that, as it is never identical with what it models, is a heuristic device to enable understanding of what it models. Stories, metaphors, etc., can also be termed heuristic in this sense. A classic example is the notion of utopia as described in Plato's best-known work, '' The Republic''. This means that the "ideal city" as depicted in ''The Republic'' is not given as something to be pursued, or to present an orientation-point for development. Rather, it shows how things would have to be connected, and how one thing would lead to another (often with highly problematic results), if one opted for certain principles and carried them through rigorously.

''Heuristic'' is also often used as a noun to describe a rule-of-thumb, procedure, or method. Philosophers of science have emphasised the importance of heuristics in creative thought and the construction of scientific theories. Seminal works include Karl Popper's ''The Logic of Scientific Discovery'' and others by Imre Lakatos, Lindley Darden, and William C. Wimsatt.

Law

In legal theory, especially in the theory of law and economics, heuristics are used in the law when case-by-case analysis would be impractical, insofar as "practicality" is defined by the interests of a governing body.

The present securities regulation regime largely assumes that all investors act as perfectly rational persons.
In truth, actual investors face cognitive limitations from biases, heuristics, and framing effects. For instance, in all states in the United States the legal drinking age for unsupervised persons is 21 years, because it is argued that people need to be mature enough to make decisions involving the risks of alcohol consumption. However, assuming people mature at different rates, the specific age of 21 would be too late for some and too early for others. In this case, the somewhat arbitrary deadline is used because it is impossible or impractical to tell whether an individual is sufficiently mature for society to trust them with that kind of responsibility. Some proposed changes, however, have included the completion of an alcohol education course rather than the attainment of 21 years of age as the criterion for legal alcohol possession. This would put youth alcohol policy more on a case-by-case basis and less on a heuristic one, since the completion of such a course would presumably be voluntary and not uniform across the population.

The same reasoning applies to patent law. Patents are justified on the grounds that inventors must be protected so they have incentive to invent. It is therefore argued that it is in society's best interest that inventors receive a temporary government-granted monopoly on their idea, so that they can recoup investment costs and make economic profit for a limited period. In the United States, the length of this temporary monopoly is 20 years from the date the patent application was filed, though the monopoly does not actually begin until the application has matured into a patent. However, like the drinking-age problem above, the specific length of time would need to be different for every product to be efficient. A 20-year term is used because it is difficult to tell what the number should be for any individual patent. More recently, some, including University of North Dakota law professor Eric E. Johnson, have argued that patents in different kinds of industries – such as software patents – should be protected for different lengths of time.

Stereotyping

Stereotyping is a type of heuristic that people use to form opinions or make judgements about things they have never seen or experienced. They work as a mental shortcut to assess everything from the social status of a person (based on their actions), to whether a plant is a tree based on the assumption that it is tall, has a trunk and has leaves (even though the person making the evaluation might never have seen that particular type of tree before).

Stereotypes, as first described by journalist Walter Lippmann in his book ''Public Opinion'' (1922), are the pictures we have in our heads that are built around experiences as well as what we are told about the world.

Artificial intelligence

A heuristic can be used in artificial intelligence systems while searching a solution space. The heuristic is derived by using some function that is put into the system by the designer, or by adjusting the weight of branches based on how likely each branch is to lead to a goal node.

See also

* Algorithm
* Behavioral economics
* Failure mode and effects analysis
* Heuristics in judgment and decision-making
* Ideal type
* List of biases in judgment and decision making
* Neuroheuristics
* Predictive coding
* Priority heuristic
* Social heuristics
* Thought experiment

References

Further reading

* ''How To Solve It: Modern Heuristics'', Zbigniew Michalewicz and David B. Fogel, Springer Verlag, 2000.
*

The Problem of Thinking Too Much
11 December 2002, Persi Diaconis

{{Authority control

Problem solving methods