In

crossover
CrossOver is a Microsoft Windows
Microsoft Windows, commonly referred to as Windows, is a group of several Proprietary software, proprietary graphical user interface, graphical operating system families, all of which are developed and markete ...

probability and population size to find reasonable settings for the problem class being worked on. A very small mutation rate may lead to

fitness function{{no footnotes, date=May 2015
A fitness function is a particular type of objective function that is used to summarise, as a single figure of merit, how close a given design solution is to achieving the set aims. Fitness functions are used in genetic ...

evaluations. In real world problems such as structural optimization problems, a single function evaluation may require several hours to several days of complete simulation. Typical optimization methods cannot deal with such types of problem. In this case, it may be necessary to forgo an exact evaluation and use an approximated fitness that is computationally efficient. It is apparent that amalgamation of approximate models may be one of the most promising approaches to convincingly use GA to solve complex real life problems.
* Genetic algorithms do not scale well with complexity. That is, where the number of elements which are exposed to mutation is large there is often an exponential increase in search space size. This makes it extremely difficult to use the technique on problems such as designing an engine, a house or a plane . In order to make such problems tractable to evolutionary search, they must be broken down into the simplest representation possible. Hence we typically see evolutionary algorithms encoding designs for fan blades instead of engines, building shapes instead of detailed construction plans, and airfoils instead of whole aircraft designs. The second problem of complexity is the issue of how to protect parts that have evolved to represent good solutions from further destructive mutation, particularly when their fitness assessment requires them to combine well with other parts.
* The "better" solution is only in comparison to other solutions. As a result, the stop criterion is not clear in every problem.
* In many problems, GAs have a tendency to converge towards evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

,

evolution strategies
In computer science, an evolution strategy (ES) is an optimization (mathematics), optimization technique based on ideas of evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.
History
The 'ev ...

and evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

. The notion of real-valued genetic algorithms has been offered but is really a misnomer because it does not really represent the building block theory that was proposed by

mutation
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...

in combination with crossover
CrossOver is a Microsoft Windows
Microsoft Windows, commonly referred to as Windows, is a group of several Proprietary software, proprietary graphical user interface, graphical operating system families, all of which are developed and markete ...

, is designed to move the population away from

evolution strategies
In computer science, an evolution strategy (ES) is an optimization (mathematics), optimization technique based on ideas of evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.
History
The 'ev ...

. Another approach was the evolutionary programming technique of Lawrence J. Fogel, which was proposed for generating artificial intelligence.

Provides a list of resources in the genetic algorithms field

An Overview of the History and Flavors of Evolutionary Algorithms

An excellent introduction to GA by John Holland and with an application to the Prisoner's Dilemma * [http://www.i4ai.org/EA-demo/ An online interactive Genetic Algorithm tutorial for a reader to practise or learn how a GA works]: Learn step by step or watch global convergence in batch, change the population size, crossover rates/bounds, mutation rates/bounds and selection mechanisms, and add constraints.

A Genetic Algorithm Tutorial by Darrell Whitley Computer Science Department Colorado State University

An excellent tutorial with much theory

"Essentials of Metaheuristics"

2009 (225 p). Free open text by Sean Luke.

Global Optimization Algorithms – Theory and Application

Tutorial with the intuition behind GAs and Python implementation.

Genetic Algorithms evolves to solve the prisoner's dilemma.

Written by Robert Axelrod. {{DEFAULTSORT:Genetic Algorithm Genetic algorithms, Evolutionary algorithms Search algorithms Cybernetics Digital organisms Machine learning sv:Genetisk programmering#Genetisk algoritm

computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of computation, automation, a ...

and operations research, a genetic algorithm (GA) is a metaheuristic
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , , a ...

inspired by the process of natural selection
Natural selection is the differential survival and reproduction of individuals due to differences in phenotype
right , Here the relation between genotype and phenotype is illustrated, using a Punnett square, for the character of peta ...

that belongs to the larger class of evolutionary algorithm
In computational intelligence (CI), an evolutionary algorithm (EA) is a subset of evolutionary computation, a generic population-based metaheuristic optimization (mathematics), optimization algorithm. An EA uses mechanisms inspired by biological ev ...

s (EA). Genetic algorithms are commonly used to generate high-quality solutions to optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. Optimization problems of sorts arise i ...

and search problem
In computational complexity theory
Computational complexity theory focuses on classifying computational problem
In theoretical computer science
An artistic representation of a Turing machine. Turing machines are used to model general computing ...

s by relying on biologically inspired operators such as mutation
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...

, crossover
CrossOver is a Microsoft Windows
Microsoft Windows, commonly referred to as Windows, is a group of several Proprietary software, proprietary graphical user interface, graphical operating system families, all of which are developed and markete ...

and selection
Selection may refer to:
In science:
* Selection (biology)
Natural selection is the differential survival and reproduction of individuals due to differences in phenotype
right , Here the relation between genotype and phenotype is ill ...

. Some examples of GA applications include optimizing decision trees
A decision tree is a decision support tool that uses a tree-like model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility
Within economics, the concept of utility is used to model wo ...

for better performance, automatically solve sudoku puzzles, hyperparameter optimizationIn machine learning, hyperparameter optimization or tuning is the problem of choosing a set of optimal Hyperparameter (machine learning), hyperparameters for a learning algorithm. A hyperparameter is a parameter whose value is used to control the lea ...

, etc.
Methodology

Optimization problems

In a genetic algorithm, apopulation
Population typically refers the number of people in a single area whether it be a city or town, region, country, or the world. Governments typically quantify the size of the resident population within their jurisdiction by a process called a ...

of candidate solution
In mathematical optimization, a feasible region, feasible set, search space, or solution space is the set of all possible points (sets of values of the choice variables) of an optimization problem that satisfy the problem's Constraint (mathemati ...

s (called individuals, creatures, or phenotype
In genetics
Genetics is a branch of biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular inter ...

s) to an optimization problem is evolved toward better solutions. Each candidate solution has a set of properties (its chromosome
A chromosome is a long DNA molecule with part or all of the genome, genetic material of an organism. Most eukaryotic chromosomes include packaging proteins called histones which, aided by Chaperone (protein), chaperone proteins, bind to and ...

s or genotype
The genotype of an organism is its complete set of genetic material. Genotype can also be used to refer to the or variants an individual carries in a particular gene or genetic location. The number of alleles an individual can have in a specific ...

) which can be mutated and altered; traditionally, solutions are represented in binary as strings of 0s and 1s, but other encodings are also possible.
The evolution usually starts from a population of randomly generated individuals, and is an iterative process, with the population in each iteration called a ''generation''. In each generation, the fitness of every individual in the population is evaluated; the fitness is usually the value of the objective functionIn mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event (probability theory), event or values of one or more variables onto a real number intuitive ...

in the optimization problem being solved. The more fit individuals are stochastically selected from the current population, and each individual's genome is modified ( recombined and possibly randomly mutated) to form a new generation. The new generation of candidate solutions is then used in the next iteration of the algorithm
In and , an algorithm () is a finite sequence of , computer-implementable instructions, typically to solve a class of problems or to perform a computation. Algorithms are always and are used as specifications for performing s, , , and other ...

. Commonly, the algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population.
A typical genetic algorithm requires:
# a genetic representation
In computer programming, genetic representation is a way of representing solutions/individuals in evolutionary computation methods. Genetic representation can encode appearance, behavior, physical qualities of individuals. Designing a good genetic r ...

of the solution domain,
# a fitness function{{no footnotes, date=May 2015
A fitness function is a particular type of objective function that is used to summarise, as a single figure of merit, how close a given design solution is to achieving the set aims. Fitness functions are used in genetic ...

to evaluate the solution domain.
A standard representation of each candidate solution is as an array of bits (also called ''bit set'' or ''bit string''). Arrays of other types and structures can be used in essentially the same way. The main property that makes these genetic representations convenient is that their parts are easily aligned due to their fixed size, which facilitates simple crossover
CrossOver is a Microsoft Windows
Microsoft Windows, commonly referred to as Windows, is a group of several Proprietary software, proprietary graphical user interface, graphical operating system families, all of which are developed and markete ...

operations. Variable length representations may also be used, but crossover implementation is more complex in this case. Tree-like representations are explored in genetic programming
In artificial intelligence, genetic programming (GP) is a technique of evolving programs, starting from a population of unfit (usually random) programs, fit for a particular task by applying operations analogous to natural genetic processes to the ...

and graph-form representations are explored in evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

; a mix of both linear chromosomes and trees is explored in gene expression programming
In computer programming, gene expression programming (GEP) is an evolutionary algorithm that creates computer programs or models. These computer programs are complex tree structures that learn and adapt by changing their sizes, shapes, and composi ...

.
Once the genetic representation and the fitness function are defined, a GA proceeds to initialize a population of solutions and then to improve it through repetitive application of the mutation, crossover, inversion and selection operators.
Initialization

The population size depends on the nature of the problem, but typically contains several hundreds or thousands of possible solutions. Often, the initial population is generated randomly, allowing the entire range of possible solutions (the ). Occasionally, the solutions may be "seeded" in areas where optimal solutions are likely to be found.Selection

During each successive generation, a portion of the existing population is selected to breed a new generation. Individual solutions are selected through a ''fitness-based'' process, where fitter solutions (as measured by afitness function{{no footnotes, date=May 2015
A fitness function is a particular type of objective function that is used to summarise, as a single figure of merit, how close a given design solution is to achieving the set aims. Fitness functions are used in genetic ...

) are typically more likely to be selected. Certain selection methods rate the fitness of each solution and preferentially select the best solutions. Other methods rate only a random sample of the population, as the former process may be very time-consuming.
The fitness function is defined over the genetic representation and measures the ''quality'' of the represented solution. The fitness function is always problem dependent. For instance, in the knapsack problem
The knapsack problem is a problem in combinatorial optimization
Combinatorial optimization is a subfield of mathematical optimization that is related to operations research, algorithm, algorithm theory, and computational complexity theory. It ha ...

one wants to maximize the total value of objects that can be put in a knapsack of some fixed capacity. A representation of a solution might be an array of bits, where each bit represents a different object, and the value of the bit (0 or 1) represents whether or not the object is in the knapsack. Not every such representation is valid, as the size of objects may exceed the capacity of the knapsack. The ''fitness'' of the solution is the sum of values of all objects in the knapsack if the representation is valid, or 0 otherwise.
In some problems, it is hard or even impossible to define the fitness expression; in these cases, a simulation
A simulation is the imitation of the operation of a real-world process or system over time. Simulations require the use of models; the model represents the key characteristics or behaviors of the selected system or process, whereas the simulat ...

may be used to determine the fitness function value of a phenotype
In genetics
Genetics is a branch of biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular inter ...

(e.g. computational fluid dynamics#REDIRECT Computational fluid dynamics
Computational fluid dynamics (CFD) is a branch of fluid mechanics that uses numerical analysis and data structures to analyze and solve problems that involve fluid dynamics, fluid flows. Computers are used ...

is used to determine the air resistance of a vehicle whose shape is encoded as the phenotype), or even interactive genetic algorithms are used.
Genetic operators

The next step is to generate a second generation population of solutions from those selected through a combination ofgenetic operatorA genetic operator is an Operator (programming), operator used in genetic algorithms to guide the algorithm towards a solution to a given problem. There are three main types of operators (Mutation (genetic algorithm) , mutation, Crossover (genetic al ...

s: crossover
CrossOver is a Microsoft Windows
Microsoft Windows, commonly referred to as Windows, is a group of several Proprietary software, proprietary graphical user interface, graphical operating system families, all of which are developed and markete ...

(also called recombination), and mutation
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...

.
For each new solution to be produced, a pair of "parent" solutions is selected for breeding from the pool selected previously. By producing a "child" solution using the above methods of crossover and mutation, a new solution is created which typically shares many of the characteristics of its "parents". New parents are selected for each new child, and the process continues until a new population of solutions of appropriate size is generated.
Although reproduction methods that are based on the use of two parents are more "biology inspired", some research suggests that more than two "parents" generate higher quality chromosomes.
These processes ultimately result in the next generation population of chromosomes that is different from the initial generation. Generally, the average fitness will have increased by this procedure for the population, since only the best organisms from the first generation are selected for breeding, along with a small proportion of less fit solutions. These less fit solutions ensure genetic diversity within the genetic pool of the parents and therefore ensure the genetic diversity of the subsequent generation of children.
Opinion is divided over the importance of crossover versus mutation. There are many references in Fogel (2006) that support the importance of mutation-based search.
Although crossover and mutation are known as the main genetic operators, it is possible to use other operators such as regrouping, colonization-extinction, or migration in genetic algorithms.
It is worth tuning parameters such as the mutation
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...

probability, genetic drift
Genetic drift (allelic drift or the Sewall Wright effect) is the change in the frequency of an existing gene
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical stru ...

(which is non-ergodic
In mathematics, ergodicity expresses the idea that a point of a moving system, either a dynamical system or a stochastic process, will eventually visit all parts of the space that the system moves in, in a uniform and random sense. This implies that ...

in nature). A recombination rate that is too high may lead to premature convergence of the genetic algorithm. A mutation rate that is too high may lead to loss of good solutions, unless elitist selection is employed. An adequate population size ensures sufficient genetic diversity for the problem at hand, but can lead to a waste of computational resources if set to a value larger than required.
Heuristics

In addition to the main operators above, otherheuristic
A heuristic (; ), or heuristic technique, is any approach to problem solving or self-discovery that employs a practical method that is not guaranteed to be Mathematical optimisation, optimal, perfect, or Rationality, rational, but is nevertheless ...

s may be employed to make the calculation faster or more robust. The ''speciation'' heuristic penalizes crossover between candidate solutions that are too similar; this encourages population diversity and helps prevent premature convergence
Convergence may refer to:
Arts and media Literature
*Convergence (book series), ''Convergence'' (book series), edited by Ruth Nanda Anshen
*Convergence (comics), "Convergence" (comics), two separate story lines published by DC Comics:
**A four-par ...

to a less optimal solution.
Termination

This generational process is repeated until a termination condition has been reached. Common terminating conditions are: * A solution is found that satisfies minimum criteria * Fixed number of generations reached * Allocated budget (computation time/money) reached * The highest ranking solution's fitness is reaching or has reached a plateau such that successive iterations no longer produce better results * Manual inspection * Combinations of the aboveThe building block hypothesis

Genetic algorithms are simple to implement, but their behavior is difficult to understand. In particular, it is difficult to understand why these algorithms frequently succeed at generating solutions of high fitness when applied to practical problems. The building block hypothesis (BBH) consists of: # A description of a heuristic that performs adaptation by identifying and recombining "building blocks", i.e. low order, low defining-length schemata with above average fitness. # A hypothesis that a genetic algorithm performs adaptation by implicitly and efficiently implementing this heuristic. Goldberg describes the heuristic as follows: :"Short, low order, and highly fit schemata are sampled, recombined rossed over and resampled to form strings of potentially higher fitness. In a way, by working with these particular schemata he building blocks we have reduced the complexity of our problem; instead of building high-performance strings by trying every conceivable combination, we construct better and better strings from the best partial solutions of past samplings. :"Because highly fit schemata of low defining length and low order play such an important role in the action of genetic algorithms, we have already given them a special name: building blocks. Just as a child creates magnificent fortresses through the arrangement of simple blocks of wood, so does a genetic algorithm seek near optimal performance through the juxtaposition of short, low-order, high-performance schemata, or building blocks." Despite the lack of consensus regarding the validity of the building-block hypothesis, it has been consistently evaluated and used as reference throughout the years. Manyestimation of distribution algorithmImage:Eda mono-variant gauss iterations.svg, 350px, Estimation of distribution algorithm. For each iteration ''i'', a random draw is performed for a population ''P'' in a distribution ''PDu''. The distribution parameters ''PDe'' are then estimated us ...

s, for example, have been proposed in an attempt to provide an environment in which the hypothesis would hold. Although good results have been reported for some classes of problems, skepticism concerning the generality and/or practicality of the building-block hypothesis as an explanation for GAs efficiency still remains. Indeed, there is a reasonable amount of work that attempts to understand its limitations from the perspective of estimation of distribution algorithms.
Limitations

There are limitations of the use of a genetic algorithm compared to alternative optimization algorithms: * Repeatedfitness function{{no footnotes, date=May 2015
A fitness function is a particular type of objective function that is used to summarise, as a single figure of merit, how close a given design solution is to achieving the set aims. Fitness functions are used in genetic ...

evaluation for complex problems is often the most prohibitive and limiting segment of artificial evolutionary algorithms. Finding the optimal solution to complex high-dimensional, multimodal problems often requires very expensive local optima
233px, Polynomial of degree 4: the trough on the right is a local minimum and the one on the left is the global minimum. The peak in the center is a local maximum.
In applied mathematics
Applied mathematics is the application of mathematical ...

or even arbitrary points rather than the global optimum
In mathematical analysis, the maxima and minima (the respective plurals of maximum and minimum) of a function, known collectively as extrema (the plural of extremum), are the largest and smallest value of the function, either within a given ra ...

of the problem. This means that it does not "know how" to sacrifice short-term fitness to gain longer-term fitness. The likelihood of this occurring depends on the shape of the fitness landscape
In evolutionary biology
Evolutionary biology is the subfield of biology that studies the evolution, evolutionary processes (natural selection, common descent, speciation) that produced the Biodiversity, diversity of life on Earth. In the 1930s ...

: certain problems may provide an easy ascent towards a global optimum, others may make it easier for the function to find the local optima. This problem may be alleviated by using a different fitness function, increasing the rate of mutation, or by using selection techniques that maintain a diverse population of solutions, although the No Free Lunch theorem proves that there is no general solution to this problem. A common technique to maintain diversity is to impose a "niche penalty", wherein, any group of individuals of sufficient similarity (niche radius) have a penalty added, which will reduce the representation of that group in subsequent generations, permitting other (less similar) individuals to be maintained in the population. This trick, however, may not be effective, depending on the landscape of the problem. Another possible technique would be to simply replace part of the population with randomly generated individuals, when most of the population is too similar to each other. Diversity is important in genetic algorithms (and genetic programming
In artificial intelligence, genetic programming (GP) is a technique of evolving programs, starting from a population of unfit (usually random) programs, fit for a particular task by applying operations analogous to natural genetic processes to the ...

) because crossing over a homogeneous population does not yield new solutions. In evolution strategies
In computer science, an evolution strategy (ES) is an optimization (mathematics), optimization technique based on ideas of evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.
History
The 'ev ...

and evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

, diversity is not essential because of a greater reliance on mutation.
* Operating on dynamic data sets is difficult, as genomes begin to converge early on towards solutions which may no longer be valid for later data. Several methods have been proposed to remedy this by increasing genetic diversity somehow and preventing early convergence, either by increasing the probability of mutation when the solution quality drops (called ''triggered hypermutation''), or by occasionally introducing entirely new, randomly generated elements into the gene pool (called ''random immigrants''). Again, evolution strategies
In computer science, an evolution strategy (ES) is an optimization (mathematics), optimization technique based on ideas of evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.
History
The 'ev ...

and evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

can be implemented with a so-called "comma strategy" in which parents are not maintained and new parents are selected only from offspring. This can be more effective on dynamic problems.
* GAs cannot effectively solve problems in which the only fitness measure is a single right/wrong measure (like decision problem
In computability theory and computational complexity theory, a decision problem is a problem that can be posed as a yes–no question of the input values. An example of a decision problem is deciding whether a given natural number is prime. Anot ...

s), as there is no way to converge on the solution (no hill to climb). In these cases, a random search may find a solution as quickly as a GA. However, if the situation allows the success/failure trial to be repeated giving (possibly) different results, then the ratio of successes to failures provides a suitable fitness measure.
* For specific optimization problems and problem instances, other optimization algorithms may be more efficient than genetic algorithms in terms of speed of convergence. Alternative and complementary algorithms include evolution strategies
In computer science, an evolution strategy (ES) is an optimization (mathematics), optimization technique based on ideas of evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.
History
The 'ev ...

, simulated annealing
Simulated annealing (SA) is a probabilistic technique for approximating the global optimum
In mathematical analysis, the maxima and minima (the respective plurals of maximum and minimum) of a function, known collectively as extrema ( ...

, Gaussian adaptation, hill climbing
In numerical analysis, hill climbing is a Optimization (mathematics), mathematical optimization technique which belongs to the family of Local search (optimization), local search. It is an iterative algorithm that starts with an arbitrary solut ...

, and swarm intelligence
Swarm intelligence (SI) is the collective behavior
The expression collective behavior was first used by Franklin Henry Giddings
Franklin Henry Giddings (March 23, 1855 – June 11, 1931) was an American sociologist and economist
...

(e.g.: ant colony optimization
In computer science and operations research, the ant colony optimization algorithm (ACO) is a probability, probabilistic technique for solving computational problems which can be reduced to finding good paths through Graph (discrete mathematics) ...

, particle swarm optimization
In , particle swarm optimization (PSO) is a computational method that a problem by trying to improve a with regard to a given measure of quality. It solves a problem by having a population of candidate solutions, here dubbed s, and moving the ...

) and methods based on integer linear programming
An integer programming problem is a mathematical optimization or Constraint satisfaction problem, feasibility program in which some or all of the variables are restricted to be integers. In many settings the term refers to integer linear programming ...

. The suitability of genetic algorithms is dependent on the amount of knowledge of the problem; well known problems often have better, more specialized approaches.
Variants

Chromosome representation

The simplest algorithm represents each chromosome as abit string
A bit array (also known as bit map, bit set, bit string, or bit vector) is an array data structure
ARRAY, also known as ARRAY Now, is an independent distribution company launched by film maker and former publicist Ava DuVernay
Ava Marie DuV ...

. Typically, numeric parameters can be represented by integer
An integer (from the Latin
Latin (, or , ) is a classical language
A classical language is a language
A language is a structured system of communication
Communication (from Latin ''communicare'', meaning "to share" or "to ...

s, though it is possible to use floating point
In computing
Computing is any goal-oriented activity requiring, benefiting from, or creating computing machinery. It includes the study and experimentation of algorithmic processes and development of both computer hardware , hardware and soft ...

representations. The floating point representation is natural to John Henry Holland
John Henry Holland (February 2, 1929 – August 9, 2015) was an American scientist and Professor of psychology and Professor of electrical engineering and computer science at the University of Michigan, Ann Arbor. He was a pioneer in what became k ...

in the 1970s. This theory is not without support though, based on theoretical and experimental results (see below). The basic algorithm performs crossover and mutation at the bit level. Other variants treat the chromosome as a list of numbers which are indexes into an instruction table, nodes in a linked list
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , ...

, hashes, objects, or any other imaginable data structure
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of ...

. Crossover and mutation are performed so as to respect data element boundaries. For most data types, specific variation operators can be designed. Different chromosomal data types seem to work better or worse for different specific problem domains.
When bit-string representations of integers are used, Gray coding
The reflected binary code (RBC), also known just as reflected binary (RB) or Gray code after Frank Gray
Francis Tierney Gray (born 27 October 1954) is a Scottish Association football, football manager (association football), manager and for ...

is often employed. In this way, small changes in the integer can be readily affected through mutations or crossovers. This has been found to help prevent premature convergence at so-called ''Hamming walls'', in which too many simultaneous mutations (or crossover events) must occur in order to change the chromosome to a better solution.
Other approaches involve using arrays of real-valued numbers instead of bit strings to represent chromosomes. Results from the theory of schemata suggest that in general the smaller the alphabet, the better the performance, but it was initially surprising to researchers that good results were obtained from using real-valued chromosomes. This was explained as the set of real values in a finite population of chromosomes as forming a ''virtual alphabet'' (when selection and recombination are dominant) with a much lower cardinality than would be expected from a floating point representation.
An expansion of the Genetic Algorithm accessible problem domain can be obtained through more complex encoding of the solution pools by concatenating several types of heterogenously encoded genes into one chromosome. This particular approach allows for solving optimization problems that require vastly disparate definition domains for the problem parameters. For instance, in problems of cascaded controller tuning, the internal loop controller structure can belong to a conventional regulator of three parameters, whereas the external loop could implement a linguistic controller (such as a fuzzy system) which has an inherently different description. This particular form of encoding requires a specialized crossover mechanism that recombines the chromosome by section, and it is a useful tool for the modelling and simulation of complex adaptive systems, especially evolution processes.
Elitism

A practical variant of the general process of constructing a new population is to allow the best organism(s) from the current generation to carry over to the next, unaltered. This strategy is known as ''elitist selection'' and guarantees that the solution quality obtained by the GA will not decrease from one generation to the next.Parallel implementations

Parallel
Parallel may refer to:
Computing
* Parallel algorithm
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their a ...

implementations of genetic algorithms come in two flavors. Coarse-grained parallel genetic algorithms assume a population on each of the computer nodes and migration of individuals among the nodes. Fine-grained parallel genetic algorithms assume an individual on each processor node which acts with neighboring individuals for selection and reproduction.
Other variants, like genetic algorithms for online optimization Online optimization is a field of optimization theory, more popular in computer science and operations research, that deals with optimization problems having no or incomplete knowledge of the future (online). These kind of problems are denoted as on ...

problems, introduce time-dependence or noise in the fitness function.
Adaptive GAs

Genetic algorithms with adaptive parameters (adaptive genetic algorithms, AGAs) is another significant and promising variant of genetic algorithms. The probabilities of crossover (pc) and mutation (pm) greatly determine the degree of solution accuracy and the convergence speed that genetic algorithms can obtain. Instead of using fixed values of ''pc'' and ''pm'', AGAs utilize the population information in each generation and adaptively adjust the ''pc'' and ''pm'' in order to maintain the population diversity as well as to sustain the convergence capacity. In AGA (adaptive genetic algorithm), the adjustment of ''pc'' and ''pm'' depends on the fitness values of the solutions. In ''CAGA'' (clustering-based adaptive genetic algorithm), through the use of clustering analysis to judge the optimization states of the population, the adjustment of ''pc'' and ''pm'' depends on these optimization states. It can be quite effective to combine GA with other optimization methods. GA tends to be quite good at finding generally good global solutions, but quite inefficient at finding the last few mutations to find the absolute optimum. Other techniques (such as simple hill climbing) are quite efficient at finding absolute optimum in a limited region. Alternating GA and hill climbing can improve the efficiency of GA while overcoming the lack of robustness of hill climbing. This means that the rules of genetic variation may have a different meaning in the natural case. For instance – provided that steps are stored in consecutive order – crossing over may sum a number of steps from maternal DNA adding a number of steps from paternal DNA and so on. This is like adding vectors that more probably may follow a ridge in the phenotypic landscape. Thus, the efficiency of the process may be increased by many orders of magnitude. Moreover, the inversion operator has the opportunity to place steps in consecutive order or any other suitable order in favour of survival or efficiency. A variation, where the population as a whole is evolved rather than its individual members, is known as gene pool recombination. A number of variations have been developed to attempt to improve performance of GAs on problems with a high degree of fitness epistasis, i.e. where the fitness of a solution consists of interacting subsets of its variables. Such algorithms aim to learn (before exploiting) these beneficial phenotypic interactions. As such, they are aligned with the Building Block Hypothesis in adaptively reducing disruptive recombination. Prominent examples of this approach include the mGA, GEMGA and LLGA.Problem domains

Problems which appear to be particularly appropriate for solution by genetic algorithms include and scheduling problems, and many scheduling software packages are based on GAs. GAs have also been applied toengineering
Engineering is the use of scientific principles to design and build machines, structures, and other items, including bridges, tunnels, roads, vehicles, and buildings. The discipline of engineering encompasses a broad range of more speciali ...

. Genetic algorithms are often applied as an approach to solve global optimization
Global optimization is a branch of applied mathematics and numerical analysis that attempts to find the global minima or maxima of a function or a set of functions on a given set. It is usually described as a minimization problem because the max ...

problems.
As a general rule of thumb genetic algorithms might be useful in problem domains that have a complex fitness landscape
In evolutionary biology
Evolutionary biology is the subfield of biology that studies the evolution, evolutionary processes (natural selection, common descent, speciation) that produced the Biodiversity, diversity of life on Earth. In the 1930s ...

as mixing, i.e., local optima
233px, Polynomial of degree 4: the trough on the right is a local minimum and the one on the left is the global minimum. The peak in the center is a local maximum.
In applied mathematics
Applied mathematics is the application of mathematical ...

that a traditional hill climbing
In numerical analysis, hill climbing is a Optimization (mathematics), mathematical optimization technique which belongs to the family of Local search (optimization), local search. It is an iterative algorithm that starts with an arbitrary solut ...

algorithm might get stuck in. Observe that commonly used crossover operators cannot change any uniform population. Mutation alone can provide ergodicity of the overall genetic algorithm process (seen as a Markov chain
A Markov chain or Markov process is a stochastic model
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory tr ...

).
Examples of problems solved by genetic algorithms include: mirrors designed to funnel sunlight to a solar collector, antennae designed to pick up radio signals in space, walking methods for computer figures, optimal design of aerodynamic bodies in complex flowfields
In his ''Algorithm Design Manual'', advises against genetic algorithms for any task:
History

In 1950,Alan Turing
Alan Mathison Turing (; 23 June 1912 – 7 June 1954) was an English mathematician
A mathematician is someone who uses an extensive knowledge of mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such to ...

proposed a "learning machine" which would parallel the principles of evolution. Computer simulation of evolution started as early as in 1954 with the work of Nils Aall Barricelli
Nils Aall Barricelli (24 January 1912 – 27 January 1993) was a Norwegians, Norwegian-Italians, Italian mathematician.
Barricelli's early computer-assisted experiments in symbiogenesis and evolution are considered pioneering in artificial lif ...

, who was using the computer at the Institute for Advanced Study
The Institute for Advanced Study (IAS), located in Princeton, New Jersey, in the United States, is an independent center for theoretical research and intellectual inquiry. It has served as the academic home of internationally preeminent scholar ...

in Princeton, New Jersey
Princeton is a municipality with a borough
A borough is an administrative division in various English language, English-speaking countries. In principle, the term ''borough'' designates a self-governing walled town, although in practice, off ...

. His 1954 publication was not widely noticed. Starting in 1957, the Australian quantitative geneticist Alex Fraser published a series of papers on simulation of artificial selection
This Chihuahua (dog), Chihuahua mixed-breed dog, mix and Great Dane shows the wide range of dog breed sizes created using selective breeding.
Selective breeding (also called artificial selection) is the process by which humans use animal breed ...

of organisms with multiple loci controlling a measurable trait. From these beginnings, computer simulation of evolution by biologists became more common in the early 1960s, and the methods were described in books by Fraser and Burnell (1970) and Crosby (1973). Fraser's simulations included all of the essential elements of modern genetic algorithms. In addition, Hans-Joachim Bremermann
Hans-Joachim Bremermann (1926–1996) was a German-American mathematician and biophysicist. He worked on computer science and evolution, introducing new ideas of how mating generates new gene combinations. Bremermann's limit, named after him, is ...

published a series of papers in the 1960s that also adopted a population of solution to optimization problems, undergoing recombination, mutation, and selection. Bremermann's research also included the elements of modern genetic algorithms. Other noteworthy early pioneers include Richard Friedberg, George Friedman, and Michael Conrad. Many early papers are reprinted by Fogel (1998).
Although Barricelli, in work he reported in 1963, had simulated the evolution of ability to play a simple game, artificial evolution
In computational intelligence (CI), an evolutionary algorithm (EA) is a subset of evolutionary computation, a generic population-based metaheuristic optimization (mathematics), optimization algorithm. An EA uses mechanisms inspired by biological ev ...

only became a widely recognized optimization method as a result of the work of Ingo Rechenberg
Ingo Rechenberg (born November 20, 1934) is a Germany, German researcher and professor currently in the field of bionics. Rechenberg is a pioneer of the fields of evolutionary computation and artificial evolution. In the 1960s and 1970s he invente ...

and Hans-Paul Schwefel
Hans-Paul Schwefel (born December 4, 1940) is a German computer scientist and professor emeritus at University of Dortmund (now Dortmund University of Technology), where he held the chair of systems analysis from 1985 until 2006. He is one of the p ...

in the 1960s and early 1970s – Rechenberg's group was able to solve complex engineering problems through Evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

originally used finite state machines for predicting environments, and used variation and selection to optimize the predictive logics. Genetic algorithms in particular became popular through the work of John Holland in the early 1970s, and particularly his book ''Adaptation in Natural and Artificial Systems'' (1975). His work originated with studies of cellular automata
A cellular automaton (pl. cellular automata, abbrev. CA) is a discrete model of computation studied in automata theory. Cellular automata are also called cellular spaces, tessellation automata, homogeneous structures, cellular structures, tesse ...

, conducted by Holland
Holland is a geographical region
In geography, regions are areas that are broadly divided by physical characteristics (physical geography), human impact characteristics (human geography), and the interaction of humanity and the environment (en ...

and his students at the University of Michigan
, mottoeng = "Arts, Knowledge, Truth"
, former_names = Catholepistemiad, or University of Michigania (1817–1821)
, budget = $8.99 billion (2018)
, endowment = $17 billion (2021)As of October 25, 2021. ...

. Holland introduced a formalized framework for predicting the quality of the next generation, known as Holland's Schema TheoremHolland's schema theorem, also called the fundamental theorem of genetic algorithms, is an inequality that results from coarse-graining an equation for evolutionary dynamics. The Schema Theorem says that short, low-order schemata with above-average ...

. Research in GAs remained largely theoretical until the mid-1980s, when The First International Conference on Genetic Algorithms was held in Pittsburgh, Pennsylvania
Pittsburgh ( ) is a city in the Commonwealth
A commonwealth is a traditional English term for a political community founded for the common good
In philosophy
Philosophy (from , ) is the study of general and fundamental questions ...

.
Commercial products

In the late 1980s, General Electric started selling the world's first genetic algorithm product, a mainframe-based toolkit designed for industrial processes. In 1989, Axcelis, Inc. released Evolver, the world's first commercial GA product for desktop computers.The New York Times
''The New York Times'' is an American daily newspaper based in New York City with a worldwide readership. Founded in 1851, the ''Times'' has since won List of Pulitzer Prizes awarded to The New York Times, 132 Pulitzer Prizes, the most of a ...

technology writer John Markoff
John Gregory Markoff (born October 24, 1949) is a journalist best known for his work at ''The New York Times'', and a book and series of articles about the 1990s pursuit and capture of Hacker (computer security), hacker Kevin Mitnick.
Biography ...

wrote about Evolver in 1990, and it remained the only interactive commercial genetic algorithm until 1995. Evolver was sold to Palisade in 1997, translated into several languages, and is currently in its 6th version. Since the 1990s, MATLAB
MATLAB (an abbreviation of "MATrix LABoratory") is a and environment developed by . MATLAB allows manipulations, plotting of and data, implementation of s, creation of s, and interfacing with programs written in other languages.
Althoug ...

has built in three derivative-free optimization heuristic algorithms (simulated annealing, particle swarm optimization, genetic algorithm) and two direct search algorithms (simplex search, pattern search).
Related techniques

Parent fields

Genetic algorithms are a sub-field: *Evolutionary algorithms
In computational intelligence (CI), an evolutionary algorithm (EA) is a subset of evolutionary computation, a generic population-based metaheuristic optimization (mathematics), optimization algorithm. An EA uses mechanisms inspired by biological ev ...

*Evolutionary computing
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of Alg ...

*Metaheuristic
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , , a ...

s
*Stochastic optimization
Stochastic optimization (SO) methods are optimization method
Method ( grc, μέθοδος, methodos) literally means a pursuit of knowledge, investigation, mode of prosecuting such inquiry, or system. In recent centuries it more often means a presc ...

*Optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. Optimization problems of sorts arise i ...

Related fields

Evolutionary algorithms

Evolutionary algorithms is a sub-field ofevolutionary computing
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of Alg ...

.
* Evolution strategies
In computer science, an evolution strategy (ES) is an optimization (mathematics), optimization technique based on ideas of evolution. It belongs to the general class of evolutionary computation or artificial evolution methodologies.
History
The 'ev ...

(ES, see Rechenberg, 1994) evolve individuals by means of mutation and intermediate or discrete recombination. ES algorithms are designed particularly to solve problems in the real-value domain. They use self-adaptation to adjust control parameters of the search. De-randomization of self-adaptation has led to the contemporary Covariance Matrix Adaptation Evolution Strategy (CMA-ESCovariance matrix adaptation evolution strategy (CMA-ES) is a particular kind of strategy for numerical optimization. evolution strategy, Evolution strategies (ES) are stochastic, Derivative-free optimization, derivative-free methods for numerical op ...

).
* Evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

(EP) involves populations of solutions with primarily mutation and selection and arbitrary representations. They use self-adaptation to adjust parameters, and can include other variation operations such as combining information from multiple parents.
* Estimation of Distribution AlgorithmImage:Eda mono-variant gauss iterations.svg, 350px, Estimation of distribution algorithm. For each iteration ''i'', a random draw is performed for a population ''P'' in a distribution ''PDu''. The distribution parameters ''PDe'' are then estimated us ...

(EDA) substitutes traditional reproduction operators by model-guided operators. Such models are learned from the population by employing machine learning techniques and represented as Probabilistic Graphical Models, from which new solutions can be sampled or generated from guided-crossover.
* Genetic programming
In artificial intelligence, genetic programming (GP) is a technique of evolving programs, starting from a population of unfit (usually random) programs, fit for a particular task by applying operations analogous to natural genetic processes to the ...

(GP) is a related technique popularized by John KozaJohn R. Koza is a computer scientist
A computer scientist is a person
A person (plural people or persons) is a being that has certain capacities or attributes such as reason, morality, consciousness or self-consciousness, and being a part of a ...

in which computer programs, rather than function parameters, are optimized. Genetic programming often uses tree-based internal data structure
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of ...

s to represent the computer programs for adaptation instead of the list
A ''list'' is any set of items. List or lists may also refer to:
People
* List (surname)List or Liste is a European surname. Notable people with the surname include:
List
* Friedrich List (1789–1846), German economist
* Garrett List (194 ...

structures typical of genetic algorithms. There are many variants of Genetic Programming, including Cartesian genetic programming, Gene expression programming
In computer programming, gene expression programming (GEP) is an evolutionary algorithm that creates computer programs or models. These computer programs are complex tree structures that learn and adapt by changing their sizes, shapes, and composi ...

, Grammatical Evolution, Linear genetic programming, Multi expression programming etc.
* Grouping genetic algorithm (GGA) is an evolution of the GA where the focus is shifted from individual items, like in classical GAs, to groups or subset of items. The idea behind this GA evolution proposed by Emanuel Falkenauer is that solving some complex problems, a.k.a. ''clustering'' or ''partitioning'' problems where a set of items must be split into disjoint group of items in an optimal way, would better be achieved by making characteristics of the groups of items equivalent to genes. These kind of problems include bin packing, line balancing, clustering with respect to a distance measure, equal piles, etc., on which classic GAs proved to perform poorly. Making genes equivalent to groups implies chromosomes that are in general of variable length, and special genetic operators that manipulate whole groups of items. For bin packing in particular, a GGA hybridized with the Dominance Criterion of Martello and Toth, is arguably the best technique to date.
* Interactive evolutionary algorithms are evolutionary algorithms that use human evaluation. They are usually applied to domains where it is hard to design a computational fitness function, for example, evolving images, music, artistic designs and forms to fit users' aesthetic preference.
Swarm intelligence

Swarm intelligence is a sub-field ofevolutionary computing
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of Alg ...

.
* Ant colony optimization
In computer science and operations research, the ant colony optimization algorithm (ACO) is a probability, probabilistic technique for solving computational problems which can be reduced to finding good paths through Graph (discrete mathematics) ...

(ACO) uses many ants (or agents) equipped with a pheromone model to traverse the solution space and find locally productive areas.
*Although considered an Estimation of distribution algorithmImage:Eda mono-variant gauss iterations.svg, 350px, Estimation of distribution algorithm. For each iteration ''i'', a random draw is performed for a population ''P'' in a distribution ''PDu''. The distribution parameters ''PDe'' are then estimated us ...

, Particle swarm optimization
In , particle swarm optimization (PSO) is a computational method that a problem by trying to improve a with regard to a given measure of quality. It solves a problem by having a population of candidate solutions, here dubbed s, and moving the ...

(PSO) is a computational method for multi-parameter optimization which also uses population-based approach. A population (swarm) of candidate solutions (particles) moves in the search space, and the movement of the particles is influenced both by their own best known position and swarm's global best known position. Like genetic algorithms, the PSO method depends on information sharing among population members. In some problems the PSO is often more computationally efficient than the GAs, especially in unconstrained problems with continuous variables.
Evolutionary algorithms combined with Swarm intelligence

* Mayfly optimization algorithm (MA) combines major advantages ofevolutionary algorithms
In computational intelligence (CI), an evolutionary algorithm (EA) is a subset of evolutionary computation, a generic population-based metaheuristic optimization (mathematics), optimization algorithm. An EA uses mechanisms inspired by biological ev ...

and swarm intelligence algorithms.
Other evolutionary computing algorithms

Evolutionary computation is a sub-field of themetaheuristic
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of , , a ...

methods.
* Memetic algorithm
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of Algor ...

(MA), often called ''hybrid genetic algorithm'' among others, is a population-based method in which solutions are also subject to local improvement phases. The idea of memetic algorithms comes from meme
A meme ( ) is an idea, behavior, or style that spreads by means of imitation from person to person within a culture and often carries symbolic meaning representing a particular phenomenon or theme. A meme acts as a unit for carrying cultural ...

s, which unlike genes, can adapt themselves. In some problem areas they are shown to be more efficient than traditional evolutionary algorithms.
* Bacteriologic algorithms (BA) inspired by evolutionary ecology
of living things
Evolutionary ecology lies at the intersection of ecology
Ecology (from el, οἶκος, "house" and el, -λογία, label=none, "study of") is the study of the relationships between living organisms, including humans, an ...

and, more particularly, bacteriologic adaptation. Evolutionary ecology is the study of living organisms in the context of their environment, with the aim of discovering how they adapt. Its basic concept is that in a heterogeneous environment, there is not one individual that fits the whole environment. So, one needs to reason at the population level. It is also believed BAs could be successfully applied to complex positioning problems (antennas for cell phones, urban planning, and so on) or data mining.
* Cultural algorithm (CA) consists of the population component almost identical to that of the genetic algorithm and, in addition, a knowledge component called the belief space.
* Differential evolution (DE) inspired by migration of superorganisms.
* Gaussian adaptation (normal or natural adaptation, abbreviated NA to avoid confusion with GA) is intended for the maximisation of manufacturing yield of signal processing systems. It may also be used for ordinary parametric optimisation. It relies on a certain theorem valid for all regions of acceptability and all Gaussian distributions. The efficiency of NA relies on information theory and a certain theorem of efficiency. Its efficiency is defined as information divided by the work needed to get the information. Because NA maximises mean fitness rather than the fitness of the individual, the landscape is smoothed such that valleys between peaks may disappear. Therefore it has a certain "ambition" to avoid local peaks in the fitness landscape. NA is also good at climbing sharp crests by adaptation of the moment matrix, because NA may maximise the disorder (average information) of the Gaussian simultaneously keeping the mean fitness constant.
Other metaheuristic methods

Metaheuristic methods broadly fall within Stochastic optimization, stochastic optimisation methods. * Simulated annealing (SA) is a related global optimization technique that traverses the search space by testing random mutations on an individual solution. A mutation that increases fitness is always accepted. A mutation that lowers fitness is accepted probabilistically based on the difference in fitness and a decreasing temperature parameter. In SA parlance, one speaks of seeking the lowest energy instead of the maximum fitness. SA can also be used within a standard GA algorithm by starting with a relatively high rate of mutation and decreasing it over time along a given schedule. * Tabu search (TS) is similar to simulated annealing in that both traverse the solution space by testing mutations of an individual solution. While simulated annealing generates only one mutated solution, tabu search generates many mutated solutions and moves to the solution with the lowest energy of those generated. In order to prevent cycling and encourage greater movement through the solution space, a tabu list is maintained of partial or complete solutions. It is forbidden to move to a solution that contains elements of the tabu list, which is updated as the solution traverses the solution space. * Extremal optimization (EO) Unlike GAs, which work with a population of candidate solutions, EO evolves a single solution and makes local search (optimization), local modifications to the worst components. This requires that a suitable representation be selected which permits individual solution components to be assigned a quality measure ("fitness"). The governing principle behind this algorithm is that of ''emergent'' improvement through selectively removing low-quality components and replacing them with a randomly selected component. This is decidedly at odds with a GA that selects good solutions in an attempt to make better solutions.Other stochastic optimisation methods

* The Cross-entropy method, cross-entropy (CE) method generates candidate solutions via a parameterized probability distribution. The parameters are updated via cross-entropy minimization, so as to generate better samples in the next iteration. * Reactive search optimization (RSO) advocates the integration of sub-symbolic machine learning techniques into search heuristics for solving complex optimization problems. The word reactive hints at a ready response to events during the search through an internal online feedback loop for the self-tuning of critical parameters. Methodologies of interest for Reactive Search include machine learning and statistics, in particular reinforcement learning, Active learning (machine learning), active or query learning, neural networks, and metaheuristics.See also

*Genetic programming
In artificial intelligence, genetic programming (GP) is a technique of evolving programs, starting from a population of unfit (usually random) programs, fit for a particular task by applying operations analogous to natural genetic processes to the ...

* List of genetic algorithm applications
* particle filter, Genetic algorithms in signal processing (a.k.a. particle filters)
* Propagation of schema
* Universal Darwinism
* Metaheuristics
* Learning classifier system
* Rule-based machine learning
References

Bibliography

* * * * * * * * * * * * * * * Rechenberg, Ingo (1994): Evolutionsstrategie '94, Stuttgart: Fromman-Holzboog. * * * * Schwefel, Hans-Paul (1974): Numerische Optimierung von Computer-Modellen (PhD thesis). Reprinted by Birkhäuser (1977). * *External links

Resources

Provides a list of resources in the genetic algorithms field

An Overview of the History and Flavors of Evolutionary Algorithms

Tutorials

An excellent introduction to GA by John Holland and with an application to the Prisoner's Dilemma * [http://www.i4ai.org/EA-demo/ An online interactive Genetic Algorithm tutorial for a reader to practise or learn how a GA works]: Learn step by step or watch global convergence in batch, change the population size, crossover rates/bounds, mutation rates/bounds and selection mechanisms, and add constraints.

A Genetic Algorithm Tutorial by Darrell Whitley Computer Science Department Colorado State University

An excellent tutorial with much theory

"Essentials of Metaheuristics"

2009 (225 p). Free open text by Sean Luke.

Global Optimization Algorithms – Theory and Application

Tutorial with the intuition behind GAs and Python implementation.

Genetic Algorithms evolves to solve the prisoner's dilemma.

Written by Robert Axelrod. {{DEFAULTSORT:Genetic Algorithm Genetic algorithms, Evolutionary algorithms Search algorithms Cybernetics Digital organisms Machine learning sv:Genetisk programmering#Genetisk algoritm