In computational intelligence (CI), an evolutionary algorithm (EA) is a mutation
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...

, recombination, and fitness function{{no footnotes, date=May 2015
A fitness function is a particular type of objective function that is used to summarise, as a single figure of merit, how close a given design solution is to achieving the set aims. Fitness functions are used in genetic ...

determines the quality of the solutions (see also

crossover
CrossOver is a Microsoft Windows
Microsoft Windows, commonly referred to as Windows, is a group of several Proprietary software, proprietary graphical user interface, graphical operating system families, all of which are developed and markete ...

and mutation
In biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...

operations to give birth to

File:Two-population EA search (2).gif, A two-population EA search over a constrained with bounded global optimum.
File:Two-population EA search (3).gif, A two-population EA search over a constrained . Global optimum is not bounded.
File:Estimation of Distribution Algorithm animation.gif,

An Overview of the History and Flavors of Evolutionary Algorithms

Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms

', Oxford Univ. Press. * Bäck, T., Fogel, D., Michalewicz, Z. (1997), ''Handbook of Evolutionary Computation'', Oxford Univ. Press. * Banzhaf, W., Nordin, P., Keller, R., Francone, F. (1998), ''Genetic Programming - An Introduction'', Morgan Kaufmann, San Francisco * Eiben, A.E., Smith, J.E. (2003), ''Introduction to Evolutionary Computing'', Springer. * Holland, J. H. (1992),

Adaptation in Natural and Artificial Systems

', The University of Michigan Press, Ann Arbor * Michalewicz Z., Fogel D.B. (2004). How To Solve It: Modern Heuristics, Springer. * * * Price, K., Storn, R.M., Lampinen, J.A., (2005)

"Differential Evolution: A Practical Approach to Global Optimization"

Springer. *

Evolutionary Optimization Algorithms

Wiley. *

Computational Intelligence: A Methodological Introduction

' by Kruse, Borgelt, Klawonn, Moewes, Steinbrecher, Held, 2013, Springer, * {{DEFAULTSORT:Evolutionary Algorithm

subset
In mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). ...

of evolutionary computation
In computer science, evolutionary computation is a family of algorithms for global optimization inspired by biological evolution, and the subfield of artificial intelligence and soft computing studying these algorithms. In technical terms, they ...

, a generic population-based metaheuristic
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of Algorit ...

optimization
File:Nelder-Mead Simionescu.gif, Nelder-Mead minimum search of Test functions for optimization, Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt=
Mathematical optimization (alter ...

algorithm
In and , an algorithm () is a finite sequence of , computer-implementable instructions, typically to solve a class of problems or to perform a computation. Algorithms are always and are used as specifications for performing s, , , and other ...

. An EA uses mechanisms inspired by biological evolution
Evolution is change in the heritable characteristics of biological population
In biology, a population is a number of all the organisms of the same group or species
In biology, a species is the basic unit of biological classific ...

, such as reproduction
Reproduction (or procreation or breeding) is the biological process
Biological processes are those processes that are vital for an organism
In biology, an organism (from Ancient Greek, Greek: ὀργανισμός, ''organismos'') is ...

, selection
Selection may refer to:
In science:
* Selection (biology)
Natural selection is the differential survival and reproduction of individuals due to differences in phenotype
right , Here the relation between genotype and phenotype is ill ...

. Candidate solution
A closed feasible region of a linear programming problem with three variables is a convex polyhedron.">polyhedron.html" ;"title="linear programming problem with three variables is a convex polyhedron">linear programming problem with three variabl ...

s to the optimization problem
In mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...

play the role of individuals in a population, and the loss functionIn mathematical optimization
File:Nelder-Mead Simionescu.gif, Nelder-Mead minimum search of Test functions for optimization, Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt=
Math ...

). Evolution
Evolution is change in the heritable
Heredity, also called inheritance or biological inheritance, is the passing on of Phenotypic trait, traits from parents to their offspring; either through asexual reproduction or sexual reproduction, ...

of the population then takes place after the repeated application of the above operators.
Evolutionary algorithms often perform well approximating solutions to all types of problems because they ideally do not make any assumption about the underlying fitness landscape
In evolutionary biology
Evolutionary biology is the subfield of biology that studies the evolution, evolutionary processes (natural selection, common descent, speciation) that produced the Biodiversity, diversity of life on Earth. In the 1930s ...

. Techniques from evolutionary algorithms applied to the modeling of biological evolution are generally limited to explorations of microevolutionary processes and planning models based upon cellular processes. In most real applications of EAs, computational complexity is a prohibiting factor. In fact, this computational complexity is due to fitness function evaluation. Fitness approximation
Fitness approximation Y. JinA comprehensive survey of fitness approximation in evolutionary computation Soft Computing, 9:3–12, 2005 aims to approximate the objective or fitness functions in evolutionary optimization by building up machine learn ...

is one of the solutions to overcome this difficulty. However, seemingly simple EA can solve often complex problems; therefore, there may be no direct link between algorithm complexity and problem complexity.
Implementation

The following is an example of a generic single-objectivegenetic algorithm
spacecraft antenna. This complicated shape was found by an evolutionary computer design program to create the best radiation pattern. It is known as an evolved antenna.
In computer science
Computer science deals with the theoretical found ...

.
Step One: Generate the initial population
Population typically refers the number of people in a single area whether it be a city or town, region, country, or the world. Governments typically quantify the size of the resident population within their jurisdiction by a process called a ...

of individual
An individual is that which exists as a distinct entity
Entity may refer to:
Computing
* Character entity reference, replacement text for a character in HTML or XML
* Entity class, a thing of interest within an entity–relationship model or d ...

s randomly. (First generation)
Step Two: Repeat the following regenerational steps until termination:
#Evaluate the fitness of each individual in the population (time limit, sufficient fitness achieved, etc.)
#Select the fittest individuals for reproduction
Reproduction (or procreation or breeding) is the biological process
Biological processes are those processes that are vital for an organism
In biology, an organism (from Ancient Greek, Greek: ὀργανισμός, ''organismos'') is ...

. (Parents)
# Breed
A breed is a specific group of domestic animals having homogeneous appearance (phenotype), homogeneous behavior, and/or other characteristics that distinguish it from other organisms of the same species. In literature, there exist several slight ...

new individuals through offspring
In biology, offspring are the young creation of living organisms, produced either by a Asexual reproduction, single organism or, in the case of sexual reproduction, two organisms. Collective offspring may be known as a brood or progeny in a more ...

.
# Replace the least-fit individuals of the population with new individuals.
Types

Similar techniques differ ingenetic representation
In computer programming, genetic representation is a way of representing solutions/individuals in evolutionary computation methods. Genetic representation can encode appearance, behavior, physical qualities of individuals. Designing a good genetic r ...

and other implementation details, and the nature of the particular applied problem.
*Genetic algorithm
spacecraft antenna. This complicated shape was found by an evolutionary computer design program to create the best radiation pattern. It is known as an evolved antenna.
In computer science
Computer science deals with the theoretical found ...

– This is the most popular type of EA. One seeks the solution of a problem in the form of strings of numbers (traditionally binary, although the best representations are usually those that reflect something about the problem being solved), by applying operators such as recombination and mutation (sometimes one, sometimes both). This type of EA is often used in optimization
File:Nelder-Mead Simionescu.gif, Nelder-Mead minimum search of Test functions for optimization, Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt=
Mathematical optimization (alter ...

problems.
*Genetic programming
In artificial intelligence, genetic programming (GP) is a technique of evolving programs, starting from a population of unfit (usually random) programs, fit for a particular task by applying operations analogous to natural genetic processes to the ...

– Here the solutions are in the form of computer programs, and their fitness is determined by their ability to solve a computational problem. There are many variants of Genetic Programming, including Cartesian genetic programming, Gene expression programming
In computer programming, gene expression programming (GEP) is an evolutionary algorithm that creates computer programs or models. These computer programs are complex tree structures that learn and adapt by changing their sizes, shapes, and composi ...

, Grammatical Evolution, Linear genetic programming, Multi expression programming etc.
*Evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms
In science
Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...

– Similar to genetic programming, but the structure of the program is fixed and its numerical parameters are allowed to evolve.
*Evolution strategy
In computer science, an evolution strategy (ES) is an optimization technique based on ideas of evolution
Evolution is change in the Heredity, heritable Phenotypic trait, characteristics of biological populations over successive generations. T ...

– Works with vectors of real numbers as representations of solutions, and typically uses self-adaptive mutation rates.
* Differential evolution – Based on vector differences and is therefore primarily suited for numerical optimization
Nelder-Mead minimum search of Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt=
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming i ...

problems.
* Neuroevolution – Similar to genetic programming but the genomes represent artificial neural networks by describing structure and connection weights. The genome encoding can be direct or indirect.
*Learning classifier system
Learning classifier systems, or LCS, are a paradigm of rule-based machine learningRule-based machine learning (RBML) is a term in computer science
Computer science deals with the theoretical foundations of information, algorithms and the archi ...

– Here the solution is a set of classifiers (rules or conditions). A Michigan-LCS evolves at the level of individual classifiers whereas a Pittsburgh-LCS uses populations of classifier-sets. Initially, classifiers were only binary, but now include real, neural net, or S-expression
In computer programming
Computer programming is the process of designing and building an executable computer program to accomplish a specific computing result or to perform a particular task. Programming involves tasks such as analysis, gener ...

types. Fitness is typically determined with either a strength or accuracy based reinforcement learning
Reinforcement learning (RL) is an area of machine learning
Machine learning (ML) is the study of computer algorithms that can improve automatically through experience and by the use of data. It is seen as a part of artificial intelligence. M ...

or supervised learning
Supervised learning (SL) is the machine learning task of learning a function that Map (mathematics), maps an input to an output based on example input-output pairs. It infers a function from ' consisting of a set of ''training examples''. In supe ...

approach.
Comparison to biological processes

A possible limitation of many evolutionary algorithms is their lack of a cleargenotype–phenotype distinction
The genotype–phenotype distinction is drawn in genetics
Genetics is a branch of biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical proce ...

. In nature, the fertilized egg cell undergoes a complex process known as embryogenesis
An embryo is the early stage of development of a multicellular organism
A multicellular organism is an organism
In biology, an organism () is any organic, life, living system that functions as an individual entity. All organisms ar ...

to become a mature phenotype
In genetics
Genetics is a branch of biology
Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular inter ...

. This indirect encoding
In communication
Communication (from Latin
Latin (, or , ) is a classical language
A classical language is a language
A language is a structured system of communication
Communication (from Latin ''communicare'', mean ...

is believed to make the genetic search more robust (i.e. reduce the probability of fatal mutations), and also may improve the evolvability
Evolvability is defined as the capacity of a system for adaptive evolution
In biology, adaptation has three related meanings. Firstly, it is the dynamic evolutionary process that fits organisms to their environment, enhancing their Fitness ( ...

of the organism. Such indirect (also known as generative or developmental) encodings also enable evolution to exploit the regularity in the environment. Recent work in the field of artificial embryogeny, or artificial developmental systems, seeks to address these concerns. And gene expression programming
In computer programming, gene expression programming (GEP) is an evolutionary algorithm that creates computer programs or models. These computer programs are complex tree structures that learn and adapt by changing their sizes, shapes, and composi ...

successfully explores a genotype–phenotype system, where the genotype consists of linear multigenic chromosomes of fixed length and the phenotype consists of multiple expression trees or computer programs of different sizes and shapes.
Related techniques

Swarm algorithms include *Ant colony optimization
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of c ...

is based on the ideas of ant foraging by pheromone communication to form paths. Primarily suited for combinatorial optimization
Combinatorial optimization is a subfield of mathematical optimization that consists of finding an optimal object from a finite set of objects, where the set of Candidate solution, feasible solutions is Discrete set, discrete or can be reduced t ...

and graph
Graph may refer to:
Mathematics
*Graph (discrete mathematics), a structure made of vertices and edges
**Graph theory, the study of such graphs and their properties
*Graph (topology), a topological space resembling a graph in the sense of discret ...

problems.
* The runner-root algorithm (RRA) is inspired by the function of runners and roots of plants in nature
* Artificial bee colony algorithm is based on the honey bee foraging behaviour. Primarily proposed for numerical optimization and extended to solve combinatorial, constrained and multi-objective optimization problems.
* Bees algorithm is based on the foraging behaviour of honey bees. It has been applied in many applications such as routing and scheduling.
* Cuckoo search is inspired by the brooding parasitism of the cuckoo
Cuckoos are bird
Birds are a group of warm-blooded vertebrate
Vertebrates () comprise all species of animal
Animals (also called Metazoa) are multicellular eukaryotic organisms that form the Kingdom (biology), biological ...

species. It also uses Lévy flights, and thus it suits for global optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. Optimization problems of sorts arise i ...

problems.
*Particle swarm optimization
In , particle swarm optimization (PSO) is a computational method that a problem by trying to improve a with regard to a given measure of quality. It solves a problem by having a population of candidate solutions, here dubbed s, and moving the ...

is based on the ideas of animal flocking behaviour. Also primarily suited for numerical optimization
Nelder-Mead minimum search of Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt=
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming i ...

problems.
Other population-based metaheuristic methods

* Hunting Search – A method inspired by the group hunting of some animals such as wolves that organize their position to surround the prey, each of them relative to the position of the others and especially that of their leader. It is a continuous optimization method adapted as a combinatorial optimization method. * Adaptive dimensional search – Unlike nature-inspired metaheuristic techniques, an adaptive dimensional search algorithm does not implement any metaphor as an underlying principle. Rather it uses a simple performance-oriented method, based on the update of the search dimensionality ratio (SDR) parameter at each iteration. * Firefly algorithm is inspired by the behavior of fireflies, attracting each other by flashing light. This is especially useful for multimodal optimization. * Harmony search – Based on the ideas of musicians' behavior in searching for better harmonies. This algorithm is suitable for combinatorial optimization as well as parameter optimization. * Gaussian adaptation – Based on information theory. Used for maximization of manufacturing yield, mean fitness or average information. See for instanceEntropy in thermodynamics and information theoryThe mathematical expressions for thermodynamic entropy in the statistical thermodynamics formulation established by Ludwig Boltzmann and J. Willard Gibbs in the 1870s are similar to the information entropy by Claude Elwood Shannon, Claude Shannon and ...

.
*Memetic algorithm
In computer science
Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application.
Computer science is the study of Algor ...

– A hybrid method, inspired by Richard Dawkins
Richard Dawkins (born 26 March 1941) is a British evolutionary biology, evolutionary biologist and author. He is an Oxford fellow, emeritus fellow of New College, Oxford and was Simonyi Professor for the Public Understanding of Science, Prof ...

's notion of a meme, it commonly takes the form of a population-based algorithm coupled with individual learning procedures capable of performing local refinements. Emphasizes the exploitation of problem-specific knowledge, and tries to orchestrate local and global search in a synergistic way.
Examples

In 2020,Google
Google LLC is an American multinational
Multinational may refer to:
* Multinational corporation, a corporate organization operating in multiple countries
* Multinational force, a military body from multiple countries
* Multinational stat ...

stated that their AutoML-Zero can successfully rediscover classic algorithms such as the concept of neural networks.
The computer simulations ''Tierra
Tierra may refer to:
Astronomy
*Earth in the Spanish and Asturian language
Computing and games
* Tierra (computer simulation), a computer simulation of life by the ecologist Thomas S. Ray
* Tierra Entertainment, now known as AGD Interactive, a ...

'' and ''Avida
Avida is an artificial life software platform to study the evolutionary biology
Evolutionary biology is the subfield of biology that studies the evolution, evolutionary processes (natural selection, common descent, speciation) that produced t ...

'' attempt to model macroevolution
Macroevolution in the modern sense is evolution that is guided by selection among interspecific variation, as opposed to selection among intraspecific variation in microevolution
Microevolution is the change in allele frequencies that occurs ...

ary dynamics.
Gallery

Estimation of distribution algorithmImage:Eda mono-variant gauss iterations.svg, 350px, Estimation of distribution algorithm. For each iteration ''i'', a random draw is performed for a population ''P'' in a distribution ''PDu''. The distribution parameters ''PDe'' are then estimated us ...

over Keane's function
File:Two population EA animation.gif, A two-population EA search of a bounded optima of Simionescu's function.
References

External links

An Overview of the History and Flavors of Evolutionary Algorithms

Bibliography

* Ashlock, D. (2006), ''Evolutionary Computation for Modeling and Optimization'', Springer, . * Bäck, T. (1996),Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms

', Oxford Univ. Press. * Bäck, T., Fogel, D., Michalewicz, Z. (1997), ''Handbook of Evolutionary Computation'', Oxford Univ. Press. * Banzhaf, W., Nordin, P., Keller, R., Francone, F. (1998), ''Genetic Programming - An Introduction'', Morgan Kaufmann, San Francisco * Eiben, A.E., Smith, J.E. (2003), ''Introduction to Evolutionary Computing'', Springer. * Holland, J. H. (1992),

Adaptation in Natural and Artificial Systems

', The University of Michigan Press, Ann Arbor * Michalewicz Z., Fogel D.B. (2004). How To Solve It: Modern Heuristics, Springer. * * * Price, K., Storn, R.M., Lampinen, J.A., (2005)

"Differential Evolution: A Practical Approach to Global Optimization"

Springer. *

Ingo Rechenberg
Ingo Rechenberg (born November 20, 1934) is a Germany, German researcher and professor currently in the field of bionics. Rechenberg is a pioneer of the fields of evolutionary computation and artificial evolution. In the 1960s and 1970s he invente ...

(1971): Evolutionsstrategie - Optimierung technischer Systeme nach Prinzipien der biologischen Evolution (PhD thesis). Reprinted by Fromman-Holzboog (1973).
* Hans-Paul Schwefel (1974): Numerische Optimierung von Computer-Modellen (PhD thesis). Reprinted by Birkhäuser (1977).
* Simon, D. (2013)Evolutionary Optimization Algorithms

Wiley. *

Computational Intelligence: A Methodological Introduction

' by Kruse, Borgelt, Klawonn, Moewes, Steinbrecher, Held, 2013, Springer, * {{DEFAULTSORT:Evolutionary Algorithm

Cybernetics
Cybernetics is a Transdisciplinarity, transdisciplinary approach for exploring regulatory systems with feedback, their structures, constraints, and possibilities. Cybernetics is relevant to the study of systems, such as mechanical, physical, biol ...

Evolution
Optimization algorithms and methods