HOME

TheInfoList




In computational intelligence (CI), an evolutionary algorithm (EA) is a
subset In mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). ...

subset
of
evolutionary computation In computer science, evolutionary computation is a family of algorithms for global optimization inspired by biological evolution, and the subfield of artificial intelligence and soft computing studying these algorithms. In technical terms, they ...
, a generic population-based
metaheuristic In computer science Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application. Computer science is the study of Algorit ...
optimization File:Nelder-Mead Simionescu.gif, Nelder-Mead minimum search of Test functions for optimization, Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt= Mathematical optimization (alter ...
algorithm In and , an algorithm () is a finite sequence of , computer-implementable instructions, typically to solve a class of problems or to perform a computation. Algorithms are always and are used as specifications for performing s, , , and other ...

algorithm
. An EA uses mechanisms inspired by
biological evolution Evolution is change in the heritable characteristics of biological population In biology, a population is a number of all the organisms of the same group or species In biology, a species is the basic unit of biological classific ...
, such as
reproduction Reproduction (or procreation or breeding) is the biological process Biological processes are those processes that are vital for an organism In biology, an organism (from Ancient Greek, Greek: ὀργανισμός, ''organismos'') is ...

reproduction
,
mutation In biology Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...
, recombination, and
selection Selection may refer to: In science: * Selection (biology) Natural selection is the differential survival and reproduction of individuals due to differences in phenotype right , Here the relation between genotype and phenotype is ill ...
.
Candidate solution A closed feasible region of a linear programming problem with three variables is a convex polyhedron.">polyhedron.html" ;"title="linear programming problem with three variables is a convex polyhedron">linear programming problem with three variabl ...
s to the
optimization problem In mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...
play the role of individuals in a population, and the
fitness function{{no footnotes, date=May 2015 A fitness function is a particular type of objective function that is used to summarise, as a single figure of merit, how close a given design solution is to achieving the set aims. Fitness functions are used in genetic ...
determines the quality of the solutions (see also
loss functionIn mathematical optimization File:Nelder-Mead Simionescu.gif, Nelder-Mead minimum search of Test functions for optimization, Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt= Math ...
).
Evolution Evolution is change in the heritable Heredity, also called inheritance or biological inheritance, is the passing on of Phenotypic trait, traits from parents to their offspring; either through asexual reproduction or sexual reproduction, ...

Evolution
of the population then takes place after the repeated application of the above operators. Evolutionary algorithms often perform well approximating solutions to all types of problems because they ideally do not make any assumption about the underlying
fitness landscape In evolutionary biology Evolutionary biology is the subfield of biology that studies the evolution, evolutionary processes (natural selection, common descent, speciation) that produced the Biodiversity, diversity of life on Earth. In the 1930s ...
. Techniques from evolutionary algorithms applied to the modeling of biological evolution are generally limited to explorations of microevolutionary processes and planning models based upon cellular processes. In most real applications of EAs, computational complexity is a prohibiting factor. In fact, this computational complexity is due to fitness function evaluation.
Fitness approximation Fitness approximation Y. JinA comprehensive survey of fitness approximation in evolutionary computation Soft Computing, 9:3–12, 2005 aims to approximate the objective or fitness functions in evolutionary optimization by building up machine learn ...
is one of the solutions to overcome this difficulty. However, seemingly simple EA can solve often complex problems; therefore, there may be no direct link between algorithm complexity and problem complexity.


Implementation

The following is an example of a generic single-objective
genetic algorithm spacecraft antenna. This complicated shape was found by an evolutionary computer design program to create the best radiation pattern. It is known as an evolved antenna. In computer science Computer science deals with the theoretical found ...

genetic algorithm
. Step One: Generate the initial
population Population typically refers the number of people in a single area whether it be a city or town, region, country, or the world. Governments typically quantify the size of the resident population within their jurisdiction by a process called a ...

population
of
individual An individual is that which exists as a distinct entity Entity may refer to: Computing * Character entity reference, replacement text for a character in HTML or XML * Entity class, a thing of interest within an entity–relationship model or d ...
s randomly. (First generation) Step Two: Repeat the following regenerational steps until termination: #Evaluate the fitness of each individual in the population (time limit, sufficient fitness achieved, etc.) #Select the fittest individuals for
reproduction Reproduction (or procreation or breeding) is the biological process Biological processes are those processes that are vital for an organism In biology, an organism (from Ancient Greek, Greek: ὀργανισμός, ''organismos'') is ...
. (Parents) #
Breed A breed is a specific group of domestic animals having homogeneous appearance (phenotype), homogeneous behavior, and/or other characteristics that distinguish it from other organisms of the same species. In literature, there exist several slight ...
new individuals through
crossover CrossOver is a Microsoft Windows Microsoft Windows, commonly referred to as Windows, is a group of several Proprietary software, proprietary graphical user interface, graphical operating system families, all of which are developed and markete ...
and
mutation In biology Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular interactions, Physiology, physiological mechan ...
operations to give birth to
offspring In biology, offspring are the young creation of living organisms, produced either by a Asexual reproduction, single organism or, in the case of sexual reproduction, two organisms. Collective offspring may be known as a brood or progeny in a more ...

offspring
. # Replace the least-fit individuals of the population with new individuals.


Types

Similar techniques differ in
genetic representation In computer programming, genetic representation is a way of representing solutions/individuals in evolutionary computation methods. Genetic representation can encode appearance, behavior, physical qualities of individuals. Designing a good genetic r ...
and other implementation details, and the nature of the particular applied problem. *
Genetic algorithm spacecraft antenna. This complicated shape was found by an evolutionary computer design program to create the best radiation pattern. It is known as an evolved antenna. In computer science Computer science deals with the theoretical found ...

Genetic algorithm
– This is the most popular type of EA. One seeks the solution of a problem in the form of strings of numbers (traditionally binary, although the best representations are usually those that reflect something about the problem being solved), by applying operators such as recombination and mutation (sometimes one, sometimes both). This type of EA is often used in
optimization File:Nelder-Mead Simionescu.gif, Nelder-Mead minimum search of Test functions for optimization, Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt= Mathematical optimization (alter ...
problems. *
Genetic programming In artificial intelligence, genetic programming (GP) is a technique of evolving programs, starting from a population of unfit (usually random) programs, fit for a particular task by applying operations analogous to natural genetic processes to the ...
– Here the solutions are in the form of computer programs, and their fitness is determined by their ability to solve a computational problem. There are many variants of Genetic Programming, including Cartesian genetic programming,
Gene expression programming In computer programming, gene expression programming (GEP) is an evolutionary algorithm that creates computer programs or models. These computer programs are complex tree structures that learn and adapt by changing their sizes, shapes, and composi ...
, Grammatical Evolution, Linear genetic programming, Multi expression programming etc. *
Evolutionary programmingEvolutionary programming is one of the four major evolutionary algorithm paradigms In science Science (from the Latin word ''scientia'', meaning "knowledge") is a systematic enterprise that Scientific method, builds and Taxonomy (general), o ...
– Similar to genetic programming, but the structure of the program is fixed and its numerical parameters are allowed to evolve. *
Evolution strategy In computer science, an evolution strategy (ES) is an optimization technique based on ideas of evolution Evolution is change in the Heredity, heritable Phenotypic trait, characteristics of biological populations over successive generations. T ...
– Works with vectors of real numbers as representations of solutions, and typically uses self-adaptive mutation rates. * Differential evolution – Based on vector differences and is therefore primarily suited for
numerical optimization Nelder-Mead minimum search of Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt= Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming i ...
problems. * Neuroevolution – Similar to genetic programming but the genomes represent artificial neural networks by describing structure and connection weights. The genome encoding can be direct or indirect. *
Learning classifier system Learning classifier systems, or LCS, are a paradigm of rule-based machine learningRule-based machine learning (RBML) is a term in computer science Computer science deals with the theoretical foundations of information, algorithms and the archi ...
– Here the solution is a set of classifiers (rules or conditions). A Michigan-LCS evolves at the level of individual classifiers whereas a Pittsburgh-LCS uses populations of classifier-sets. Initially, classifiers were only binary, but now include real, neural net, or
S-expression In computer programming Computer programming is the process of designing and building an executable computer program to accomplish a specific computing result or to perform a particular task. Programming involves tasks such as analysis, gener ...
types. Fitness is typically determined with either a strength or accuracy based
reinforcement learning Reinforcement learning (RL) is an area of machine learning Machine learning (ML) is the study of computer algorithms that can improve automatically through experience and by the use of data. It is seen as a part of artificial intelligence. M ...
or
supervised learning Supervised learning (SL) is the machine learning task of learning a function that Map (mathematics), maps an input to an output based on example input-output pairs. It infers a function from ' consisting of a set of ''training examples''. In supe ...
approach.


Comparison to biological processes

A possible limitation of many evolutionary algorithms is their lack of a clear
genotype–phenotype distinction The genotype–phenotype distinction is drawn in genetics Genetics is a branch of biology Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical proce ...
. In nature, the fertilized egg cell undergoes a complex process known as
embryogenesis An embryo is the early stage of development of a multicellular organism A multicellular organism is an organism In biology, an organism () is any organic, life, living system that functions as an individual entity. All organisms ar ...

embryogenesis
to become a mature
phenotype In genetics Genetics is a branch of biology Biology is the natural science that studies life and living organisms, including their anatomy, physical structure, Biochemistry, chemical processes, Molecular biology, molecular inter ...

phenotype
. This indirect
encoding In communication Communication (from Latin Latin (, or , ) is a classical language A classical language is a language A language is a structured system of communication Communication (from Latin ''communicare'', mean ...

encoding
is believed to make the genetic search more robust (i.e. reduce the probability of fatal mutations), and also may improve the
evolvability Evolvability is defined as the capacity of a system for adaptive evolution In biology, adaptation has three related meanings. Firstly, it is the dynamic evolutionary process that fits organisms to their environment, enhancing their Fitness ( ...
of the organism. Such indirect (also known as generative or developmental) encodings also enable evolution to exploit the regularity in the environment. Recent work in the field of artificial embryogeny, or artificial developmental systems, seeks to address these concerns. And
gene expression programming In computer programming, gene expression programming (GEP) is an evolutionary algorithm that creates computer programs or models. These computer programs are complex tree structures that learn and adapt by changing their sizes, shapes, and composi ...
successfully explores a genotype–phenotype system, where the genotype consists of linear multigenic chromosomes of fixed length and the phenotype consists of multiple expression trees or computer programs of different sizes and shapes.


Related techniques

Swarm algorithms include *
Ant colony optimization In computer science Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application. Computer science is the study of c ...
is based on the ideas of ant foraging by pheromone communication to form paths. Primarily suited for
combinatorial optimization Combinatorial optimization is a subfield of mathematical optimization that consists of finding an optimal object from a finite set of objects, where the set of Candidate solution, feasible solutions is Discrete set, discrete or can be reduced t ...
and
graph Graph may refer to: Mathematics *Graph (discrete mathematics), a structure made of vertices and edges **Graph theory, the study of such graphs and their properties *Graph (topology), a topological space resembling a graph in the sense of discret ...
problems. * The runner-root algorithm (RRA) is inspired by the function of runners and roots of plants in nature * Artificial bee colony algorithm is based on the honey bee foraging behaviour. Primarily proposed for numerical optimization and extended to solve combinatorial, constrained and multi-objective optimization problems. * Bees algorithm is based on the foraging behaviour of honey bees. It has been applied in many applications such as routing and scheduling. * Cuckoo search is inspired by the brooding parasitism of the
cuckoo Cuckoos are bird Birds are a group of warm-blooded vertebrate Vertebrates () comprise all species of animal Animals (also called Metazoa) are multicellular eukaryotic organisms that form the Kingdom (biology), biological ...

cuckoo
species. It also uses Lévy flights, and thus it suits for global
optimization Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. Optimization problems of sorts arise i ...

optimization
problems. *
Particle swarm optimization In , particle swarm optimization (PSO) is a computational method that a problem by trying to improve a with regard to a given measure of quality. It solves a problem by having a population of candidate solutions, here dubbed s, and moving the ...
is based on the ideas of animal flocking behaviour. Also primarily suited for
numerical optimization Nelder-Mead minimum search of Simionescu's function. Simplex vertices are ordered by their values, with 1 having the lowest ( best) value., alt= Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming i ...
problems.


Other population-based metaheuristic methods

* Hunting Search – A method inspired by the group hunting of some animals such as wolves that organize their position to surround the prey, each of them relative to the position of the others and especially that of their leader. It is a continuous optimization method adapted as a combinatorial optimization method. * Adaptive dimensional search – Unlike nature-inspired metaheuristic techniques, an adaptive dimensional search algorithm does not implement any metaphor as an underlying principle. Rather it uses a simple performance-oriented method, based on the update of the search dimensionality ratio (SDR) parameter at each iteration. * Firefly algorithm is inspired by the behavior of fireflies, attracting each other by flashing light. This is especially useful for multimodal optimization. * Harmony search – Based on the ideas of musicians' behavior in searching for better harmonies. This algorithm is suitable for combinatorial optimization as well as parameter optimization. * Gaussian adaptation – Based on information theory. Used for maximization of manufacturing yield, mean fitness or average information. See for instance
Entropy in thermodynamics and information theoryThe mathematical expressions for thermodynamic entropy in the statistical thermodynamics formulation established by Ludwig Boltzmann and J. Willard Gibbs in the 1870s are similar to the information entropy by Claude Elwood Shannon, Claude Shannon and ...
. *
Memetic algorithm In computer science Computer science deals with the theoretical foundations of information, algorithms and the architectures of its computation as well as practical techniques for their application. Computer science is the study of Algor ...
– A hybrid method, inspired by
Richard Dawkins Richard Dawkins (born 26 March 1941) is a British evolutionary biology, evolutionary biologist and author. He is an Oxford fellow, emeritus fellow of New College, Oxford and was Simonyi Professor for the Public Understanding of Science, Prof ...

Richard Dawkins
's notion of a meme, it commonly takes the form of a population-based algorithm coupled with individual learning procedures capable of performing local refinements. Emphasizes the exploitation of problem-specific knowledge, and tries to orchestrate local and global search in a synergistic way.


Examples

In 2020,
Google Google LLC is an American multinational Multinational may refer to: * Multinational corporation, a corporate organization operating in multiple countries * Multinational force, a military body from multiple countries * Multinational stat ...

Google
stated that their AutoML-Zero can successfully rediscover classic algorithms such as the concept of neural networks. The computer simulations ''
Tierra Tierra may refer to: Astronomy *Earth in the Spanish and Asturian language Computing and games * Tierra (computer simulation), a computer simulation of life by the ecologist Thomas S. Ray * Tierra Entertainment, now known as AGD Interactive, a ...
'' and ''
Avida Avida is an artificial life software platform to study the evolutionary biology Evolutionary biology is the subfield of biology that studies the evolution, evolutionary processes (natural selection, common descent, speciation) that produced t ...
'' attempt to model
macroevolution Macroevolution in the modern sense is evolution that is guided by selection among interspecific variation, as opposed to selection among intraspecific variation in microevolution Microevolution is the change in allele frequencies that occurs ...
ary dynamics.


Gallery

File:Two-population EA search (2).gif, A two-population EA search over a constrained with bounded global optimum. File:Two-population EA search (3).gif, A two-population EA search over a constrained . Global optimum is not bounded. File:Estimation of Distribution Algorithm animation.gif,
Estimation of distribution algorithmImage:Eda mono-variant gauss iterations.svg, 350px, Estimation of distribution algorithm. For each iteration ''i'', a random draw is performed for a population ''P'' in a distribution ''PDu''. The distribution parameters ''PDe'' are then estimated us ...
over Keane's function File:Two population EA animation.gif, A two-population EA search of a bounded optima of Simionescu's function.


References


External links


An Overview of the History and Flavors of Evolutionary Algorithms


Bibliography

* Ashlock, D. (2006), ''Evolutionary Computation for Modeling and Optimization'', Springer, . * Bäck, T. (1996),
Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms
', Oxford Univ. Press. * Bäck, T., Fogel, D., Michalewicz, Z. (1997), ''Handbook of Evolutionary Computation'', Oxford Univ. Press. * Banzhaf, W., Nordin, P., Keller, R., Francone, F. (1998), ''Genetic Programming - An Introduction'', Morgan Kaufmann, San Francisco * Eiben, A.E., Smith, J.E. (2003), ''Introduction to Evolutionary Computing'', Springer. * Holland, J. H. (1992),
Adaptation in Natural and Artificial Systems
', The University of Michigan Press, Ann Arbor * Michalewicz Z., Fogel D.B. (2004). How To Solve It: Modern Heuristics, Springer. * * * Price, K., Storn, R.M., Lampinen, J.A., (2005)
"Differential Evolution: A Practical Approach to Global Optimization"
Springer. *
Ingo Rechenberg Ingo Rechenberg (born November 20, 1934) is a Germany, German researcher and professor currently in the field of bionics. Rechenberg is a pioneer of the fields of evolutionary computation and artificial evolution. In the 1960s and 1970s he invente ...
(1971): Evolutionsstrategie - Optimierung technischer Systeme nach Prinzipien der biologischen Evolution (PhD thesis). Reprinted by Fromman-Holzboog (1973). * Hans-Paul Schwefel (1974): Numerische Optimierung von Computer-Modellen (PhD thesis). Reprinted by Birkhäuser (1977). * Simon, D. (2013)
Evolutionary Optimization Algorithms
Wiley. *
Computational Intelligence: A Methodological Introduction
' by Kruse, Borgelt, Klawonn, Moewes, Steinbrecher, Held, 2013, Springer, * {{DEFAULTSORT:Evolutionary Algorithm
Cybernetics Cybernetics is a Transdisciplinarity, transdisciplinary approach for exploring regulatory systems with feedback, their structures, constraints, and possibilities. Cybernetics is relevant to the study of systems, such as mechanical, physical, biol ...
Evolution Optimization algorithms and methods