Evolving Intelligent System

In computer science, an evolving intelligent system is a fuzzy logic system which improves the own performance by evolving rules. The technique is known from machine learning, in which external patterns are learned by an algorithm. Fuzzy logic based machine learning works with neuro-fuzzy systems.

Intelligent systems have to be able to evolve, self-develop, and self-learn continuously in order to reflect a dynamically evolving environment. The concept of Evolving Intelligent Systems (EISs) was conceived around the turn of the century with the phrase EIS itself coined for the first time by Angelov and Kasabov in a 2006 IEEE newsletter and expanded in a 2010 text. EISs develop their structure, functionality and internal knowledge representation through autonomous learning from data streams generated by the possibly unknown environment and from the system self-monitoring. EISs consider a gradual development of the underlying (fuzzy or neuro-fuzzy) system structure and differ from evolutionary and genetic algorithms which consider such phenomena as chromosomes crossover, mutation, selection and reproduction, parents and off-springs. The evolutionary fuzzy and neuro systems are sometimes also called "evolving" which leads to some confusion. This was more typical for the first works on this topic in the late 1990s.

Implementations

EISs can be implemented, for example, using neural networks or fuzzy rule-based models. The first neural networks which consider an evolving structure were published in. These were later expanded by N. Kasabov and P. Angelov for the neuro-fuzzy models. P. Angelov introduced the evolving fuzzy rule-based systems (EFSs) as the first mathematical self-learning model that can dynamically evolve its internal structure and is human interpretable and coined the phrase EFS. Contemporarily, the offline incremental approach for learning an EIS, namely, EFuNN, was proposed by N. Kasabov. P. Angelov, D. Filev, N. Kasabov and O. Cordon organised the first IEEE Symposium on EFSs in 2006 (the proceedings of the conference can be found in). EFSs include a formal (and mathematically sound) learning mechanism to extract it from streaming data. One of the earliest and the most widely cited comprehensive survey on EFSs was done in 2008. Later comprehensive surveys on EFS methods with real applications were done in 2011 and 2016 by E. Lughofer.

Other works that contributed further to this area in the following years expanded it to evolving participatory learning, evolving grammar, evolving decision trees, evolving human behaviour modelling, self-calibrating (evolving) sensors (eSensors), evolving fuzzy rule-based classifiers, evolving fuzzy controllers, autonomous fault detectors. More recently, the stability of the evolving fuzzy rule-based systems that consist of the structure learning and the fuzzily weighted recursive least square parameter update method has been proven by Rong. Generalized EFS, which allow rules to be arbitrarily rotated in the feature space and thus to improve their data representability, have been proposed in with significant extensions in towards 'smartness' of the rule bases (thus, termed as "Generalized Smart EFS"), allowing more interpretability and reducing curse of dimensionality. The generalized rule structure was also successfully used in the context of evolving neuro-fuzzy systems. Several facets and challenges for achieving more transparent and understandable rule bases in EFS have been discussed by E. Lughofer in.

EISs form the theoretical and methodological basis for the Autonomous Learning Machines (ALMA) and autonomous multi-model systems (ALMMo) as well as of the Autonomous Learning Systems. Evolving Fuzzy Rule-based classifiers, in particular, is a very powerful new concept that offers much more than simply incremental or online classifiers – it can cope with new classes being added or existing classes being merged. This is much more than just adapting to new data samples being added or classification surfaces being evolved. Fuzzy rule-based classifiers are the methodological basis of a new approach to deep learning that was until now considered as a form of multi-layered neural networks. Deep Learning offers high precision levels surpassing the level of human ability and grabbed the imagination of the researchers, industry and the wider public. However, it has a number of intrinsic constraints and limitations. These include:
# The "black box", opaque internal structure which has millions of parameters and involves ad hoc decisions on the number of layers and algorithm parameters.
# The requirement for a huge amount of training data samples, computational resources (usually requiring GPUs and/or HPC) and time (usually requiring many hours of training).
# Iterative search.
# Requires retraining for new situations (is not evolving).
# Does not have proven convergence and stability.
Most, if not all, of the above limitations can be avoided with the use of the Deep (Fuzzy) Rule-based Classifiers, which were recently introduced based on ALMMo, while achieving similar or even better performance. The resulting prototype-based IF...THEN...models are fully interpretable and dynamically evolving (they can adapt quickly and automatically to new data patterns or even new classes). They are non-parametric and, therefore, their training is non-iterative and fast (it can take few milliseconds per data sample/image on a normal laptop which contrasts with the multiple hours the current deep learning methods require for training even when they use GPUs and HPC). Moreover, they can be trained incrementally, online, or in real-time. Another aspect of Evolving Fuzzy Rule-based classifiers has been proposed in, which, in case of multi-class classification problems, achieves the reduction of class imbalance by cascadability into class sub-spaces and an increased flexibility and performance for adding new classes on the fly from streaming samples.

References

{{Reflist, refs=

P. Angelov and R. Buswell,
Evolving Rule-based Models: A Tool for Intelligent Adaption
" in IFSA world congress and 20th NAFIPS international conference, 2001, pp. 1062–1067.
P. P. Angelo
Evolving rule-based models: a tool for design of flexible adaptive systems
Springer Berlin Heidelberg, 2002.
N. K. Kasabov and Q. Song,
DENFIS : Dynamic Evolving Neural-Fuzzy Inference System and Its Application for Time-Series Prediction
" IEEE Trans. Fuzzy Syst., vol. 10, no. 2, pp. 144–154, 2002.
P. Angelov, D. Filev,
On-line Design of Takagi-Sugeno Models
" In: Bilgiç T., De Baets B., Kaynak O. (eds) Fuzzy Sets and Systems — IFSA 2003. IFSA 2003. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence), vol 2715. Springer, Berlin, Heidelberg
P. P. Angelov and D. P. Filev,
An approach to online identification of Takagi-Sugeno fuzzy models
" IEEE Trans. Syst. Man, Cybern. - Part B Cybern., vol. 34, no. 1, pp. 484–498, 2004.
P. Angelov, N. Kasabov
eIS: Evolving Intelligent Systems
2006, In: IEEE SMC eNewsLetter, Vol. 15, 2006, p. 1-13.
P. Angelov, D. Filev, N. Kasabov

Wiley-IEEE Press, 2010.
P. Angelov

John Wiley & Sons, Ltd., 2012.
B. Carse, T. C. Fogarty and A. Munro,
Evolving fuzzy rule based controllers using genetic algorithms
" Fuzzy sets and systems, 80(3), pp. 273-293, 1996.
N. Kasabov,
Evolving Fuzzy Neural Networks—Algorithms, Applications and Biological Motivation
, in Methodologies for the conception, design and application of soft computing, World Scientific, pp. 271– 274, 1998.
P. P. Angelov ,
Evolving Fuzzy Rule-Based Models
, Proc. 8th IFSA World Congress, Taiwan, vol.1, pp.19-23, 1999.
P. P. Angelov, and D. P. Filev,
Flexible models with evolving structure
" in: Proc. First International IEEE Symposium "Intelligent Systems", v. II, pp.28-33, IEEE Press, {{ISBN, 0-7803-7134-8/02, 2002.
T. Martinetz and K. Schulten. "A "neural gas" network learns topologies" Artificial Neural Networks. Elsevier. pp. 397–402, 1991.
B. Fritzke,
A growing neural gas network learns topologies
" Advances in neural information processing systems. 1995.
C. F. Juang and C. T. Lin,
An online self-constructing neural fuzzy inference network and its applications
" in IEEE transactions on Fuzzy Systems, vol.6 no. 1, pp.12-32, 1998.
S. Wu and M. J. Er,
Dynamic fuzzy neural networks-a novel approach to function approximation
. in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 30, no. 2, pp. 358-364, 2000.
S. Wu, M. J. Er and Y. Gao,
A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks
. in IEEE Transactions on Fuzzy Systems, vol. 9, no. 4, pp. 578-594, 2001.
Proceedings of 2006 International Symposium on Evolving Fuzzy Systems
{{ISBN, 0-7803-9718-5, IEEE Catalog number 06EX1440.
N. Kasabov,
Evolving fuzzy neural networks-algorithms, applications and biological motivation
" in Methodologies for the conception, design and application of soft computing, World Scientific, pp.271-274, 1998.
N. Kasabov,
Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning
" in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 31, no.6, pp. 902-918, 2001.
P. P. Angelov,
Evolving fuzzy systems
" http://www.scholarpedia.org/article/Evolving_fuzzy_systems, 2008.
E. Lughofer
Evolving Fuzzy Systems: Methodologies, Advanced Concepts and Applications
Studies in Fuzzy and Soft Computing, Springer, 2011.
E. Lughofer
Evolving Fuzzy Systems --- Fundamentals, Reliability, Interpretability and Useability
Handbook of Computational Intelligence, editor. P. P. Angelov, World Scientific, pp. 67-135, 2016.
E. Lima, F. Gomide and R. Ballini,
Participatory Evolving Fuzzy Modeling
" in IEEE International Symposium on Evolving Fuzzy Systems, 2006, pp. 36-41.
N. M. Sharef, T. Martin, and Y. Shen,
Order independent incremental evolving fuzzy grammar fragment learner
" in IEEE International Conference on Intelligent Systems Design and Applications, 2009, pp. 1221–1226.
A. Shaker, R. Senge, and E. Hüllermeier,
Evolving fuzzy pattern trees for binary classification on data streams
" Inf. Sci. (Ny)., vol. 220, pp. 34–45, 2013.
J. A. Iglesias, P. Angelov, A. Ledezma and A. Sanchis,
Creating Evolving User Behavior Profiles Automatically
" in IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 5, pp. 854-867, 2012.
P. Angelov and A. Kordon,
Adaptive Inferential Sensors Based on Evolving Fuzzy Models
" in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 40, no. 2, pp. 529-539, 2010.
C. Xydeas, P. Angelov, S. Chiao, and M. Reoullas,
Advances in classification of EEG signals via evolving fuzzy classifiers and dependant multiple HMMs
" in Computers in Biology and Medicine, vol. 3, no.10, pp. 1064-1083, 2006.
P. Angelov, X. Zhou and F. Klawonn,
Evolving Fuzzy Rule-based Classifiers
" in IEEE Symposium on Computational Intelligence in Image and Signal Processing, 2007, pp. 220-225.
P. Angelov, E. Lughofer and X. Zhou
Evolving fuzzy classifiers using different model architectures
" in Fuzzy Sets and Systems, vol. 159, no. 23, pp. 3160-3182, 2008.
P. P. Angelov and X. Zhou,
Evolving Fuzzy-Rule-Based Classifiers From Data Streams
" in IEEE Transactions on Fuzzy Systems, vol. 16, no. 6, pp. 1462-1475, 2008.
P. Angelov, X. Zhou, D. Filev and E. Lughofer,
Architectures for evolving fuzzy rule-based classifiers
" in IEEE International Conference on Systems, Man and Cybernetics, 2007, pp. 2050-2055.
D. Dovžan, V. Logar and I. Škrjanc,
Implementation of an Evolving Fuzzy Model (eFuMo) in a Monitoring System for a Waste-Water Treatment Process
" in IEEE Transactions on Fuzzy Systems, vol. 23, no. 5, pp. 1761-1776, 2015.
P. Angelov, I. Škrjanc and S. Blažič,
Robust evolving cloud-based controller for a hydraulic plant
" in IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), Singapore, 2013, pp. 1-8.
B.S.J. Costa, P.P. Angelov and L.A. Guedes,
Real-Time Fault Detection Using Recursive Density Estimation
J Control Autom Electr Syst, vol. 25, no. 4, pp. 428–437, 2014.
H. Rong, P. Angelov, X. Gu and J. Bai,
Stability of Evolving Fuzzy Systems based on Data Clouds, IEEE Transactions on Fuzzy Systems
DOI: 10.1109/TFUZZ.2018.2793258, 2018.
A. Lemos, W. Caminhas and F. Gomide,
Multivariable Gaussian Evolving Fuzzy Modeling System, IEEE Transactions on Fuzzy Systems
, vol. 19 (1), pp. 91-104, 2011.
E. Lughofer, C. Cernuda, S. Kindermann and M. Pratama,
Generalized Smart Evolving Fuzzy Systems, Evolving Systems
, vol. 6 (4), pp. 269-292, 2015.
E. Lughofer,
On-line Assurance of Interpretability Criteria in Evolving Fuzzy Systems --- Achievements, New Concepts and Open Issues
Information Sciences, vol. 251, pp. 22-46, 2013.
P.P. Angelov, "Autonomous Machine Learning (ALMA): generating rules from data streams" in Special International Conference on Complex Systems, 2011, pp. 249-256.
P.P. Angelov, X Gu, J Príncipe,
Autonomous learning multi-model systems from data streams
, in IEEE Transactions on Fuzzy Systems, DOI:10.1109/TFUZZ.2017.2769039, 2017.
I. Goodfellow, Y. Bengio, and A. Courville
Deep learning
Crambridge, MA: MIT Press, 2016.
P. Angelov, X. Gu,
MICE: Multi-layer multi-model images classifier ensemble
" in IEEE International Conference on Cybernetics (CYBCONF), 2017, pp. 1-8.
P. Angelov, X. Gu,
A cascade of deep learning fuzzy rule-based image classifier and SVM
" in IEEE International Conference on Systems, Man, and Cybernetics (SMC2017), 2017, pp.746-751
E. Lughofer, O. Buchtala,
Reliable All-Pairs Evolving Fuzzy Classifiers
" in IEEE Transactions on Fuzzy Systems, vol. 21 (4), pp. 625-641, 2013.
E. Lughofer and E. Weigl and W. Heidl and C. Eitzinger and T. Radauer,
Integrating New Classes On the Fly in Evolving Fuzzy Classifier Designs and its Application in Visual Inspection
" in Applied Soft Computing, vol. 35, pp. 558-582, 2015

Fuzzy logic