Infomax is an optimization principle for
artificial neural networks
Artificial neural networks (ANNs), usually simply called neural networks (NNs) or neural nets, are computing systems inspired by the biological neural networks that constitute animal brains.
An ANN is based on a collection of connected unit ...
and other information processing systems. It prescribes that a function that maps a set of input values ''I'' to a set of output values ''O'' should be chosen or learned so as to maximize the average
Shannon mutual information between ''I'' and ''O'', subject to a set of specified constraints and/or noise processes. Infomax algorithms are learning
algorithms that perform this optimization process. The principle was described by Linsker in 1988.
Infomax, in its zero-noise limit, is related to the principle of redundancy reduction proposed for biological sensory processing by
Horace Barlow in 1961, and applied quantitatively to retinal processing by Atick and Redlich.
One of the applications of infomax has been to an
independent component analysis algorithm that finds independent signals by maximizing
entropy. Infomax-based ICA was described by Bell and
Sejnowski, and Nadal and Parga in 1995.
See also
*
FastICA
FastICA is an efficient and popular algorithm for independent component analysis invented by Aapo Hyvärinen at Helsinki University of Technology. Like most ICA algorithms, FastICA seeks an orthogonal rotation of prewhitened data, through a fixed- ...
References
*
*
*
Artificial neural networks
Computational neuroscience
{{mathapplied-stub