HOME

TheInfoList



OR:

Connectionism refers to both an approach in the field of cognitive science that hopes to explain mental phenomena using
artificial neural networks Artificial neural networks (ANNs), usually simply called neural networks (NNs) or neural nets, are computing systems inspired by the biological neural networks that constitute animal brains. An ANN is based on a collection of connected uni ...
(ANN) and to a wide range of techniques and algorithms using ANNs in the context of
artificial intelligence Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech re ...
to build more intelligent machines. Connectionism presents a cognitive theory based on simultaneously occurring, distributed signal activity via connections that can be represented numerically, where learning occurs by modifying connection strengths based on experience. Some advantages of the connectionist approach include its applicability to a broad array of functions, structural approximation to biological neurons, low requirements for innate structure, and capacity for
graceful degradation Fault tolerance is the property that enables a system to continue operating properly in the event of the failure of one or more faults within some of its components. If its operating quality decreases at all, the decrease is proportional to the ...
. Some disadvantages include the difficulty in deciphering how ANNs process information, or account for the compositionality of mental representations, and a resultant difficulty explaining phenomena at a higher level. The success of
deep learning Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised. D ...
networks in the past decade has greatly increased the popularity of this approach, but the complexity and scale of such networks has brought with them increased interpretability problems. Connectionism is seen by many to offer an alternative to classical theories of mind based on symbolic computation, but the extent to which the two approaches are compatible has been the subject of much debate since their inception.


Basic principles

The central connectionist principle is that mental phenomena can be described by interconnected networks of simple and often uniform units. The form of the connections and the units can vary from model to model. For example, units in the network could represent
neurons A neuron, neurone, or nerve cell is an electrically excitable cell that communicates with other cells via specialized connections called synapses. The neuron is the main component of nervous tissue in all animals except sponges and placozoa. N ...
and the connections could represent
synapses In the nervous system, a synapse is a structure that permits a neuron (or nerve cell) to pass an electrical or chemical signal to another neuron or to the target effector cell. Synapses are essential to the transmission of nervous impulses from ...
, as in the
human brain The human brain is the central organ of the human nervous system, and with the spinal cord makes up the central nervous system. The brain consists of the cerebrum, the brainstem and the cerebellum. It controls most of the activities of the ...
.


Spreading activation

In most connectionist models, networks change over time. A closely related and very common aspect of connectionist models is ''activation''. At any time, a unit in the network has an activation, which is a numerical value intended to represent some aspect of the unit. For example, if the units in the model are neurons, the activation could represent the
probability Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speakin ...
that the neuron would generate an
action potential An action potential occurs when the membrane potential of a specific cell location rapidly rises and falls. This depolarization then causes adjacent locations to similarly depolarize. Action potentials occur in several types of animal cells, ...
spike. Activation typically spreads to all the other units connected to it. Spreading activation is always a feature of neural network models, and it is very common in connectionist models used by
cognitive psychologists Cognition refers to "the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses". It encompasses all aspects of intellectual functions and processes such as: perception, attention, thought ...
.


Neural networks

Neural networks are by far the most commonly used connectionist model today. Though there are a large variety of neural network models, they almost always follow two basic principles regarding the mind: # Any mental state can be described as an (N)-dimensional
vector Vector most often refers to: *Euclidean vector, a quantity with a magnitude and a direction *Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism Vector may also refer to: Mathematic ...
of numeric activation values over neural units in a network. # Memory is created by modifying the strength of the connections between neural units. The connection strengths, or "weights", are generally represented as an N×M
matrix Matrix most commonly refers to: * ''The Matrix'' (franchise), an American media franchise ** ''The Matrix'', a 1999 science-fiction action film ** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
. Most of the variety among neural network models comes from: * ''Interpretation of units'': Units can be interpreted as neurons or groups of neurons. * ''Definition of activation'': Activation can be defined in a variety of ways. For example, in a Boltzmann machine, the activation is interpreted as the probability of generating an action potential spike, and is determined via a
logistic function A logistic function or logistic curve is a common S-shaped curve ( sigmoid curve) with equation f(x) = \frac, where For values of x in the domain of real numbers from -\infty to +\infty, the S-curve shown on the right is obtained, with th ...
on the sum of the inputs to a unit. * ''Learning algorithm'': Different networks modify their connections differently. In general, any mathematically defined change in connection weights over time is referred to as the "learning algorithm". Connectionists are in agreement that
recurrent neural networks A recurrent neural network (RNN) is a class of artificial neural networks where connections between nodes can create a cycle, allowing output from some nodes to affect subsequent input to the same nodes. This allows it to exhibit temporal dynamic ...
(directed networks wherein connections of the network can form a directed cycle) are a better model of the brain than feedforward neural networks (directed networks with no cycles, called DAG). Many recurrent connectionist models also incorporate
dynamical systems theory Dynamical systems theory is an area of mathematics used to describe the behavior of complex dynamical systems, usually by employing differential equations or difference equations. When differential equations are employed, the theory is called ' ...
. Many researchers, such as the connectionist
Paul Smolensky Paul Smolensky (born May 5, 1955) is Krieger-Eisenhower Professor of Cognitive Science at the Johns Hopkins University and a Senior Principal Researcher at Microsoft Research, Redmond Washington. Along with Alan Prince, in 1993 he developed Opt ...
, have argued that connectionist models will evolve toward fully
continuous Continuity or continuous may refer to: Mathematics * Continuity (mathematics), the opposing concept to discreteness; common examples include ** Continuous probability distribution or random variable in probability and statistics ** Continuous g ...
, high-dimensional,
non-linear In mathematics and science, a nonlinear system is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathematicians, and many other ...
,
dynamic systems In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water in ...
approaches.


Biological realism

Connectionist work in general does not need to be biologically realistic and therefore suffers from a lack of neuroscientific plausibility. However, the structure of neural networks is derived from that of biological
neuron A neuron, neurone, or nerve cell is an electrically excitable cell that communicates with other cells via specialized connections called synapses. The neuron is the main component of nervous tissue in all animals except sponges and placozoa. ...
s, and this parallel in low-level structure is often argued to be an advantage of connectionism in modeling cognitive structures compared with other approaches. One area where connectionist models are thought to be biologically implausible is with respect to error-propagation networks that are needed to support learning, but error propagation can explain some of the biologically-generated electrical activity seen at the scalp in
event-related potential An event-related potential (ERP) is the measured brain response that is the direct result of a specific sensory, cognitive, or motor event. More formally, it is any stereotyped electrophysiological response to a stimulus. The study of the brain ...
s such as the N400 and P600, and this provides some biological support for one of the key assumptions of connectionist learning procedures.


Learning

The weights in a neural network are adjusted according to some
learning rule An artificial neural network's learning rule or learning process is a method, mathematical logic or algorithm which improves the network's performance and/or training time. Usually, this rule is applied repeatedly over the network. It is done by ...
or algorithm, such as
Hebbian learning Hebbian theory is a neuroscientific theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptatio ...
. Thus, connectionists have created many sophisticated learning procedures for neural networks. Learning always involves modifying the connection weights. In general, these involve mathematical formulas to determine the change in weights when given sets of data consisting of activation vectors for some subset of the neural units. Several studies have been focused on designing teaching-learning methods based on connectionism. By formalizing learning in such a way, connectionists have many tools. A very common strategy in connectionist learning methods is to incorporate
gradient descent In mathematics, gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of th ...
over an error surface in a space defined by the weight matrix. All gradient descent learning in connectionist models involves changing each weight by the
partial derivative In mathematics, a partial derivative of a function of several variables is its derivative with respect to one of those variables, with the others held constant (as opposed to the total derivative, in which all variables are allowed to vary). Par ...
of the error surface with respect to the weight.
Backpropagation In machine learning, backpropagation (backprop, BP) is a widely used algorithm for training feedforward artificial neural networks. Generalizations of backpropagation exist for other artificial neural networks (ANNs), and for functions gene ...
(BP), first made popular in the 1980s, is probably the most commonly known connectionist gradient descent algorithm today. Connectionism can be traced to ideas more than a century old, which were little more than speculation until the mid-to-late 20th century.


Parallel distributed processing

The prevailing connectionist approach today was originally known as parallel distributed processing (PDP). It was an
artificial neural network Artificial neural networks (ANNs), usually simply called neural networks (NNs) or neural nets, are computing systems inspired by the biological neural networks that constitute animal brains. An ANN is based on a collection of connected units ...
approach that stressed the parallel nature of neural processing, and the distributed nature of neural representations. It provided a general mathematical framework for researchers to operate in. The framework involved eight major aspects: * A set of ''processing units'', represented by a set of integers. * An ''activation'' for each unit, represented by a vector of time-dependent functions. * An ''output function'' for each unit, represented by a vector of functions on the activations. * A ''pattern of connectivity'' among units, represented by a matrix of real numbers indicating connection strength. * A ''propagation rule'' spreading the activations via the connections, represented by a function on the output of the units. * An ''activation rule'' for combining inputs to a unit to determine its new activation, represented by a function on the current activation and propagation. * A ''
learning rule An artificial neural network's learning rule or learning process is a method, mathematical logic or algorithm which improves the network's performance and/or training time. Usually, this rule is applied repeatedly over the network. It is done by ...
'' for modifying connections based on experience, represented by a change in the weights based on any number of variables. * An ''environment'' that provides the system with experience, represented by sets of activation vectors for some
subset In mathematics, set ''A'' is a subset of a set ''B'' if all elements of ''A'' are also elements of ''B''; ''B'' is then a superset of ''A''. It is possible for ''A'' and ''B'' to be equal; if they are unequal, then ''A'' is a proper subset o ...
of the units. A lot of the research that led to the development of PDP was done in the 1970s, but PDP became popular in the 1980s with the release of the books ''Parallel Distributed Processing: Explorations in the Microstructure of Cognition - Volume 1 (foundations)'' and ''Volume 2 (Psychological and Biological Models)'', by James L. McClelland,
David E. Rumelhart David Everett Rumelhart (June 12, 1942 – March 13, 2011) was an American psychologist who made many contributions to the formal analysis of human cognition, working primarily within the frameworks of mathematical psychology, symbolic artifi ...
and the PDP Research Group. The books are now considered seminal connectionist works, and it is now common to fully equate PDP and connectionism, although the term "connectionism" is not used in the books. Following the PDP model, researchers have theorized systems based on the principles of perpendicular distributed processing (PDP).


Earlier work

PDP's direct roots were the
perceptron In machine learning, the perceptron (or McCulloch-Pitts neuron) is an algorithm for supervised classification, supervised learning of binary classification, binary classifiers. A binary classifier is a function which can decide whether or not an ...
theories of researchers such as
Frank Rosenblatt Frank Rosenblatt (July 11, 1928July 11, 1971) was an American psychologist notable in the field of artificial intelligence. He is sometimes called the father of deep learning. Life and career Rosenblatt was born in New Rochelle, New York as son o ...
from the 1950s and 1960s. But perceptron models were made very unpopular by the book ''Perceptrons'' by
Marvin Minsky Marvin Lee Minsky (August 9, 1927 – January 24, 2016) was an American cognitive and computer scientist concerned largely with research of artificial intelligence (AI), co-founder of the Massachusetts Institute of Technology's AI laboratory, an ...
and
Seymour Papert Seymour Aubrey Papert (; 29 February 1928 – 31 July 2016) was a South African-born American mathematician, computer scientist, and educator, who spent most of his career teaching and researching at MIT. He was one of the pioneers of artificia ...
, published in 1969. It demonstrated the limits on the sorts of functions that single-layered (no hidden layer) perceptrons can calculate, showing that even simple functions like the
exclusive disjunction Exclusive or or exclusive disjunction is a logical operation that is true if and only if its arguments differ (one is true, the other is false). It is symbolized by the prefix operator J and by the infix operators XOR ( or ), EOR, EXOR, , , ...
(XOR) could not be handled properly. The PDP books overcame this limitation by showing that multi-level, non-linear neural networks were far more robust and could be used for a vast array of functions. Many earlier researchers advocated connectionist style models, for example in the 1940s and 1950s,
Warren McCulloch Warren Sturgis McCulloch (November 16, 1898 – September 24, 1969) was an American neurophysiologist and cybernetician, known for his work on the foundation for certain brain theories and his contribution to the cybernetics movement.Ken Aizawa ( ...
and
Walter Pitts Walter Harry Pitts, Jr. (23 April 1923 – 14 May 1969) was a logician who worked in the field of computational neuroscience.Smalheiser, Neil R"Walter Pitts", ''Perspectives in Biology and Medicine'', Volume 43, Number 2, Winter 2000, pp. 2 ...
( MP neuron), Donald Olding Hebb, and
Karl Lashley Karl Spencer Lashley (June 7, 1890 – August 7, 1958) was a psychologist and behaviorist remembered for his contributions to the study of learning and memory. A ''Review of General Psychology'' survey, published in 2002, ranked Lashley as the 61 ...
. McCulloch and Pitts showed how neural systems could implement
first-order logic First-order logic—also known as predicate logic, quantificational logic, and first-order predicate calculus—is a collection of formal systems used in mathematics, philosophy, linguistics, and computer science. First-order logic uses quantifi ...
: Their classic paper "A Logical Calculus of Ideas Immanent in Nervous Activity" (1943) is important in this development here. They were influenced by the important work of
Nicolas Rashevsky Nicolas Rashevsky (November 9, 1899 – January 16, 1972) was an American theoretical physicist who was one of the pioneers of mathematical biology, and is also considered the father of mathematical biophysics and theoretical biology. Rober ...
in the 1930s. Hebb contributed greatly to speculations about neural functioning, and proposed a learning principle,
Hebbian learning Hebbian theory is a neuroscientific theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptatio ...
, that is still used today. Lashley argued for distributed representations as a result of his failure to find anything like a localized engram in years of lesion experiments.


Connectionism apart from PDP

Though PDP is the dominant form of connectionism, other theoretical work should also be classified as connectionist. Many connectionist principles can be traced to early work in
psychology Psychology is the scientific study of mind and behavior. Psychology includes the study of conscious and unconscious phenomena, including feelings and thoughts. It is an academic discipline of immense scope, crossing the boundaries between ...
, such as that of
William James William James (January 11, 1842 – August 26, 1910) was an American philosopher, historian, and psychologist, and the first educator to offer a psychology course in the United States. James is considered to be a leading thinker of the la ...
. Psychological theories based on knowledge about the human brain were fashionable in the late 19th century. As early as 1869, the neurologist
John Hughlings Jackson John Hughlings Jackson, FRS (4 April 1835 – 7 October 1911) was an English neurologist. He is best known for his research on epilepsy. Biography He was born at Providence Green, Green Hammerton, near Harrogate, Yorkshire, the youngest so ...
argued for multi-level, distributed systems. Following from this lead,
Herbert Spencer Herbert Spencer (27 April 1820 – 8 December 1903) was an English philosopher, psychologist, biologist, anthropologist, and sociologist famous for his hypothesis of social Darwinism. Spencer originated the expression "survival of the fittest ...
's ''Principles of Psychology'', 3rd edition (1872), and
Sigmund Freud Sigmund Freud ( , ; born Sigismund Schlomo Freud; 6 May 1856 – 23 September 1939) was an Austrian neurologist and the founder of psychoanalysis, a clinical method for evaluating and treating pathologies explained as originating in conflicts in ...
's ''Project for a Scientific Psychology'' (composed 1895) propounded connectionist or proto-connectionist theories. These tended to be speculative theories. But by the early 20th century,
Edward Thorndike Edward Lee Thorndike (August 31, 1874 – August 9, 1949) was an American psychologist who spent nearly his entire career at Teachers College, Columbia University. His work on comparative psychology and the learning process led to the theory of ...
was experimenting on learning that posited a connectionist type network.
Friedrich Hayek Friedrich August von Hayek ( , ; 8 May 189923 March 1992), often referred to by his initials F. A. Hayek, was an Austrian–British economist, legal theorist and philosopher who is best known for his defense of classical liberalism. Haye ...
independently conceived the Hebbian synapse learning model in a paper presented in 1920 and developed that model into global brain theory constituted of networks Hebbian synapses building into larger systems of maps and memory network . Hayek's breakthrough work was cited by Frank Rosenblatt in his perceptron paper. Another form of connectionist model was the relational network framework developed by the
linguist Linguistics is the scientific study of human language. It is called a scientific study because it entails a comprehensive, systematic, objective, and precise analysis of all aspects of language, particularly its nature and structure. Lingui ...
Sydney Lamb Sydney MacDonald Lamb (born May 4, 1929 in Denver, Colorado) is an American linguist and professor at Rice University, whose stratificational grammar is a significant alternative theory to Chomsky's transformational grammar. He has specializ ...
in the 1960s. Relational networks have been only used by linguists, and were never unified with the PDP approach. As a result, they are now used by very few researchers. There are also hybrid connectionist models, mostly mixing symbolic representations with neural network models. The hybrid approach has been advocated by some researchers (such as Ron Sun).


Connectionism vs. computationalism debate

As connectionism became increasingly popular in the late 1980s, some researchers (including
Jerry Fodor Jerry Alan Fodor (; April 22, 1935 – November 29, 2017) was an American philosopher and the author of many crucial works in the fields of philosophy of mind and cognitive science. His writings in these fields laid the groundwork for the modul ...
,
Steven Pinker Steven Arthur Pinker (born September 18, 1954) is a Canadian-American cognitive psychologist, psycholinguist, popular science author, and public intellectual. He is an advocate of evolutionary psychology and the computational theory of mind. P ...
and others) reacted against it. They argued that connectionism, as then developing, threatened to obliterate what they saw as the progress being made in the fields of cognitive science and psychology by the classical approach of
computationalism In philosophy of mind, the computational theory of mind (CTM), also known as computationalism, is a family of views that hold that the human mind is an information processing system and that cognition and consciousness together are a form of comp ...
. Computationalism is a specific form of cognitivism that argues that mental activity is
computational Computation is any type of arithmetic or non-arithmetic calculation that follows a well-defined model (e.g., an algorithm). Mechanical or electronic devices (or, historically, people) that perform computations are known as '' computers''. An ...
, that is, that the mind operates by performing purely formal operations on symbols, like a
Turing machine A Turing machine is a mathematical model of computation describing an abstract machine that manipulates symbols on a strip of tape according to a table of rules. Despite the model's simplicity, it is capable of implementing any computer algo ...
. Some researchers argued that the trend in connectionism represented a reversion toward associationism and the abandonment of the idea of a
language of thought The language of thought hypothesis (LOTH), sometimes known as thought ordered mental expression (TOME), is a view in linguistics, philosophy of mind and cognitive science, forwarded by American philosopher Jerry Fodor. It describes the nature of th ...
, something they saw as mistaken. In contrast, those very tendencies made connectionism attractive for other researchers. Connectionism and computationalism need not be at odds, but the debate in the late 1980s and early 1990s led to opposition between the two approaches. Throughout the debate, some researchers have argued that connectionism and computationalism are fully compatible, though full consensus on this issue has not been reached. Differences between the two approaches include the following: * Computationalists posit symbolic models that are structurally similar to underlying brain structure, whereas connectionists engage in "low-level" modeling, trying to ensure that their models resemble neurological structures. * Computationalists in general focus on the structure of explicit symbols (
mental models A mental model is an explanation of someone's thought process about how something works in the real world. It is a representation of the surrounding world, the relationships between its various parts and a person's intuitive perception about thei ...
) and syntactical rules for their internal manipulation, whereas connectionists focus on learning from environmental stimuli and storing this information in a form of connections between neurons. * Computationalists believe that internal mental activity consists of manipulation of explicit symbols, whereas connectionists believe that the manipulation of explicit symbols provides a poor model of mental activity. * Computationalists often posit domain specific symbolic sub-systems designed to support learning in specific areas of cognition (e.g., language, intentionality, number), whereas connectionists posit one or a small set of very general learning-mechanisms. Despite these differences, some theorists have proposed that the connectionist architecture is simply the manner in which organic brains happen to implement the symbol-manipulation system. This is logically possible, as it is well known that connectionist models can implement symbol-manipulation systems of the kind used in computationalist models, as indeed they must be able if they are to explain the human ability to perform symbol-manipulation tasks. Several cognitive models combining both symbol-manipulative and connectionist architectures have been proposed, notably among them
Paul Smolensky Paul Smolensky (born May 5, 1955) is Krieger-Eisenhower Professor of Cognitive Science at the Johns Hopkins University and a Senior Principal Researcher at Microsoft Research, Redmond Washington. Along with Alan Prince, in 1993 he developed Opt ...
's Integrated Connectionist/Symbolic Cognitive Architecture (ICS). But the debate rests on whether this symbol manipulation forms the foundation of cognition in general, so this is not a potential vindication of computationalism. Nonetheless, computational descriptions may be helpful high-level descriptions of cognition of logic, for example. The debate was largely centred on logical arguments about whether connectionist networks could produce the syntactic structure observed in this sort of reasoning. This was later achieved although using fast-variable binding abilities outside of those standardly assumed in connectionist models. Part of the appeal of computational descriptions is that they are relatively easy to interpret, and thus may be seen as contributing to our understanding of particular mental processes, whereas connectionist models are in general more opaque, to the extent that they may be describable only in very general terms (such as specifying the learning algorithm, the number of units, etc.), or in unhelpfully low-level terms. In this sense connectionist models may instantiate, and thereby provide evidence for, a broad theory of cognition (i.e., connectionism), without representing a helpful theory of the particular process that is being modelled. In this sense the debate might be considered as to some extent reflecting a mere difference in the level of analysis in which particular theories are framed. Some researchers suggest that the analysis gap is the consequence of connectionist mechanisms giving rise to
emergent phenomena In philosophy, systems theory, science, and art, emergence occurs when an entity is observed to have properties its parts do not have on their own, properties or behaviors that emerge only when the parts interact in a wider whole. Emergence ...
that may be describable in computational terms. In the 2000s the popularity of
dynamical systems In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water in ...
in
philosophy of mind Philosophy of mind is a branch of philosophy that studies the ontology and nature of the mind and its relationship with the body. The mind–body problem is a paradigmatic issue in philosophy of mind, although a number of other issues are addre ...
have added a new perspective on the debate; some authors now argue that any split between connectionism and computationalism is more conclusively characterized as a split between computationalism and
dynamical systems In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water in ...
. In 2014,
Alex Graves Alexander John Graves (born July 23, 1965) is an American film director, television director, television producer and screenwriter. Early life Alex Graves was born in Kansas City, Missouri. His father, William Graves, was a reporter for ''The K ...
and others from
DeepMind DeepMind Technologies is a British artificial intelligence subsidiary of Alphabet Inc. and research laboratory founded in 2010. DeepMind was acquired by Google in 2014 and became a wholly owned subsidiary of Alphabet Inc, after Google's rest ...
published a series of papers describing a novel Deep Neural Network structure called the Neural Turing Machine able to read symbols on a tape and store symbols in memory. Relational Networks, another Deep Network module published by DeepMind, are able to create object-like representations and manipulate them to answer complex questions. Relational Networks and Neural Turing Machines are further evidence that connectionism and computationalism need not be at odds.


Symbolism vs. connectionism debate

Smolensky's Subsymbolic Paradigm has to meet the Fodor-Pylyshyn challenge formulated by classical symbol theory for a convincing theory of cognition in modern connectionism. In order to be an adequate alternative theory of cognition, Smolensky's Subsymbolic Paradigm would have to explain the existence of systematicity or systematic relations in language cognition without the assumption that cognitive processes are causally sensitive to the classical constituent structure of mental representations. The subsymbolic paradigm, or connectionism in general, would thus have to explain the existence of systematicity and compositionality without relying on the mere implementation of a classical cognitive architecture. This challenge implies a dilemma: If the Subsymbolic Paradigm could contribute nothing to the systematicity and compositionality of mental representations, it would be insufficient as a basis for an alternative theory of cognition. However, if the Subsymbolic Paradigm's contribution to systematicity requires mental processes grounded in the classical constituent structure of mental representations, the theory of cognition it develops would be, at best, an implementation architecture of the classical model of symbol theory and thus not a genuine alternative (connectionist) theory of cognition. The classical model of symbolism is characterized by (1.) a combinatorial syntax and semantics of mental representations and (2.) mental operations as structure-sensitive processes, based on the fundamental principle of syntactic and semantic constituent structure of mental representations as used in Fodor's "Language of Thought (LOT)". This can be used to explain the following closely related properties of human cognition, namely its (1.) productivity, (2.) systematicity, (3.) compositionality, and (4.) inferential coherence. This challenge has been met in modern connectionism, for example, not only by Smolensky's "Integrated Connectionist/Symbolic (ICS) Cognitive Architecture", but also by Werning's and Maye's "Oscillatory Networks". An overview of this is given for example by Bechtel & Abrahamsen, Marcus and Maurer.H. Maurer: Cognitive science: Integrative synchronization mechanisms in cognitive neuroarchitectures of the modern connectionism. CRC Press, Boca Raton/FL, 2021, ISBN 978-1-351-04352-6. https://doi.org/10.1201/9781351043526


See also

* Associationism *
Artificial intelligence Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech re ...
*
Behaviorism Behaviorism is a systematic approach to understanding the behavior of humans and animals. It assumes that behavior is either a reflex evoked by the pairing of certain antecedent stimuli in the environment, or a consequence of that individual ...
* Catastrophic interference *
Calculus of relations In mathematical logic, algebraic logic is the reasoning obtained by manipulating equations with free variables. What is now usually called classical algebraic logic focuses on the identification and algebraic description of models appropriate for ...
*
Cybernetics Cybernetics is a wide-ranging field concerned with circular causality, such as feedback, in regulatory and purposive systems. Cybernetics is named after an example of circular causal feedback, that of steering a ship, where the helmsperson ma ...
*
Deep learning Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised. D ...
*
Eliminative materialism Eliminative materialism (also called eliminativism) is a materialist position in the philosophy of mind. It is the idea that majority of the mental states in folk psychology do not exist. Some supporters of eliminativism argue that no coher ...
*
Feature integration theory Feature integration theory is a theory of attention developed in 1980 by Anne Treisman and Garry Gelade that suggests that when perceiving a stimulus, features are "registered early, automatically, and in parallel, while objects are identified separ ...
*
Genetic algorithm In computer science and operations research, a genetic algorithm (GA) is a metaheuristic inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms (EA). Genetic algorithms are commonly used to gene ...
* Harmonic grammar *
Machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machin ...
*
Pandemonium architecture Pandemonium architecture is a theory in cognitive science that describes how visual images are processed by the brain. It has applications in artificial intelligence and pattern recognition. The theory was developed by the artificial intelligence ...
*
Self-organizing map A self-organizing map (SOM) or self-organizing feature map (SOFM) is an unsupervised learning, unsupervised machine learning technique used to produce a dimensionality reduction, low-dimensional (typically two-dimensional) representation of a hig ...


Notes


References

* Rumelhart, D.E., J.L. McClelland and the PDP Research Group (1986). ''Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations'', Cambridge, Massachusetts:
MIT Press The MIT Press is a university press affiliated with the Massachusetts Institute of Technology (MIT) in Cambridge, Massachusetts (United States). It was established in 1962. History The MIT Press traces its origins back to 1926 when MIT publish ...
, * McClelland, J.L., D.E. Rumelhart and the PDP Research Group (1986). ''Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 2: Psychological and Biological Models'', Cambridge, Massachusetts: MIT Press, * Pinker, Steven and Mehler, Jacques (1988). ''Connections and Symbols'', Cambridge MA: MIT Press, * Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, Kim Plunkett (1996). ''Rethinking Innateness: A connectionist perspective on development'', Cambridge MA: MIT Press, * Marcus, Gary F. (2001). ''The Algebraic Mind: Integrating Connectionism and Cognitive Science (Learning, Development, and Conceptual Change)'', Cambridge, Massachusetts: MIT Press, * * Maurer, Harald (2021). ''Cognitive Science: Integrative Synchronization Mechanisms in Cognitive Neuroarchitectures of the Modern Connectionism'', Boca Raton/FL: CRC Press, https://doi.org/10.1201/9781351043526,


External links


Dictionary of Philosophy of Mind entry on connectionism
*

* * ttps://sapienlabs.org/the-crisis-of-computational-neuroscience/ Critique of connectionism {{Authority control Cognitive science Computational neuroscience Theory of mind Learning Philosophy of artificial intelligence Emergence