neuroscience Neuroscience is the scientific study of the nervous system (the brain, spinal cord, and peripheral nervous system), its functions and disorders. It is a multidisciplinary science that combines physiology, anatomy, molecular biology, development ...

and

computer science Computer science is the study of computation, automation, and information. Computer science spans theoretical disciplines (such as algorithms, theory of computation, information theory, and automation) to Applied science, practical discipli ...

, synaptic weight refers to the strength or

amplitude The amplitude of a periodic variable is a measure of its change in a single period (such as time or spatial period). The amplitude of a non-periodic signal is its magnitude compared with a reference value. There are various definitions of amplit ...

of a connection between two nodes, corresponding in biology to the amount of influence the

firing Dismissal (also called firing) is the termination of employment by an employer against the will of the employee. Though such a decision can be made by an employer for a variety of reasons, ranging from an economic downturn to performance-related ...

of one

neuron A neuron, neurone, or nerve cell is an electrically excitable cell that communicates with other cells via specialized connections called synapses. The neuron is the main component of nervous tissue in all animals except sponges and placozoa. N ...

has on another. The term is typically used in

artificial Artificiality (the state of being artificial or manmade) is the state of being the product of intentional human manufacture, rather than occurring naturally through processes not involving or requiring human activity. Connotations Artificiality ...

and

biological Biology is the scientific study of life. It is a natural science with a broad scope but has several unifying themes that tie it together as a single, coherent field. For instance, all organisms are made up of cells that process hereditary in ...

neural network A neural network is a network or circuit of biological neurons, or, in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus, a neural network is either a biological neural network, made up of biological ...

research.

Computation

In a computational neural network, a

vector Vector most often refers to: *Euclidean vector, a quantity with a magnitude and a direction *Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism Vector may also refer to: Mathematic ...

or set of inputs

\textbf

and outputs

\textbf

, or pre- and post-synaptic neurons respectively, are interconnected with synaptic weights represented by the matrix

w

, where for a linear neuron :

y_j = \sum_i w_ x_i ~~\textrm~~ \textbf = w\textbf

. where the rows of the synaptic matrix represent the vector of synaptic weights for the output indexed by

j

. The synaptic weight is changed by using a learning rule, the most basic of which is

Hebb's rule Hebbian theory is a neuroscientific theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptatio ...

, which is usually stated in biological terms as

''Neurons that fire together, wire together.''

Computationally, this means that if a large signal from one of the input neurons results in a large signal from one of the output neurons, then the synaptic weight between those two neurons will increase. The rule is unstable, however, and is typically modified using such variations as Oja's rule,

radial basis functions A radial basis function (RBF) is a real-valued function \varphi whose value depends only on the distance between the input and some fixed point, either the origin, so that \varphi(\mathbf) = \hat\varphi(\left\, \mathbf\right\, ), or some other fixed ...

or the

backpropagation In machine learning, backpropagation (backprop, BP) is a widely used algorithm for training feedforward neural network, feedforward artificial neural networks. Generalizations of backpropagation exist for other artificial neural networks (ANN ...

algorithm.

Biology

For biological networks, the effect of synaptic weights is not as simple as for linear neurons or

Hebbian learning Hebbian theory is a neuroscientific theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptation ...

. However,

biophysical Biophysics is an interdisciplinary science that applies approaches and methods traditionally used in physics to study biological phenomena. Biophysics covers all scales of biological organization, from molecular to organismic and populations. Bi ...

models such as

BCM theory BCM theory, BCM synaptic modification, or the BCM rule, named for Elie Bienenstock, Leon Cooper, and Paul Munro, is a physical theory of learning in the visual cortex developed in 1981. The BCM model proposes a sliding threshold for long-term pote ...

have seen some success in mathematically describing these networks. In the mammalian

central nervous system The central nervous system (CNS) is the part of the nervous system consisting primarily of the brain and spinal cord. The CNS is so named because the brain integrates the received information and coordinates and influences the activity of all par ...

, signal transmission is carried out by interconnected networks of nerve cells, or neurons. For the basic

pyramidal neuron Pyramidal cells, or pyramidal neurons, are a type of multipolar neuron found in areas of the brain including the cerebral cortex, the hippocampus, and the amygdala. Pyramidal neurons are the primary excitation units of the mammalian prefrontal cor ...

, the input signal is carried by the

axon An axon (from Greek ἄξων ''áxōn'', axis), or nerve fiber (or nerve fibre: see spelling differences), is a long, slender projection of a nerve cell, or neuron, in vertebrates, that typically conducts electrical impulses known as action po ...

, which releases neurotransmitter chemicals into the

synapse In the nervous system, a synapse is a structure that permits a neuron (or nerve cell) to pass an electrical or chemical signal to another neuron or to the target effector cell. Synapses are essential to the transmission of nervous impulses from ...

which is picked up by the

dendrites Dendrites (from Greek δένδρον ''déndron'', "tree"), also dendrons, are branched protoplasmic extensions of a nerve cell that propagate the electrochemical stimulation received from other neural cells to the cell body, or soma, of the n ...

of the next neuron, which can then generate an

action potential An action potential occurs when the membrane potential of a specific cell location rapidly rises and falls. This depolarization then causes adjacent locations to similarly depolarize. Action potentials occur in several types of animal cells, ...

which is analogous to the output signal in the computational case. The synaptic weight in this process is determined by several variable factors: * How well the input signal propagates through the axon (see

myelination Myelin is a lipid-rich material that surrounds nerve cell axons (the nervous system's "wires") to insulate them and increase the rate at which electrical impulses (called action potentials) are passed along the axon. The myelinated axon can be l ...

), * The amount of neurotransmitter released into the synapse and the amount that can be absorbed in the following cell (determined by the number of

AMPA α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid, better known as AMPA, is a compound that is a specific agonist for the AMPA receptor, where it mimics the effects of the neurotransmitter glutamate. There are several types of glutamatergic ...

and

NMDA receptor The ''N''-methyl-D-aspartate receptor (also known as the NMDA receptor or NMDAR), is a glutamate receptor and ion channel found in neurons. The NMDA receptor is one of three types of ionotropic glutamate receptors, the other two being AMPA rece ...

s on the cell membrane and the amount of intracellular

calcium Calcium is a chemical element with the symbol Ca and atomic number 20. As an alkaline earth metal, calcium is a reactive metal that forms a dark oxide-nitride layer when exposed to air. Its physical and chemical properties are most similar to ...

and other ions), * The number of such connections made by the axon to the dendrites, * How well the signal propagates and integrates in the postsynaptic cell. The changes in synaptic weight that occur is known as

synaptic plasticity In neuroscience, synaptic plasticity is the ability of synapses to strengthen or weaken over time, in response to increases or decreases in their activity. Since memories are postulated to be represented by vastly interconnected neural circuit ...

, and the process behind long-term changes (

long-term potentiation In neuroscience, long-term potentiation (LTP) is a persistent strengthening of synapses based on recent patterns of activity. These are patterns of synaptic activity that produce a long-lasting increase in signal transmission between two neurons ...

and depression) is still poorly understood. Hebb's original learning rule was originally applied to biological systems, but has had to undergo many modifications as a number of theoretical and experimental problems came to light.

Computation

Biology

References

See also