
A
Boltzmann
Ludwig Eduard Boltzmann (; 20 February 1844 – 5 September 1906) was an Austrian physicist and philosopher. His greatest achievements were the development of statistical mechanics, and the statistical explanation of the second law of thermodyn ...
machine (also called Sherrington–Kirkpatrick model with external field or stochastic Ising–Lenz–Little model) is a stochastic spin-glass model with an external field, i.e., a
Sherrington–Kirkpatrick model
In condensed matter physics, a spin glass is a magnetic state characterized by randomness, besides cooperative behavior in freezing of spins at a temperature called 'freezing temperature' ''Tf''. In ferromagnetic solids, component atoms' mag ...
, that is a stochastic
Ising model. It is a
statistical physics technique applied in the context of
cognitive science. It is also classified as a
Markov random field.
Boltzmann machines are theoretically intriguing because of the locality and
Hebbian nature of their training algorithm (being trained by Hebb's rule), and because of their
parallelism and the resemblance of their dynamics to simple
physical processes. Boltzmann machines with unconstrained connectivity have not been proven useful for practical problems in machine learning or inference, but if the connectivity is properly constrained, the learning can be made efficient enough to be useful for practical problems.
They are named after the
Boltzmann distribution in
statistical mechanics
In physics, statistical mechanics is a mathematical framework that applies statistical methods and probability theory to large assemblies of microscopic entities. It does not assume or postulate any natural laws, but explains the macroscopic be ...
, which is used in their
sampling function
In mathematics, a Dirac comb (also known as shah function, impulse train or sampling function) is a periodic function with the formula
\operatorname_(t) \ := \sum_^ \delta(t - k T)
for some given period T. Here ''t'' is a real variable and th ...
. They were heavily popularized and promoted by
Geoffrey Hinton,
Terry Sejnowski and
Yann LeCun in cognitive sciences communities and in
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
. As a more general class within
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
these models are called "
energy based model
An energy-based model (EBM) is a form of generative model (GM) imported directly from statistical physics to learning. GMs learn an underlying data distribution by analyzing a sample dataset. Once trained, a GM can produce other datasets that als ...
s" (EBM), because
Hamiltonians of
spin glasses
In condensed matter physics, a spin glass is a magnetic state characterized by randomness, besides cooperative behavior in freezing of spins at a temperature called 'freezing temperature' ''Tf''. In ferromagnetic solids, component atoms' mag ...
are used as a starting point to define the learning task.
Structure

A Boltzmann machine, like a
Sherrington–Kirkpatrick model
In condensed matter physics, a spin glass is a magnetic state characterized by randomness, besides cooperative behavior in freezing of spins at a temperature called 'freezing temperature' ''Tf''. In ferromagnetic solids, component atoms' mag ...
, is a network of units with a total "energy" (
Hamiltonian
Hamiltonian may refer to:
* Hamiltonian mechanics, a function that represents the total energy of a system
* Hamiltonian (quantum mechanics), an operator corresponding to the total energy of that system
** Dyall Hamiltonian, a modified Hamiltonian ...
) defined for the overall network. Its units produce
binary
Binary may refer to:
Science and technology Mathematics
* Binary number, a representation of numbers using only two digits (0 and 1)
* Binary function, a function that takes two arguments
* Binary operation, a mathematical operation that t ...
results. Boltzmann machine weights are
stochastic
Stochastic (, ) refers to the property of being well described by a random probability distribution. Although stochasticity and randomness are distinct in that the former refers to a modeling approach and the latter refers to phenomena themselv ...
. The global energy
in a Boltzmann machine is identical in form to that of
Hopfield networks and
Ising models:
:
Where:
*
is the connection strength between unit
and unit
.
*
is the state,
, of unit
.
*
is the bias of unit
in the global energy function. (
is the activation threshold for the unit.)
Often the weights
are represented as a symmetric matrix