
A sigmoid function is any
mathematical function
In mathematics, a function from a set (mathematics), set to a set assigns to each element of exactly one element of .; the words ''map'', ''mapping'', ''transformation'', ''correspondence'', and ''operator'' are sometimes used synonymously. ...
whose
graph has a characteristic S-shaped or sigmoid curve.
A common example of a sigmoid function is the
logistic function
A logistic function or logistic curve is a common S-shaped curve ( sigmoid curve) with the equation
f(x) = \frac
where
The logistic function has domain the real numbers, the limit as x \to -\infty is 0, and the limit as x \to +\infty is L.
...
, which is defined by the formula
:
Other sigmoid functions are given in the
Examples section. In some fields, most notably in the context of
artificial neural network
In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks.
A neural network consists of connected ...
s, the term "sigmoid function" is used as a synonym for "logistic function".
Special cases of the sigmoid function include the
Gompertz curve (used in modeling systems that saturate at large values of ''x'') and the
ogee curve (used in the
spillway
A spillway is a structure used to provide the controlled release of water downstream from a dam or levee, typically into the riverbed of the dammed river itself. In the United Kingdom, they may be known as overflow channels. Spillways ensure tha ...
of some
dams). Sigmoid functions have domain of all
real number
In mathematics, a real number is a number that can be used to measure a continuous one- dimensional quantity such as a duration or temperature. Here, ''continuous'' means that pairs of values can have arbitrarily small differences. Every re ...
s, with return (response) value commonly
monotonically increasing but could be decreasing. Sigmoid functions most often show a return value (''y'' axis) in the range 0 to 1. Another commonly used range is from −1 to 1.
A wide variety of sigmoid functions including the logistic and
hyperbolic tangent functions have been used as the
activation function of
artificial neurons. Sigmoid curves are also common in statistics as
cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ever ...
s (which go from 0 to 1), such as the integrals of the
logistic density, the
normal density, and
Student's ''t'' probability density functions. The logistic sigmoid function is invertible, and its inverse is the
logit
In statistics, the logit ( ) function is the quantile function associated with the standard logistic distribution. It has many uses in data analysis and machine learning, especially in Data transformation (statistics), data transformations.
Ma ...
function.
Definition
A sigmoid function is a
bounded,
differentiable
In mathematics, a differentiable function of one real variable is a function whose derivative exists at each point in its domain. In other words, the graph of a differentiable function has a non- vertical tangent line at each interior point in ...
, real function that is defined for all real input values and has a positive derivative at each point.
Properties
In general, a sigmoid function is
monotonic
In mathematics, a monotonic function (or monotone function) is a function between ordered sets that preserves or reverses the given order. This concept first arose in calculus, and was later generalized to the more abstract setting of ord ...
, and has a first
derivative
In mathematics, the derivative is a fundamental tool that quantifies the sensitivity to change of a function's output with respect to its input. The derivative of a function of a single variable at a chosen input value, when it exists, is t ...
which is
bell shaped. Conversely, the
integral
In mathematics, an integral is the continuous analog of a Summation, sum, which is used to calculate area, areas, volume, volumes, and their generalizations. Integration, the process of computing an integral, is one of the two fundamental oper ...
of any continuous, non-negative, bell-shaped function (with one local maximum and no local minimum, unless
degenerate) will be sigmoidal. Thus the
cumulative distribution function
In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x.
Ever ...
s for many common
probability distribution
In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
s are sigmoidal. One such example is the
error function
In mathematics, the error function (also called the Gauss error function), often denoted by , is a function \mathrm: \mathbb \to \mathbb defined as:
\operatorname z = \frac\int_0^z e^\,\mathrm dt.
The integral here is a complex Contour integrat ...
, which is related to the cumulative distribution function of a
normal distribution
In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is
f(x) = \frac ...
; another is the
arctan function, which is related to the cumulative distribution function of a
Cauchy distribution.
A sigmoid function is constrained by a pair of
horizontal asymptotes as
.
A sigmoid function is
convex
Convex or convexity may refer to:
Science and technology
* Convex lens, in optics
Mathematics
* Convex set, containing the whole line segment that joins points
** Convex polygon, a polygon which encloses a convex set of points
** Convex polytop ...
for values less than a particular point, and it is
concave for values greater than that point: in many of the examples here, that point is 0.
Examples
*
Logistic function
A logistic function or logistic curve is a common S-shaped curve ( sigmoid curve) with the equation
f(x) = \frac
where
The logistic function has domain the real numbers, the limit as x \to -\infty is 0, and the limit as x \to +\infty is L.
...
*
Hyperbolic tangent (shifted and scaled version of the logistic function, above)
*
Arctangent function
*
Gudermannian function
*
Error function
In mathematics, the error function (also called the Gauss error function), often denoted by , is a function \mathrm: \mathbb \to \mathbb defined as:
\operatorname z = \frac\int_0^z e^\,\mathrm dt.
The integral here is a complex Contour integrat ...
*
Generalised logistic function
*
Smoothstep function
* Some
algebraic functions, for example
* and in a more general form
* Up to shifts and scaling, many sigmoids are special cases of
where
is the inverse of the negative
Box–Cox transformation, and
and
are shape parameters.
*
Smooth transition function normalized to (−1,1):
using the hyperbolic tangent mentioned above. Here,
is a free parameter encoding the slope at
, which must be greater than or equal to
because any smaller value will result in a function with multiple inflection points, which is therefore not a true sigmoid. This function is unusual because it actually attains the limiting values of −1 and 1 within a finite range, meaning that its value is constant at −1 for all
and at 1 for all
. Nonetheless, it is
smooth (infinitely differentiable,
) ''everywhere'', including at
.
Applications

Many natural processes, such as those of complex system
learning curve
A learning curve is a graphical representation of the relationship between how proficient people are at a task and the amount of experience they have. Proficiency (measured on the vertical axis) usually increases with increased experience (the ...
s, exhibit a progression from small beginnings that accelerates and approaches a climax over time.
When a specific mathematical model is lacking, a sigmoid function is often used.
The
van Genuchten–Gupta model is based on an inverted S-curve and applied to the response of crop yield to
soil salinity
Soil salinity is the salt (chemistry), salt content in the soil; the process of increasing the salt content is known as salinization (also called salination in American and British English spelling differences, American English). Salts occur nat ...
.
Examples of the application of the logistic S-curve to the response of crop yield (wheat) to both the soil salinity and depth to
water table
The water table is the upper surface of the phreatic zone or zone of saturation. The zone of saturation is where the pores and fractures of the ground are saturated with groundwater, which may be fresh, saline, or brackish, depending on the loc ...
in the soil are shown in
modeling crop response in agriculture.
In
artificial neural network
In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks.
A neural network consists of connected ...
s, sometimes non-smooth functions are used instead for efficiency; these are known as
hard sigmoids.
In
audio signal processing
Audio signal processing is a subfield of signal processing that is concerned with the electronic manipulation of audio signals. Audio signals are electronic representations of sound waves—longitudinal waves which travel through air, consisting ...
, sigmoid functions are used as
waveshaper transfer function
In engineering, a transfer function (also known as system function or network function) of a system, sub-system, or component is a function (mathematics), mathematical function that mathematical model, models the system's output for each possible ...
s to emulate the sound of
analog circuitry clipping.
In
biochemistry
Biochemistry, or biological chemistry, is the study of chemical processes within and relating to living organisms. A sub-discipline of both chemistry and biology, biochemistry may be divided into three fields: structural biology, enzymology, a ...
and
pharmacology
Pharmacology is the science of drugs and medications, including a substance's origin, composition, pharmacokinetics, pharmacodynamics, therapeutic use, and toxicology. More specifically, it is the study of the interactions that occur betwee ...
, the
Hill
A hill is a landform that extends above the surrounding terrain. It often has a distinct summit, and is usually applied to peaks which are above elevation compared to the relative landmass, though not as prominent as Mountain, mountains. Hills ...
and
Hill–Langmuir equations are sigmoid functions.
In computer graphics and real-time rendering, some of the sigmoid functions are used to blend colors or geometry between two values, smoothly and without visible seams or discontinuities.
Titration curves between strong acids and strong bases have a sigmoid shape due to the logarithmic nature of the
pH scale.
The logistic function can be calculated efficiently by utilizing
type III Unums.
An hierarchy of sigmoid growth models with increasing complexity (number of parameters) was built
with the primary goal to re-analyze kinetic data, the so called N-t curves, from heterogeneous
nucleation
In thermodynamics, nucleation is the first step in the formation of either a new Phase (matter), thermodynamic phase or Crystal structure, structure via self-assembly or self-organization within a substance or mixture. Nucleation is typically def ...
experiments,
in
electrochemistry
Electrochemistry is the branch of physical chemistry concerned with the relationship between Electric potential, electrical potential difference and identifiable chemical change. These reactions involve Electron, electrons moving via an electronic ...
. The hierarchy includes at present three models, with 1, 2 and 3 parameters, if not counting the maximal number of nuclei N
max, respectively—a tanh
2 based model called α
21 originally devised to describe diffusion-limited crystal growth (not aggregation!) in 2D, the Johnson-Mehl-Avrami-Kolmogorov (JMAKn) model,
and the Richards model.
It was shown that for the concrete purpose even the simplest model works and thus it was implied that the experiments revisited are an example of two-step nucleation with the first step being the growth of the metastable phase in which the nuclei of the stable phase form.
See also
*
*
*
*
*
*
*
*
*
*
*
References
Further reading
* . (NB. In particular see "Chapter 4: Artificial Neural Networks" (in particular pp. 96–97) where Mitchell uses the word "logistic function" and the "sigmoid function" synonymously – this function he also calls the "squashing function" – and the sigmoid (aka logistic) function is used to compress the outputs of the "neurons" in multi-layer neural nets.)
* (NB. Properties of the sigmoid, including how it can shift along axes and how its domain may be transformed.)
External links
*
{{Artificial intelligence navbox
Elementary special functions
Artificial neural networks