HOME

TheInfoList



OR:

A sigmoid function is any
mathematical function In mathematics, a function from a set (mathematics), set to a set assigns to each element of exactly one element of .; the words ''map'', ''mapping'', ''transformation'', ''correspondence'', and ''operator'' are sometimes used synonymously. ...
whose graph has a characteristic S-shaped or sigmoid curve. A common example of a sigmoid function is the
logistic function A logistic function or logistic curve is a common S-shaped curve ( sigmoid curve) with the equation f(x) = \frac where The logistic function has domain the real numbers, the limit as x \to -\infty is 0, and the limit as x \to +\infty is L. ...
, which is defined by the formula :\sigma(x) = \frac = \frac = 1 - \sigma(-x). Other sigmoid functions are given in the Examples section. In some fields, most notably in the context of
artificial neural network In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks. A neural network consists of connected ...
s, the term "sigmoid function" is used as a synonym for "logistic function". Special cases of the sigmoid function include the Gompertz curve (used in modeling systems that saturate at large values of ''x'') and the ogee curve (used in the
spillway A spillway is a structure used to provide the controlled release of water downstream from a dam or levee, typically into the riverbed of the dammed river itself. In the United Kingdom, they may be known as overflow channels. Spillways ensure tha ...
of some dams). Sigmoid functions have domain of all
real number In mathematics, a real number is a number that can be used to measure a continuous one- dimensional quantity such as a duration or temperature. Here, ''continuous'' means that pairs of values can have arbitrarily small differences. Every re ...
s, with return (response) value commonly monotonically increasing but could be decreasing. Sigmoid functions most often show a return value (''y'' axis) in the range 0 to 1. Another commonly used range is from −1 to 1. A wide variety of sigmoid functions including the logistic and hyperbolic tangent functions have been used as the activation function of artificial neurons. Sigmoid curves are also common in statistics as
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ever ...
s (which go from 0 to 1), such as the integrals of the logistic density, the normal density, and Student's ''t'' probability density functions. The logistic sigmoid function is invertible, and its inverse is the
logit In statistics, the logit ( ) function is the quantile function associated with the standard logistic distribution. It has many uses in data analysis and machine learning, especially in Data transformation (statistics), data transformations. Ma ...
function.


Definition

A sigmoid function is a bounded,
differentiable In mathematics, a differentiable function of one real variable is a function whose derivative exists at each point in its domain. In other words, the graph of a differentiable function has a non- vertical tangent line at each interior point in ...
, real function that is defined for all real input values and has a positive derivative at each point.


Properties

In general, a sigmoid function is
monotonic In mathematics, a monotonic function (or monotone function) is a function between ordered sets that preserves or reverses the given order. This concept first arose in calculus, and was later generalized to the more abstract setting of ord ...
, and has a first
derivative In mathematics, the derivative is a fundamental tool that quantifies the sensitivity to change of a function's output with respect to its input. The derivative of a function of a single variable at a chosen input value, when it exists, is t ...
which is bell shaped. Conversely, the
integral In mathematics, an integral is the continuous analog of a Summation, sum, which is used to calculate area, areas, volume, volumes, and their generalizations. Integration, the process of computing an integral, is one of the two fundamental oper ...
of any continuous, non-negative, bell-shaped function (with one local maximum and no local minimum, unless degenerate) will be sigmoidal. Thus the
cumulative distribution function In probability theory and statistics, the cumulative distribution function (CDF) of a real-valued random variable X, or just distribution function of X, evaluated at x, is the probability that X will take a value less than or equal to x. Ever ...
s for many common
probability distribution In probability theory and statistics, a probability distribution is a Function (mathematics), function that gives the probabilities of occurrence of possible events for an Experiment (probability theory), experiment. It is a mathematical descri ...
s are sigmoidal. One such example is the
error function In mathematics, the error function (also called the Gauss error function), often denoted by , is a function \mathrm: \mathbb \to \mathbb defined as: \operatorname z = \frac\int_0^z e^\,\mathrm dt. The integral here is a complex Contour integrat ...
, which is related to the cumulative distribution function of a
normal distribution In probability theory and statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is f(x) = \frac ...
; another is the arctan function, which is related to the cumulative distribution function of a Cauchy distribution. A sigmoid function is constrained by a pair of horizontal asymptotes as x \rightarrow \pm \infty. A sigmoid function is
convex Convex or convexity may refer to: Science and technology * Convex lens, in optics Mathematics * Convex set, containing the whole line segment that joins points ** Convex polygon, a polygon which encloses a convex set of points ** Convex polytop ...
for values less than a particular point, and it is concave for values greater than that point: in many of the examples here, that point is 0.


Examples

*
Logistic function A logistic function or logistic curve is a common S-shaped curve ( sigmoid curve) with the equation f(x) = \frac where The logistic function has domain the real numbers, the limit as x \to -\infty is 0, and the limit as x \to +\infty is L. ...
f(x) = \frac * Hyperbolic tangent (shifted and scaled version of the logistic function, above) f(x) = \tanh x = \frac * Arctangent function f(x) = \arctan x * Gudermannian function f(x) = \operatorname(x) = \int_0^x \frac = 2\arctan\left(\tanh\left(\frac\right)\right) *
Error function In mathematics, the error function (also called the Gauss error function), often denoted by , is a function \mathrm: \mathbb \to \mathbb defined as: \operatorname z = \frac\int_0^z e^\,\mathrm dt. The integral here is a complex Contour integrat ...
f(x) = \operatorname(x) = \frac \int_0^x e^ \, dt * Generalised logistic function f(x) = \left(1 + e^ \right)^, \quad \alpha > 0 * Smoothstep function f(x) = \begin , & , x, \le 1 \\ \\ \sgn(x) & , x, \ge 1 \\ \end \quad N \in \mathbb \ge 1 * Some algebraic functions, for example f(x) = \frac * and in a more general form f(x) = \frac * Up to shifts and scaling, many sigmoids are special cases of f(x) = \varphi(\varphi(x, \beta), \alpha) , where \varphi(x, \lambda) = \begin (1 - \lambda x)^ & \lambda \ne 0 \\e^ & \lambda = 0 \\ \end is the inverse of the negative Box–Cox transformation, and \alpha < 1 and \beta < 1 are shape parameters. * Smooth transition function normalized to (−1,1): \beginf(x) &= \begin , & , x, < 1 \\ \\ \sgn(x) & , x, \ge 1 \\ \end \\ &= \begin , & , x, < 1 \\ \\ \sgn(x) & , x, \ge 1 \\ \end\end using the hyperbolic tangent mentioned above. Here, m is a free parameter encoding the slope at x=0, which must be greater than or equal to \sqrt because any smaller value will result in a function with multiple inflection points, which is therefore not a true sigmoid. This function is unusual because it actually attains the limiting values of −1 and 1 within a finite range, meaning that its value is constant at −1 for all x \leq -1 and at 1 for all x \geq 1. Nonetheless, it is smooth (infinitely differentiable, C^\infty) ''everywhere'', including at x = \pm 1.


Applications

Many natural processes, such as those of complex system
learning curve A learning curve is a graphical representation of the relationship between how proficient people are at a task and the amount of experience they have. Proficiency (measured on the vertical axis) usually increases with increased experience (the ...
s, exhibit a progression from small beginnings that accelerates and approaches a climax over time. When a specific mathematical model is lacking, a sigmoid function is often used. The van Genuchten–Gupta model is based on an inverted S-curve and applied to the response of crop yield to
soil salinity Soil salinity is the salt (chemistry), salt content in the soil; the process of increasing the salt content is known as salinization (also called salination in American and British English spelling differences, American English). Salts occur nat ...
. Examples of the application of the logistic S-curve to the response of crop yield (wheat) to both the soil salinity and depth to
water table The water table is the upper surface of the phreatic zone or zone of saturation. The zone of saturation is where the pores and fractures of the ground are saturated with groundwater, which may be fresh, saline, or brackish, depending on the loc ...
in the soil are shown in modeling crop response in agriculture. In
artificial neural network In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a computational model inspired by the structure and functions of biological neural networks. A neural network consists of connected ...
s, sometimes non-smooth functions are used instead for efficiency; these are known as hard sigmoids. In
audio signal processing Audio signal processing is a subfield of signal processing that is concerned with the electronic manipulation of audio signals. Audio signals are electronic representations of sound waves—longitudinal waves which travel through air, consisting ...
, sigmoid functions are used as waveshaper
transfer function In engineering, a transfer function (also known as system function or network function) of a system, sub-system, or component is a function (mathematics), mathematical function that mathematical model, models the system's output for each possible ...
s to emulate the sound of analog circuitry clipping. In
biochemistry Biochemistry, or biological chemistry, is the study of chemical processes within and relating to living organisms. A sub-discipline of both chemistry and biology, biochemistry may be divided into three fields: structural biology, enzymology, a ...
and
pharmacology Pharmacology is the science of drugs and medications, including a substance's origin, composition, pharmacokinetics, pharmacodynamics, therapeutic use, and toxicology. More specifically, it is the study of the interactions that occur betwee ...
, the
Hill A hill is a landform that extends above the surrounding terrain. It often has a distinct summit, and is usually applied to peaks which are above elevation compared to the relative landmass, though not as prominent as Mountain, mountains. Hills ...
and Hill–Langmuir equations are sigmoid functions. In computer graphics and real-time rendering, some of the sigmoid functions are used to blend colors or geometry between two values, smoothly and without visible seams or discontinuities. Titration curves between strong acids and strong bases have a sigmoid shape due to the logarithmic nature of the pH scale. The logistic function can be calculated efficiently by utilizing type III Unums. An hierarchy of sigmoid growth models with increasing complexity (number of parameters) was built with the primary goal to re-analyze kinetic data, the so called N-t curves, from heterogeneous
nucleation In thermodynamics, nucleation is the first step in the formation of either a new Phase (matter), thermodynamic phase or Crystal structure, structure via self-assembly or self-organization within a substance or mixture. Nucleation is typically def ...
experiments, in
electrochemistry Electrochemistry is the branch of physical chemistry concerned with the relationship between Electric potential, electrical potential difference and identifiable chemical change. These reactions involve Electron, electrons moving via an electronic ...
. The hierarchy includes at present three models, with 1, 2 and 3 parameters, if not counting the maximal number of nuclei Nmax, respectively—a tanh2 based model called α21 originally devised to describe diffusion-limited crystal growth (not aggregation!) in 2D, the Johnson-Mehl-Avrami-Kolmogorov (JMAKn) model, and the Richards model. It was shown that for the concrete purpose even the simplest model works and thus it was implied that the experiments revisited are an example of two-step nucleation with the first step being the growth of the metastable phase in which the nuclei of the stable phase form.


See also

* * * * * * * * * * *


References


Further reading

* . (NB. In particular see "Chapter 4: Artificial Neural Networks" (in particular pp. 96–97) where Mitchell uses the word "logistic function" and the "sigmoid function" synonymously – this function he also calls the "squashing function" – and the sigmoid (aka logistic) function is used to compress the outputs of the "neurons" in multi-layer neural nets.) * (NB. Properties of the sigmoid, including how it can shift along axes and how its domain may be transformed.)


External links

* {{Artificial intelligence navbox Elementary special functions Artificial neural networks