In
information theory, the principle of minimum Fisher information (MFI) is a
variational principle
In science and especially in mathematical studies, a variational principle is one that enables a problem to be solved using calculus of variations, which concerns finding functions that optimize the values of quantities that depend on those funct ...
which, when applied with the proper constraints needed to reproduce empirically known expectation values, determines the best
probability distribution that characterizes the system. (See also
Fisher information
In mathematical statistics, the Fisher information (sometimes simply called information) is a way of measuring the amount of information that an observable random variable ''X'' carries about an unknown parameter ''θ'' of a distribution that model ...
.)
Measures of information
Information measures (IM) are the most important tools of
information theory. They measure either the amount of positive information or of "missing" information an observer possesses with regards to any system of interest. The most famous IM is the so-called
Shannon-entropy (1948), which determines how much additional information the observer still requires in order to have all the available knowledge regarding a given system S, when all he/she has is a
probability density function
In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) ca ...
(PDF) defined on appropriate elements of such system. This is then a "missing" information measure. The IM is a function of the PDF only. If the observer does not have such
a PDF, but only a finite set of empirically determined mean values of the system, then a fundamental scientific principle called the
Maximum Entropy one (MaxEnt) asserts that the "best" PDF is the one that, reproducing the known expectation values, maximizes otherwise Shannon's IM.
Fisher's information measure
Fisher's information (FIM), named after
Ronald Fisher
Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who ...
, (1925) is another kind of measure, in two respects, namely,
1) it reflects the amount of (positive) information of the observer,
2) it depends not only on the PD but also on its first derivatives, a property that makes it a local quantity (Shannon's is instead a global one).
The corresponding counterpart of MaxEnt is now the FIM-minimization, since Fisher's measure grows when Shannon's diminishes, and vice versa. The minimization here referred to (MFI) is an important theoretical tool in a manifold of disciplines, beginning with
physics
Physics is the natural science that studies matter, its fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force. "Physical science is that department of knowledge which r ...
. In a sense it is clearly superior to MaxEnt because the later procedure yields always as the solution an exponential PD, while the MFI solution is a
differential equation
In mathematics, a differential equation is an equation that relates one or more unknown functions and their derivatives. In applications, the functions generally represent physical quantities, the derivatives represent their rates of change, an ...
for the PD, which allows for greater flexibility and versatility.
Applications of the MFI
Thermodynamics
Much effort has been devoted to Fisher's information measure, shedding much light upon the manifold physical applications. As a small sample, it can be shown that the whole field of
thermodynamics
Thermodynamics is a branch of physics that deals with heat, work, and temperature, and their relation to energy, entropy, and the physical properties of matter and radiation. The behavior of these quantities is governed by the four laws of th ...
(both
equilibrium and
non-equilibrium
Non-equilibrium thermodynamics is a branch of thermodynamics that deals with physical systems that are not in thermodynamic equilibrium but can be described in terms of macroscopic quantities (non-equilibrium state variables) that represent an ext ...
) can be derived from the MFI approach. Here FIM is specialized to the particular but important case of translation families, i.e., distribution functions whose form does not change under translational transformations. In this case, Fisher measure becomes shift-invariant. Such minimizing of Fisher's measure leads to a
Schrödinger-like equation for the probability amplitude, where the ground state describes equilibrium physics and the excited states account for non-equilibrium situations.
Scale-invariant phenomena
More recently,
Zipf's law has been shown to arise as the variational solution of the MFI when
scale invariance
In physics, mathematics and statistics, scale invariance is a feature of objects or laws that do not change if scales of length, energy, or other variables, are multiplied by a common factor, and thus represent a universality.
The technical term ...
is introduced in the measure, leading for the first time an explanation of this regularity from
first principle
In philosophy and science, a first principle is a basic proposition or assumption that cannot be deduced from any other proposition or assumption.
First principles in philosophy are from First Cause attitudes and taught by Aristotelians, and nua ...
s. It has been also shown that MFI can be used to formulate a thermodynamics based on scale invariance instead of
translational invariance
In geometry, to translate a geometric figure is to move it from one place to another without rotating it. A translation "slides" a thing by .
In physics and mathematics, continuous translational symmetry is the invariance of a system of equat ...
, allowing the definition of the
scale-free ideal gas, the scale invariant equivalent of the
ideal gas
An ideal gas is a theoretical gas composed of many randomly moving point particles that are not subject to interparticle interactions. The ideal gas concept is useful because it obeys the ideal gas law, a simplified equation of state, and is a ...
.
[{{Cite journal , last1 = Hernando , first1 = A. , last2 = Vesperinas , first2 = C. , last3 = Plastino , first3 = A. , doi = 10.1016/j.physa.2009.09.054 , title = Fisher information and the thermodynamics of scale-invariant systems , journal = Physica A: Statistical Mechanics and its Applications , volume = 389 , issue = 3 , pages = 490 , year = 2010 , arxiv = 0908.0504 , bibcode = 2010PhyA..389..490H ]
References
Information theory