HOME

TheInfoList



OR:

Information geometry is an interdisciplinary field that applies the techniques of
differential geometry Differential geometry is a mathematical discipline that studies the geometry of smooth shapes and smooth spaces, otherwise known as smooth manifolds. It uses the techniques of differential calculus, integral calculus, linear algebra and mult ...
to study
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set ...
and
statistics Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...
. It studies statistical manifolds, which are
Riemannian manifold In differential geometry, a Riemannian manifold or Riemannian space , so called after the German mathematician Bernhard Riemann, is a real, smooth manifold ''M'' equipped with a positive-definite inner product ''g'p'' on the tangent space ...
s whose points correspond to
probability distributions In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon ...
.


Introduction

Historically, information geometry can be traced back to the work of
C. R. Rao Calyampudi Radhakrishna Rao FRS (born 10 September 1920), commonly known as C. R. Rao, is an Indian-American mathematician and statistician. He is currently professor emeritus at Pennsylvania State University and Research Professor at the ...
, who was the first to treat the
Fisher matrix In mathematical statistics, the Fisher information (sometimes simply called information) is a way of measuring the amount of information that an observable random variable ''X'' carries about an unknown parameter ''θ'' of a distribution that model ...
as a
Riemannian metric In differential geometry, a Riemannian manifold or Riemannian space , so called after the German mathematician Bernhard Riemann, is a real, smooth manifold ''M'' equipped with a positive-definite inner product ''g'p'' on the tangent space '' ...
. The modern theory is largely due to
Shun'ichi Amari , is a Japanese scholar born in 1936 in Tokyo, Japan. Overviews He majored in Mathematical Engineering in 1958 from the University of Tokyo then graduated in 1963 from the Graduate School of the University of Tokyo. His M. Eng. in 1960 was e ...
, whose work has been greatly influential on the development of the field. Classically, information geometry considered a parametrized
statistical model A statistical model is a mathematical model that embodies a set of statistical assumptions concerning the generation of sample data (and similar data from a larger population). A statistical model represents, often in considerably idealized form ...
as a
Riemannian manifold In differential geometry, a Riemannian manifold or Riemannian space , so called after the German mathematician Bernhard Riemann, is a real, smooth manifold ''M'' equipped with a positive-definite inner product ''g'p'' on the tangent space ...
. For such models, there is a natural choice of Riemannian metric, known as the
Fisher information metric In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, ''i.e.'', a smooth manifold whose points are probability measures defined on a common probability spa ...
. In the special case that the statistical model is an
exponential family In probability and statistics, an exponential family is a parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, including the enabling of the user to calculate ...
, it is possible to induce the statistical manifold with a Hessian metric (i.e a Riemannian metric given by the potential of a convex function). In this case, the manifold naturally inherits two flat
affine connection In differential geometry, an affine connection is a geometric object on a smooth manifold which ''connects'' nearby tangent spaces, so it permits tangent vector fields to be differentiated as if they were functions on the manifold with values i ...
s, as well as a canonical Bregman divergence. Historically, much of the work was devoted to studying the associated geometry of these examples. In the modern setting, information geometry applies to a much wider context, including non-exponential families,
nonparametric statistics Nonparametric statistics is the branch of statistics that is not based solely on parametrized families of probability distributions (common examples of parameters are the mean and variance). Nonparametric statistics is based on either being distri ...
, and even abstract statistical manifolds not induced from a known statistical model. The results combine techniques from
information theory Information theory is the scientific study of the quantification, storage, and communication of information. The field was originally established by the works of Harry Nyquist and Ralph Hartley, in the 1920s, and Claude Shannon in the 1940s. ...
,
affine differential geometry Affine differential geometry is a type of differential geometry which studies invariants of volume-preserving affine transformations. The name ''affine differential geometry'' follows from Klein's Erlangen program. The basic difference between aff ...
,
convex analysis Convex analysis is the branch of mathematics devoted to the study of properties of convex functions and convex sets, often with applications in convex minimization, a subdomain of optimization theory. Convex sets A subset C \subseteq X of som ...
and many other fields. The standard references in the field are Shun’ichi Amari and Hiroshi Nagaoka's book, ''Methods of Information Geometry'', and the more recent book by Nihat Ay and others. A gentle introduction is given in the survey by Frank Nielsen. In 2018, the journal ''Information Geometry'' was released, which is devoted to the field.


Contributors

The history of information geometry is associated with the discoveries of at least the following people, and many others. *
Ronald Fisher Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who ...
*
Harald Cramér Harald Cramér (; 25 September 1893 – 5 October 1985) was a Swedish mathematician, actuary, and statistician, specializing in mathematical statistics and probabilistic number theory. John Kingman described him as "one of the giants of stat ...
* Calyampudi Radhakrishna Rao *
Harold Jeffreys Sir Harold Jeffreys, FRS (22 April 1891 – 18 March 1989) was a British mathematician, statistician, geophysicist, and astronomer. His book, ''Theory of Probability'', which was first published in 1939, played an important role in the revival ...
* Solomon Kullback *
Jean-Louis Koszul Jean-Louis Koszul (; January 3, 1921 – January 12, 2018) was a French mathematician, best known for studying geometry and discovering the Koszul complex. He was a second generation member of Bourbaki. Biography Koszul was educated at the in ...
* Richard Leibler *
Claude Shannon Claude Elwood Shannon (April 30, 1916 – February 24, 2001) was an American mathematician, electrical engineer, and cryptographer known as a "father of information theory". As a 21-year-old master's degree student at the Massachusetts I ...
* Imre Csiszár * N. N. Cencov (also written as Chentsov) *
Bradley Efron Bradley Efron (; born May 24, 1938) is an American statistician. Efron has been president of the American Statistical Association (2004) and of the Institute of Mathematical Statistics (1987–1988).Cochran, J. (1 September 2015), "ASA Lead ...
*
Shun'ichi Amari , is a Japanese scholar born in 1936 in Tokyo, Japan. Overviews He majored in Mathematical Engineering in 1958 from the University of Tokyo then graduated in 1963 from the Graduate School of the University of Tokyo. His M. Eng. in 1960 was e ...
* Ole Barndorff-Nielsen * Frank Nielsen * Damiano Brigo *
A. W. F. Edwards Anthony William Fairbank Edwards, FRS (born 1935) is a British statistician, geneticist and evolutionary biologist. He is the son of the surgeon Harold C. Edwards, and brother of medical geneticist John H. Edwards. He has sometimes been called ...
* Grant Hillier * Kees Jan Van Garderen


Applications

As an interdisciplinary field, information geometry has been used in various applications. Here an incomplete list: * Statistical inference * Time series and linear systems * Quantum systems * Neural networks * Machine learning * Statistical mechanics * Biology * Statistics * Mathematical finance


See also

*
Ruppeiner geometry Ruppeiner geometry is thermodynamic geometry (a type of information geometry) using the language of Riemannian geometry to study thermodynamics. George Ruppeiner proposed it in 1979. He claimed that thermodynamic systems can be represented by Riem ...
*
Kullback–Leibler divergence In mathematical statistics, the Kullback–Leibler divergence (also called relative entropy and I-divergence), denoted D_\text(P \parallel Q), is a type of statistical distance: a measure of how one probability distribution ''P'' is different fr ...
* Stochastic geometry


References


External links



Information Geometry journal by Springer
Information Geometry
overview by Cosma Rohilla Shalizi, July 2010
Information Geometry
notes by John C. Baez, John Baez, November 2012
Information geometry for neural networks(pdf )
by Daniel Wagenaar {{Differentiable computing