Kolmogorov–Sinai Entropy

In mathematics, a measure-preserving dynamical system is an object of study in the abstract formulation of dynamical systems, and ergodic theory in particular. Measure-preserving systems obey the Poincaré recurrence theorem, and are a special case of conservative systems. They provide the formal, mathematical basis for a broad range of physical systems, and, in particular, many systems from classical mechanics (in particular, most non-dissipative systems) as well as systems in thermodynamic equilibrium.

Definition

A measure-preserving dynamical system is defined as a probability space and a measure-preserving transformation on it. In more detail, it is a system

:$(X, \mathcal, \mu, T)$

with the following structure:

*$X$ is a set,
*$\mathcal B$ is a σ-algebra over $X$,
*\mu:\mathcal\rightarrow ,1/math> is a probability measure, so that $\mu (X) = 1$, and $\mu(\varnothing) = 0$,
*$T:X \rightarrow X$ is a measurable transformation which preserves the measure $\mu$, i.e., $\forall A\in \mathcal\;\; \mu(T^(A))=\mu(A)$.

Discussion

One may ask why the measure preserving transformation is defined in terms of the inverse $\mu(T^(A))=\mu(A)$ instead of the forward transformation $\mu(T(A))=\mu(A)$. This can be understood in a fairly easy fashion. Consider a mapping $\mathcal$ of power sets:
:$\mathcal:P(X)\to P(X)$
Consider now the special case of maps $\mathcal$ which preserve intersections, unions and complements (so that it is a map of Borel sets) and also sends $X$ to $X$ (because we want it to be conservative). Every such conservative, Borel-preserving map can be specified by some surjective map $T:X\to X$ by writing $\mathcal(A)=T^(A)$. Of course, one could also define $\mathcal(A)=T(A)$, but this is not enough to specify all such possible maps $\mathcal$. That is, conservative, Borel-preserving maps $\mathcal$ cannot, in general, be written in the form $\mathcal(A)=T(A);$ one might consider, for example, the map of the unit interval T: ,1) \to [0,1) given by $x \mapsto 2x\mod 1;$ this is the pushforward, whereas $\mu(T(A))$ is generically called a pullback">Bernoulli map.

Note that $\mu(T^(A))$ has the form of a Pushforward measure">pushforward, whereas $\mu(T(A))$ is generically called a pullback. Almost all properties and behaviors of dynamical systems are defined in terms of the pushforward. For example, the transfer operator is defined in terms of the pushforward of the transformation map $T$; the measure $\mu$ can now be understood as an invariant measure; it is just the Perron–Frobenius theorem, Frobenius–Perron eigenvector of the transfer operator (recall, the FP eigenvector is the largest eigenvector of a matrix; in this case it is the eigenvector which has the eigenvalue one: the invariant measure.)

There are two classification problems of interest. One, discussed below, fixes $(X, \mathcal, \mu)$ and asks about the isomorphism classes of a transformation map $T$. The other, discussed in transfer operator, fixes $(X, \mathcal)$ and $T$, and asks about maps $\mu$ that are measure-like. Measure-like, in that they preserve the Borel properties, but are no longer invariant; they are in general dissipative and so give insights into dissipative systems and the route to equilibrium.

In terms of physics, the measure-preserving dynamical system $(X, \mathcal, \mu, T)$ often describes a physical system that is in equilibrium, for example, thermodynamic equilibrium. One might ask: how did it get that way? Often, the answer is by stirring, mixing, turbulence, thermalization or other such processes. If a transformation map $T$ describes this stirring, mixing, etc. then the system $(X, \mathcal, \mu, T)$ is all that is left, after all of the transient modes have decayed away. The transient modes are precisely those eigenvectors of the transfer operator that have eigenvalue less than one; the invariant measure $\mu$ is the one mode that does not decay away. The rate of decay of the transient modes are given by (the logarithm of) their eigenvalues; the eigenvalue one corresponds to infinite half-life.

Informal example

The microcanonical ensemble from physics provides an informal example. Consider, for example, a fluid, gas or plasma in a box of width, length and height $w\times l\times h,$ consisting of $N$ atoms. A single atom in that box might be anywhere, having arbitrary velocity; it would be represented by a single point in $w\times l\times h\times \mathbb^3.$ A given collection of $N$ atoms would then be a ''single point'' somewhere in the space $(w\times l\times h)^N \times \mathbb^.$ The "ensemble" is the collection of all such points, that is, the collection of all such possible boxes (of which there are an uncountably-infinite number). This ensemble of all-possible-boxes is the space $X$ above.

In the case of an ideal gas, the measure $\mu$ is given by the Maxwell–Boltzmann distribution. It is a product measure, in that if $p_i(x,y,z,v_x,v_y,v_z)\,d^3x\,d^3p$ is the probability of atom $i$ having position and velocity $x,y,z,v_x,v_y,v_z$, then, for $N$ atoms, the probability is the product of $N$ of these. This measure is understood to apply to the ensemble. So, for example, one of the possible boxes in the ensemble has all of the atoms on one side of the box. One can compute the likelihood of this, in the Maxwell–Boltzmann measure. It will be enormously tiny, of order $\mathcal\left(2^\right).$ Of all possible boxes in the ensemble, this is a ridiculously small fraction.

The only reason that this is an "informal example" is because writing down the transition function $T$ is difficult, and, even if written down, it is hard to perform practical computations with it. Difficulties are compounded if the interaction is not an ideal-gas billiard-ball type interaction, but is instead a van der Waals interaction, or some other interaction suitable for a liquid or a plasma; in such cases, the invariant measure is no longer the Maxwell–Boltzmann distribution. The art of physics is finding reasonable approximations.

This system does exhibit one key idea from the classification of measure-preserving dynamical systems: two ensembles, having different temperatures, are inequivalent. The entropy for a given canonical ensemble depends on its temperature; as physical systems, it is "obvious" that when the temperatures differ, so do the systems. This holds in general: systems with different entropy are not isomorphic.

Examples

Unlike the informal example above, the examples below are sufficiently well-defined and tractable that explicit, formal computations can be performed.

* μ could be the normalized angle measure dθ/2π on the unit circle, and ''T'' a rotation. See equidistribution theorem;
* the Bernoulli scheme;
* the interval exchange transformation;
* with the definition of an appropriate measure, a subshift of finite type;
* the base flow of a random dynamical system;
* the flow of a Hamiltonian vector field on the tangent bundle of a closed connected smooth manifold is measure-preserving (using the measure induced on the Borel sets by the symplectic volume form) by Liouville's theorem (Hamiltonian);
* for certain maps and Markov processes, the Krylov–Bogolyubov theorem establishes the existence of a suitable measure to form a measure-preserving dynamical system.

Generalization to groups and monoids

The definition of a measure-preserving dynamical system can be generalized to the case in which ''T'' is not a single transformation that is iterated to give the dynamics of the system, but instead is a monoid (or even a group, in which case we have the action of a group upon the given probability space) of transformations ''T_s'' : ''X'' → ''X'' parametrized by ''s'' ∈ Z (or R, or N ∪ , or [0, +∞)), where each transformation ''T_s'' satisfies the same requirements as ''T'' above. In particular, the transformations obey the rules:
* $T_0 = \mathrm_X :X \rightarrow X$, the identity function on ''X'';
* $T_ \circ T_ = T_$, whenever all the terms are well-defined;
* $T_^ = T_$, whenever all the terms are well-defined.

The earlier, simpler case fits into this framework by defining ''T_s'' = ''T^s'' for ''s'' ∈ N.

Homomorphisms

The concept of a homomorphism and an isomorphism may be defined.

Consider two dynamical systems $(X, \mathcal, \mu, T)$ and $(Y, \mathcal, \nu, S)$. Then a mapping

:$\varphi:X \to Y$

is a homomorphism of dynamical systems if it satisfies the following three properties:

# The map $\varphi\$ is measurable.
# For each $B \in \mathcal$, one has $\mu (\varphi^B) = \nu(B)$.
# For $\mu$-almost all $x \in X$, one has $\varphi(Tx) = S(\varphi x)$.

The system $(Y, \mathcal, \nu, S)$ is then called a factor of $(X, \mathcal, \mu, T)$.

The map $\varphi\;$ is an isomorphism of dynamical systems if, in addition, there exists another mapping

:$\psi:Y \to X$

that is also a homomorphism, which satisfies

# for $\mu$-almost all $x \in X$, one has $x = \psi(\varphi x)$;
# for $\nu$-almost all $y \in Y$, one has $y = \varphi(\psi y)$.

Hence, one may form a category of dynamical systems and their homomorphisms.

Generic points

A point ''x'' ∈ ''X'' is called a generic point if the orbit of the point is distributed uniformly according to the measure.

Symbolic names and generators

Consider a dynamical system $(X, \mathcal, T, \mu)$, and let ''Q'' = be a partition of ''X'' into ''k'' measurable pair-wise disjoint pieces. Given a point ''x'' ∈ ''X'', clearly ''x'' belongs to only one of the ''Q_i''. Similarly, the iterated point ''Tⁿx'' can belong to only one of the parts as well. The symbolic name of ''x'', with regards to the partition ''Q'', is the sequence of integers such that

:$T^nx \in Q_.$

The set of symbolic names with respect to a partition is called the symbolic dynamics of the dynamical system. A partition ''Q'' is called a generator or generating partition if μ-almost every point ''x'' has a unique symbolic name.

Operations on partitions

Given a partition Q = and a dynamical system $(X, \mathcal, T, \mu)$, define the ''T''-pullback of ''Q'' as

:$T^Q = \.$

Further, given two partitions ''Q'' = and ''R'' = , define their refinement as

:$Q \vee R = \.$

With these two constructs, the ''refinement of an iterated pullback'' is defined as

:$\begin \bigvee_^N T^Q & = \ \\ \end$

which plays crucial role in the construction of the measure-theoretic entropy of a dynamical system.

Measure-theoretic entropy

The entropy of a partition $\mathcal$ is defined as

:$H(\mathcal)=-\sum_\mu (Q) \log \mu(Q).$

The measure-theoretic entropy of a dynamical system $(X, \mathcal, T, \mu)$ with respect to a partition ''Q'' = is then defined as

:$h_\mu(T,\mathcal) = \lim_ \frac H\left(\bigvee_^N T^\mathcal\right).$

Finally, the Kolmogorov–Sinai metric or measure-theoretic entropy of a dynamical system $(X, \mathcal,T,\mu)$ is defined as

:$h_\mu(T) = \sup_Q h_\mu(T,Q).$

where the supremum is taken over all finite measurable partitions. A theorem of Yakov Sinai in 1959 shows that the supremum is actually obtained on partitions that are generators. Thus, for example, the entropy of the Bernoulli process is log 2, since almost every real number has a unique binary expansion. That is, one may partition the unit interval into the intervals /nowiki>0, 1/2) and [1/2, 1 Every real number ''x'' is either less than 1/2 or not; and likewise so is the fractional part of 2^''n''''x''.

If the space ''X'' is compact and endowed with a topology, or is a metric space, then the topological entropy may also be defined.

Classification and anti-classification theorems

One of the primary activities in the study of measure-preserving systems is their classification according to their properties. That is, let $(X, \mathcal, \mu)$ be a measure space, and let $U$ be the set of all measure preserving systems $(X, \mathcal, \mu, T)$. An isomorphism $S\sim T$ of two transformations $S, T$ defines an equivalence relation $\mathcal\subset U\times U.$ The goal is then to describe the relation $\mathcal$. A number of classification theorems have been obtained; but quite interestingly, a number of anti-classification theorems have been found as well. The anti-classification theorems state that there are more than a countable number of isomorphism classes, and that a countable amount of information is not sufficient to classify isomorphisms.

The first anti-classification theorem, due to Hjorth, states that if $U$ is endowed with the weak topology, then the set $\mathcal$ is not a Borel set. There are a variety of other anti-classification results. For example, replacing isomorphism with Kakutani equivalence, it can be shown that there are uncountably many non-Kakutani equivalent ergodic measure-preserving transformations of each entropy type.

These stand in contrast to the classification theorems. These include:
* Ergodic measure-preserving transformations with a pure point spectrum have been classified.
* Bernoulli shifts are classified by their metric entropy. See Ornstein theory for more.

See also

* Krylov–Bogolyubov theorem on the existence of invariant measures
*Poincaré recurrence theorem

References

Further reading

* Michael S. Keane, "Ergodic theory and subshifts of finite type", (1991), appearing as Chapter 2 in ''Ergodic Theory, Symbolic Dynamics and Hyperbolic Spaces'', Tim Bedford, Michael Keane and Caroline Series, Eds. Oxford University Press, Oxford (1991). ''(Provides expository introduction, with exercises, and extensive references.)''
* Lai-Sang Young, "Entropy in Dynamical Systems"
pdfps
, appearing as Chapter 16 in ''Entropy'', Andreas Greven, Gerhard Keller, and Gerald Warnecke, eds. Princeton University Press, Princeton, NJ (2003). {{isbn, 0-691-11338-6
* T. Schürmann and I. Hoffmann, ''The entropy of strange billiards inside n-simplexes.'' J. Phys. A 28(17), page 5033, 1995
PDF-Document
''(gives a more involved example of measure-preserving dynamical system.)''

Dynamical systems
Entropy and information
Measure theory
Entropy
Information theory