In
mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
, more specifically
measure theory
In mathematics, the concept of a measure is a generalization and formalization of geometrical measures ( length, area, volume) and other common notions, such as mass and probability of events. These seemingly distinct concepts have many simil ...
, there are various notions of the convergence of measures. For an intuitive general sense of what is meant by ''convergence of measures'', consider a sequence of measures μ
''n'' on a space, sharing a common collection of measurable sets. Such a sequence might represent an attempt to construct 'better and better' approximations to a desired measure μ that is difficult to obtain directly. The meaning of 'better and better' is subject to all the usual caveats for taking
limit
Limit or Limits may refer to:
Arts and media
* ''Limit'' (manga), a manga by Keiko Suenobu
* ''Limit'' (film), a South Korean film
* Limit (music), a way to characterize harmony
* "Limit" (song), a 2016 single by Luna Sea
* "Limits", a 2019 ...
s; for any error tolerance ε > 0 we require there be ''N'' sufficiently large for ''n'' ≥ ''N'' to ensure the 'difference' between μ
''n'' and μ is smaller than ε. Various notions of convergence specify precisely what the word 'difference' should mean in that description; these notions are not equivalent to one another, and vary in strength.
Three of the most common notions of convergence are described below.
Informal descriptions
This section attempts to provide a rough intuitive description of three notions of convergence, using terminology developed in
calculus
Calculus, originally called infinitesimal calculus or "the calculus of infinitesimals", is the mathematical study of continuous change, in the same way that geometry is the study of shape, and algebra is the study of generalizations of arithm ...
courses; this section is necessarily imprecise as well as inexact, and the reader should refer to the formal clarifications in subsequent sections. In particular, the descriptions here do not address the possibility that the measure of some sets could be infinite, or that the underlying space could exhibit pathological behavior, and additional technical assumptions are needed for some of the statements. The statements in this section are however all correct if
is a sequence of probability measures on a
Polish space
In the mathematical discipline of general topology, a Polish space is a separable completely metrizable topological space; that is, a space homeomorphic to a complete metric space that has a countable dense subset. Polish spaces are so named bec ...
.
The various notions of convergence formalize the assertion that the 'average value' of each 'sufficiently nice' function should converge:
To formalize this requires a careful specification of the set of functions under consideration and how uniform the convergence should be.
The notion of ''weak convergence'' requires this convergence to take place for every continuous bounded function
.
This notion treats convergence for different functions ''f'' independently of one another, i.e., different functions ''f'' may require different values of ''N'' ≤ ''n'' to be approximated equally well (thus, convergence is non-uniform in
).
The notion of ''setwise convergence'' formalizes the assertion that the measure of each measurable set should converge:
Again, no uniformity over the set
is required.
Intuitively, considering integrals of 'nice' functions, this notion provides more uniformity than weak convergence. As a matter of fact, when considering sequences of measures with uniformly bounded
variation on a
Polish space
In the mathematical discipline of general topology, a Polish space is a separable completely metrizable topological space; that is, a space homeomorphic to a complete metric space that has a countable dense subset. Polish spaces are so named bec ...
, setwise convergence implies the convergence
for any bounded measurable function
.
As before, this convergence is non-uniform in
The notion of ''total variation convergence'' formalizes the assertion that the measure of all measurable sets should converge ''uniformly'', i.e. for every
there exists ''N'' such that
for every ''n > N'' and for every measurable set
. As before, this implies convergence of integrals against bounded measurable functions, but this time convergence is uniform over all functions bounded by any fixed constant.
Total variation convergence of measures
This is the strongest notion of convergence shown on this page and is defined as follows. Let
be a
measurable space
In mathematics, a measurable space or Borel space is a basic object in measure theory. It consists of a set and a σ-algebra, which defines the subsets that will be measured.
Definition
Consider a set X and a σ-algebra \mathcal A on X. Then the ...
. The
total variation
In mathematics, the total variation identifies several slightly different concepts, related to the (local or global) structure of the codomain of a function or a measure. For a real-valued continuous function ''f'', defined on an interval 'a'' ...
distance between two (positive) measures μ and ν is then given by
:
Here the supremum is taken over ''f'' ranging over the set of all
measurable function
In mathematics and in particular measure theory, a measurable function is a function between the underlying sets of two measurable spaces that preserves the structure of the spaces: the preimage of any measurable set is measurable. This is in di ...
s from ''X'' to
1, 1 This is in contrast, for example, to the
Wasserstein metric
In mathematics, the Leonid Vaseršteĭn, Wasserstein distance or Leonid Kantorovich, Kantorovich–Gennadii Rubinstein, Rubinstein metric is a metric (mathematics), distance function defined between Probability distribution, probability distributi ...
, where the definition is of the same form, but the supremum is taken over ''f'' ranging over the set of measurable functions from ''X'' to
1, 1which have
Lipschitz constant
In mathematical analysis, Lipschitz continuity, named after German mathematician Rudolf Lipschitz, is a strong form of uniform continuity for functions. Intuitively, a Lipschitz continuous function is limited in how fast it can change: there ex ...
at most 1; and also in contrast to the
Radon metric, where the supremum is taken over ''f'' ranging over the set of continuous functions from ''X'' to
1, 1 In the case where ''X'' is a
Polish space
In the mathematical discipline of general topology, a Polish space is a separable completely metrizable topological space; that is, a space homeomorphic to a complete metric space that has a countable dense subset. Polish spaces are so named bec ...
, the total variation metric coincides with the Radon metric.
If μ and ν are both
probability measure
In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies measure properties such as ''countable additivity''. The difference between a probability measure and the more gener ...
s, then the total variation distance is also given by
:
The equivalence between these two definitions can be seen as a particular case of the
Monge-Kantorovich duality. From the two definitions above, it is clear that the total variation distance between probability measures is always between 0 and 2.
To illustrate the meaning of the total variation distance, consider the following thought experiment. Assume that we are given two probability measures μ and ν, as well as a random variable ''X''. We know that ''X'' has law either μ or ν but we do not know which one of the two. Assume that these two measures have prior probabilities 0.5 each of being the true law of ''X''. Assume now that we are given ''one'' single sample distributed according to the law of ''X'' and that we are then asked to guess which one of the two distributions describes that law. The quantity
:
then provides a sharp upper bound on the prior probability that our guess will be correct.
Given the above definition of total variation distance, a sequence μ
''n'' of measures defined on the same measure space is said to converge to a measure ''μ'' in total variation distance if for every ''ε'' > 0, there exists an ''N'' such that for all ''n'' > ''N'', one has that
:
Setwise convergence of measures
For
a
measurable space
In mathematics, a measurable space or Borel space is a basic object in measure theory. It consists of a set and a σ-algebra, which defines the subsets that will be measured.
Definition
Consider a set X and a σ-algebra \mathcal A on X. Then the ...
, a sequence μ
''n'' is said to converge setwise to a limit ''μ'' if
:
for every set
.
Typical arrow notations are
and
.
For example, as a consequence of the
Riemann–Lebesgue lemma In mathematics, the Riemann–Lebesgue lemma, named after Bernhard Riemann and Henri Lebesgue, states that the Fourier transform or Laplace transform of an ''L''1 function vanishes at infinity. It is of importance in harmonic analysis and asymptot ...
, the sequence μ
''n'' of measures on the interval
1, 1given by μ
''n''(''dx'') = (1+ sin(''nx''))''dx'' converges setwise to Lebesgue measure, but it does not converge in total variation.
In a measure theoretical or probabilistic context setwise convergence is often referred to as strong convergence (as opposed to weak convergence). This can lead to some ambiguity because in functional analysis, strong convergence usually refers to convergence with respect to a norm.
Weak convergence of measures
In
mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
and
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, weak convergence is one of many types of convergence relating to the convergence of
measures
Measure may refer to:
* Measurement, the assignment of a number to a characteristic of an object or event
Law
* Ballot measure, proposed legislation in the United States
* Church of England Measure, legislation of the Church of England
* Measu ...
. It depends on a topology on the underlying space and thus is not a purely measure theoretic notion.
There are several equivalent
definition
A definition is a statement of the meaning of a term (a word, phrase, or other set of symbols). Definitions can be classified into two large categories: intensional definitions (which try to give the sense of a term), and extensional definitio ...
s of weak convergence of a sequence of measures, some of which are (apparently) more general than others. The equivalence of these conditions is sometimes known as the Portmanteau theorem.
Definition. Let
be a
metric space
In mathematics, a metric space is a set together with a notion of ''distance'' between its elements, usually called points. The distance is measured by a function called a metric or distance function. Metric spaces are the most general settin ...
with its
Borel -algebra . A bounded sequence of positive
probability measure
In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies measure properties such as ''countable additivity''. The difference between a probability measure and the more gener ...
s
on
is said to converge weakly to a probability measure
(denoted
) if any of the following equivalent conditions is true (here
denotes expectation or the
norm with respect to
, while
denotes expectation or the
norm with respect to
):
*