In
probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
and
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the Rademacher distribution (which is named after
Hans Rademacher
Hans Adolph Rademacher (; 3 April 1892, Wandsbeck, now Hamburg-Wandsbek – 7 February 1969, Haverford, Pennsylvania, USA) was a German-born American mathematician, known for work in mathematical analysis and number theory.
Biography
Rademacher r ...
) is a
discrete probability distribution
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
where a
random variate
In probability and statistics, a random variate or simply variate is a particular outcome of a ''random variable'': the random variates which are other outcomes of the same random variable might have different values (random numbers).
A random d ...
''X'' has a 50% chance of being +1 and a 50% chance of being -1.
A
series
Series may refer to:
People with the name
* Caroline Series (born 1951), English mathematician, daughter of George Series
* George Series (1920–1995), English physicist
Arts, entertainment, and media
Music
* Series, the ordered sets used in ...
(that is, a sum) of Rademacher distributed variables can be regarded as a simple symmetrical
random walk
In mathematics, a random walk is a random process that describes a path that consists of a succession of random steps on some mathematical space.
An elementary example of a random walk is the random walk on the integer number line \mathbb Z ...
where the step size is 1.
Mathematical formulation
The
probability mass function
In probability and statistics, a probability mass function is a function that gives the probability that a discrete random variable is exactly equal to some value. Sometimes it is also known as the discrete density function. The probability mass ...
of this distribution is
:
In terms of the
Dirac delta function
In mathematics, the Dirac delta distribution ( distribution), also known as the unit impulse, is a generalized function or distribution over the real numbers, whose value is zero everywhere except at zero, and whose integral over the entire ...
, as
:
Bounds on sums of independent Rademacher variables
There are various results in probability theory around analyzing the sum of
i.i.d.
In probability theory and statistics, a collection of random variables is independent and identically distributed if each random variable has the same probability distribution as the others and all are mutually independent. This property is us ...
Rademacher variables, including
concentration inequalities such as
Bernstein inequalities as well as
anti-concentration inequalities like Tomaszewski's conjecture.
Concentration inequalities
Let {''x
i''} be a set of random variables with a Rademacher distribution. Let {''a
i''} be a sequence of real numbers. Then
:
where , , ''a'', ,
2 is the
Euclidean norm
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean s ...
of the sequence {''a''
i}, ''t'' > 0 is a real number and Pr(''Z'') is the probability of event ''Z''.
Let ''Y'' = Σ ''x
ia
i'' and let ''Y'' be an almost surely convergent
series
Series may refer to:
People with the name
* Caroline Series (born 1951), English mathematician, daughter of George Series
* George Series (1920–1995), English physicist
Arts, entertainment, and media
Music
* Series, the ordered sets used in ...
in a
Banach space
In mathematics, more specifically in functional analysis, a Banach space (pronounced ) is a complete normed vector space. Thus, a Banach space is a vector space with a metric that allows the computation of vector length and distance between vector ...
. The for ''t'' > 0 and ''s'' ≥ 1 we have
:
for some constant ''c''.
Let ''p'' be a positive real number. Then the
Khintchine inequality In mathematics, the Khintchine inequality, named after Aleksandr Khinchin and spelled in multiple ways in the Latin alphabet, is a theorem from probability, and is also frequently used in analysis. Heuristically, it says that if we pick N complex ...
says that
:
where ''c''
1 and ''c''
2 are constants dependent only on ''p''.
For ''p'' ≥ 1,
Tomaszewski’s conjecture
In 1986, Bogusław Tomaszewski proposed a question about the distribution of the sum of independent Rademacher variables. A series of works on this question culminated into a proof in 2020 by Nathan Keller and Ohad Klein of the following conjecture.
Conjecture. Let
, where
and the
's are independent Rademacher variables. Then
:
For example, when
a_1 = a_2 = \cdots = a_n = 1/\sqrt{n}, one gets the following bound, first shown by Van Zuijlen.
:
\Pr \left( \left, \frac{ \sum_{ i = 1 }^n X_i } { \sqrt n } \ \le 1 \right) \ge 0.5.
The bound is sharp and better than that which can be derived from the normal distribution (approximately Pr > 0.31).
Applications
The Rademacher distribution has been used in
Bootstrapping (statistics), bootstrapping.
The Rademacher distribution can be used to show that
.
Random vectors with components sampled independently from the Rademacher distribution are useful for various
stochastic approximation
Stochastic approximation methods are a family of iterative methods typically used for root-finding problems or for optimization problems. The recursive update rules of stochastic approximation methods can be used, among other things, for solving l ...
s, for example:
* The
Hutchinson trace estimator Hutchinson may refer to:
Places United States
* Hutchinson, Kansas
* South Hutchinson, Kansas
* Hutchinson, Minnesota
* Hutchinson, Pennsylvania
* Hutchinson, West Virginia, in Logan County
* Hutchinson, Marion County, West Virginia
* Hutchi ...
,
which can be used to efficiently approximate the
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
of a
matrix
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** ''The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
of which the elements are not directly accessible, but rather implicitly defined via matrix-vector products.
*
SPSA, a computationally cheap, derivative-free, stochastic gradient approximation, useful for
numerical optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
.
Rademacher random variables are used in the
Symmetrization Inequality.
Related distributions
*
Bernoulli distribution
In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabil ...
: If ''X'' has a Rademacher distribution, then
\frac{X+1}{2} has a Bernoulli(1/2) distribution.
*
Laplace distribution
In probability theory and statistics, the Laplace distribution is a continuous probability distribution named after Pierre-Simon Laplace. It is also sometimes called the double exponential distribution, because it can be thought of as two exponen ...
: If ''X'' has a Rademacher distribution and ''Y'' ~ Exp(λ) is independent from ''X'', then ''XY'' ~ Laplace(0, 1/λ).
References
{{DEFAULTSORT:Rademacher Distribution
Discrete distributions
it:Distribuzione discreta uniforme#Altre distribuzioni>X, \leq 1\geq 1/2.
For example, when
a_1 = a_2 = \cdots = a_n = 1/\sqrt{n}, one gets the following bound, first shown by Van Zuijlen.
:
\Pr \left( \left, \frac{ \sum_{ i = 1 }^n X_i } { \sqrt n } \ \le 1 \right) \ge 0.5.
The bound is sharp and better than that which can be derived from the normal distribution (approximately Pr > 0.31).
Applications
The Rademacher distribution has been used in
Bootstrapping (statistics), bootstrapping.
The Rademacher distribution can be used to show that
.
Random vectors with components sampled independently from the Rademacher distribution are useful for various
stochastic approximation
Stochastic approximation methods are a family of iterative methods typically used for root-finding problems or for optimization problems. The recursive update rules of stochastic approximation methods can be used, among other things, for solving l ...
s, for example:
* The
Hutchinson trace estimator Hutchinson may refer to:
Places United States
* Hutchinson, Kansas
* South Hutchinson, Kansas
* Hutchinson, Minnesota
* Hutchinson, Pennsylvania
* Hutchinson, West Virginia, in Logan County
* Hutchinson, Marion County, West Virginia
* Hutchi ...
,
which can be used to efficiently approximate the
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
of a
matrix
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** ''The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
of which the elements are not directly accessible, but rather implicitly defined via matrix-vector products.
*
SPSA, a computationally cheap, derivative-free, stochastic gradient approximation, useful for
numerical optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
.
Rademacher random variables are used in the
Symmetrization Inequality.
Related distributions
*
Bernoulli distribution
In probability theory and statistics, the Bernoulli distribution, named after Swiss mathematician Jacob Bernoulli,James Victor Uspensky: ''Introduction to Mathematical Probability'', McGraw-Hill, New York 1937, page 45 is the discrete probabil ...
: If ''X'' has a Rademacher distribution, then
\frac{X+1}{2} has a Bernoulli(1/2) distribution.
*
Laplace distribution
In probability theory and statistics, the Laplace distribution is a continuous probability distribution named after Pierre-Simon Laplace. It is also sometimes called the double exponential distribution, because it can be thought of as two exponen ...
: If ''X'' has a Rademacher distribution and ''Y'' ~ Exp(λ) is independent from ''X'', then ''XY'' ~ Laplace(0, 1/λ).
References
{{DEFAULTSORT:Rademacher Distribution
Discrete distributions
it:Distribuzione discreta uniforme#Altre distribuzioni