TheInfoList

Probability theory is the branch of
mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and their changes (cal ...
concerned with
probability Probability is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained ... . Although there are several different
probability interpretations The word probability has been used in a variety of ways since it was first applied to the mathematical study of games of chance. Does probability measure the real, physical, tendency of something to occur, or is it a measure of how strongly one bel ...
, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of
axioms An axiom, postulate or assumption is a statement that is taken to be truth, true, to serve as a premise or starting point for further reasoning and arguments. The word comes from the Greek ''axíōma'' () 'that which is thought worthy or fit' or ...
. Typically these axioms formalise probability in terms of a
probability space In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expres ...
, which assigns a measure taking values between 0 and 1, termed the
probability measure In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies Measure (mathematics), measure properties such as ''countable additivity''. The difference between a probability meas ...
, to a set of outcomes called the
sample space In probability theory Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ... . Any specified subset of the sample space is called an
event Event may refer to: Gatherings of people * Ceremony A ceremony (, ) is a unified ritual A ritual is a sequence of activities involving gestures, words, actions, or objects, performed in a sequestered place and according to a set sequence. Rit ...
. Central subjects in probability theory include discrete and continuous
random variable A random variable is a variable whose values depend on outcomes of a random In common parlance, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no ...
s,
probability distributions In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...
, and
stochastic process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a Indexed family, family of random variables. Stochastic processes are widely used as mathematical models of systems and phen ... es, which provide mathematical abstractions of non-deterministic or uncertain processes or measured
quantities Quantity or amount is a property that can exist as a multitude Multitude is a term for a group of people who cannot be classed under any other distinct category, except for their shared fact of existence. The term has a history of use reaching ba ...
that may either be single occurrences or evolve over time in a random fashion. Although it is not possible to perfectly predict random events, much can be said about their behavior. Two major results in probability theory describing such behaviour are the
law of large numbers and the
central limit theorem In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by express ...
. As a mathematical foundation for
statistics Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data Data (; ) are individual facts, statistics, or items of information, often numeric. In a more technical sens ... , probability theory is essential to many human activities that involve quantitative analysis of data. Methods of probability theory also apply to descriptions of complex systems given only partial knowledge of their state, as in
statistical mechanics In physics Physics is the that studies , its , its and behavior through , and the related entities of and . "Physical science is that department of knowledge which relates to the order of nature, or, in other words, to the regular ...
or sequential estimation. A great discovery of twentieth-century
physics Physics is the that studies , its , its and behavior through , and the related entities of and . "Physical science is that department of knowledge which relates to the order of nature, or, in other words, to the regular succession of eve ... was the probabilistic nature of physical phenomena at atomic scales, described in
quantum mechanics Quantum mechanics is a fundamental theory A theory is a reason, rational type of abstraction, abstract thinking about a phenomenon, or the results of such thinking. The process of contemplative and rational thinking is often associated with ...
.

# History of probability

The modern mathematical theory of
probability Probability is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained ... has its roots in attempts to analyze
games of chance A game of chance is a game with separate sliding drawer, from 1390 to 1353 BC, made of glazed faience, dimensions: 5.5 × 7.7 × 21 cm, in the Brooklyn Museum (New York City) '', 1560, Pieter Bruegel the Elder File:Paul Cézanne, ...
by
Gerolamo Cardano Gerolamo (also Girolamo or Geronimo) Cardano (; french: link=no, Jérôme Cardan; la, Hieronymus Cardanus; 24 September 1501 (O. S.)– 21 September 1576 (O. S.)) was an Italian polymath A polymath ( el, πολυμαθής, ', "having learn ... in the sixteenth century, and by
Pierre de Fermat Pierre de Fermat (; between 31 October and 6 December 1607 – 12 January 1665) was a French French (french: français(e), link=no) may refer to: * Something of, from, or related to France France (), officially the French Republic (fren ... and
Blaise Pascal Blaise Pascal ( , , ; ; 19 June 1623 – 19 August 1662) was a French mathematician, physicist, inventor, philosopher, writer and Catholic Church, Catholic theologian. He was a child prodigy who was educated by his father, a tax collector i ... in the seventeenth century (for example the "
problem of points The problem of points, also called the problem of division of the stakes, is a classical problem in probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability int ...
").
Christiaan Huygens Christiaan Huygens ( , also , ; la, Hugenius; 14 April 1629 – 8 July 1695), also spelled Huyghens, was a Dutch mathematician, physicist, astronomer and inventor, who is regarded as one of the greatest scientists of all time and a major fig ... published a book on the subject in 1657 and in the 19th century,
Pierre Laplace Pierre-Simon, marquis de Laplace (; ; 23 March 1749 – 5 March 1827) was a French scholar A scholar is a person who pursues academic and intellectual activities, particularly those that develop expertise in an area of Studying, study. A ... completed what is today considered the classic interpretation. Initially, probability theory mainly considered events, and its methods were mainly
combinatorial Combinatorics is an area of mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematic ...
. Eventually, analytical considerations compelled the incorporation of variables into the theory. This culminated in modern probability theory, on foundations laid by
Andrey Nikolaevich Kolmogorov Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
. Kolmogorov combined the notion of
sample space In probability theory Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ... , introduced by
Richard von Mises Richard Edler von Mises (; 19 April 1883 – 14 July 1953) was an Austrian Austrian may refer to: * Austrians, someone from Austria or of Austrian descent ** Someone who is considered an Austrian citizen, see Austrian nationality law * Something ...
, and
measure theory Measure is a fundamental concept of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contai ...
and presented his
axiom system In mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...
for probability theory in 1933. This became the mostly undisputed axiomatic basis for modern probability theory; but, alternatives exist, such as the adoption of finite rather than countable additivity by
Bruno de Finetti Bruno de Finetti (13 June 1906 – 20 July 1985) was an Italian probabilist statistician and actuary An actuary is a business professional who deals with the measurement and management of risk In simple terms, risk is the possibility of somet ...
.

# Treatment

Most introductions to probability theory treat discrete probability distributions and continuous probability distributions separately. The measure theory-based treatment of probability covers the discrete, continuous, a mix of the two, and more.

## Motivation

Consider an
experiment An experiment is a procedure carried out to support or refute a hypothesis, or determine the efficacy or likelihood of something previously untried. Experiments provide insight into Causality, cause-and-effect by demonstrating what outcome oc ...
that can produce a number of outcomes. The set of all outcomes is called the ''
sample space In probability theory Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ... '' of the experiment. The ''
power set In mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities an ...
'' of the sample space (or equivalently, the event space) is formed by considering all different collections of possible results. For example, rolling an honest die produces one of six possible results. One collection of possible results corresponds to getting an odd number. Thus, the subset is an element of the power set of the sample space of die rolls. These collections are called ''events''. In this case, is the event that the die falls on some odd number. If the results that actually occur fall in a given event, that event is said to have occurred. Probability is a way of assigning every "event" a value between zero and one, with the requirement that the event made up of all possible results (in our example, the event ) be assigned a value of one. To qualify as a
probability distribution In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...
, the assignment of values must satisfy the requirement that if you look at a collection of mutually exclusive events (events that contain no common results, e.g., the events , , and are all mutually exclusive), the probability that any of these events occurs is given by the sum of the probabilities of the events. The probability that any one of the events , , or will occur is 5/6. This is the same as saying that the probability of event is 5/6. This event encompasses the possibility of any number except five being rolled. The mutually exclusive event has a probability of 1/6, and the event has a probability of 1, that is, absolute certainty. When doing calculations using the outcomes of an experiment, it is necessary that all those
elementary event Elementary may refer to: Arts, entertainment, and media Music * ''Elementary'' (Cindy Morgan album), 2001 * ''Elementary'' (The End album), 2007 * ''Elementary'', a Melvin "Wah-Wah Watson" Ragin album, 1977 Other uses in arts, entertainment, an ...
s have a number assigned to them. This is done using a
random variable A random variable is a variable whose values depend on outcomes of a random In common parlance, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no ...
. A random variable is a function that assigns to each elementary event in the sample space a
real number In mathematics Mathematics (from Greek: ) includes the study of such topics as numbers ( and ), formulas and related structures (), shapes and spaces in which they are contained (), and quantities and their changes ( and ). There is no g ...
. This function is usually denoted by a capital letter. In the case of a die, the assignment of a number to a certain elementary events can be done using the
identity function Graph of the identity function on the real numbers In mathematics, an identity function, also called an identity relation, identity map or identity transformation, is a function (mathematics), function that always returns the same value that was ... . This does not always work. For example, when
flipping a coin the two possible outcomes are "heads" and "tails". In this example, the random variable ''X'' could assign to the outcome "heads" the number "0" ($X\left(heads\right)=0$) and to the outcome "tails" the number "1" ($X\left(tails\right)=1$).

## Discrete probability distributions deals with events that occur in
countable In mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and ...
sample spaces. Examples: Throwing
dice Dice (singular die or dice) are small, throwable objects with marked sides that can rest in multiple positions. They are used for generating random numbers, commonly as part of tabletop game Tabletop games are game with separate sliding d ... , experiments with decks of cards,
random walk In mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities ...
, and tossing
coin A coin is a small, flat, (usually, depending on the country or value) round piece of metal A metal (from Greek Greek may refer to: Greece Anything of, from, or related to Greece Greece ( el, Ελλάδα, , ), officially the Hell ... s : Initially the probability of an event to occur was defined as the number of cases favorable for the event, over the number of total outcomes possible in an equiprobable sample space: see Classical definition of probability. For example, if the event is "occurrence of an even number when a die is rolled", the probability is given by $\tfrac=\tfrac$, since 3 faces out of the 6 have even numbers and each face has the same probability of appearing. : The modern definition starts with a finite or countable set called the
sample space In probability theory Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ... , which relates to the set of all ''possible outcomes'' in classical sense, denoted by $\Omega$. It is then assumed that for each element $x \in \Omega\,$, an intrinsic "probability" value $f\left(x\right)\,$ is attached, which satisfies the following properties: # # $\sum_ f\left(x\right) = 1\,.$ That is, the probability function ''f''(''x'') lies between zero and one for every value of ''x'' in the sample space ''Ω'', and the sum of ''f''(''x'') over all values ''x'' in the sample space ''Ω'' is equal to 1. An is defined as any
subset In mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities a ... $E\,$ of the sample space $\Omega\,$. The of the event $E\,$ is defined as :$P\left(E\right)=\sum_ f\left(x\right)\,.$ So, the probability of the entire sample space is 1, and the probability of the null event is 0. The function $f\left(x\right)\,$ mapping a point in the sample space to the "probability" value is called a abbreviated as . The modern definition does not try to answer how probability mass functions are obtained; instead, it builds a theory that assumes their existence.

## Continuous probability distributions deals with events that occur in a continuous sample space. : The classical definition breaks down when confronted with the continuous case. See Bertrand's paradox. : If the sample space of a random variable ''X'' is the set of
real numbers In mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities an ... ($\mathbb$) or a subset thereof, then a function called the (or ) $F\,$ exists, defined by $F\left(x\right) = P\left(X\le x\right) \,$. That is, ''F''(''x'') returns the probability that ''X'' will be less than or equal to ''x''. The cdf necessarily satisfies the following properties. # $F\,$ is a monotonically non-decreasing,
right-continuous In mathematics, a continuous function is a function (mathematics), function that does not have any abrupt changes in Value (mathematics), value, known as Classification of discontinuities, discontinuities. More precisely, a function is continuous ... function; # $\lim_ F\left(x\right)=0\,;$ # $\lim_ F\left(x\right)=1\,.$ If $F\,$ is
absolutely continuousIn calculus, absolute continuity is a smoothness property of function (mathematics), functions that is stronger than continuous function, continuity and uniform continuity. The notion of absolute continuity allows one to obtain generalizations of the ...
, i.e., its derivative exists and integrating the derivative gives us the cdf back again, then the random variable ''X'' is said to have a or or simply $f\left(x\right)=\frac\,.$ For a set $E \subseteq \mathbb$, the probability of the random variable ''X'' being in $E\,$ is :$P\left(X\in E\right) = \int_ dF\left(x\right)\,.$ In case the probability density function exists, this can be written as :$P\left(X\in E\right) = \int_ f\left(x\right)\,dx\,.$ Whereas the ''pdf'' exists only for continuous random variables, the ''cdf'' exists for all random variables (including discrete random variables) that take values in $\mathbb\,.$ These concepts can be generalized for
multidimensional cases on $\mathbb^n$ and other continuous sample spaces.

## Measure-theoretic probability theory

The ''
raison d'être Raison d'être is a French expression commonly used in English, meaning "reason for being" or "reason to be". Raison d'être may refer to: Music * Raison d'être (band), a Swedish dark-ambient-industrial-drone music project * Raison D'être (albu ...
'' of the measure-theoretic treatment of probability is that it unifies the discrete and the continuous cases, and makes the difference a question of which measure is used. Furthermore, it covers distributions that are neither discrete nor continuous nor mixtures of the two. An example of such distributions could be a mix of discrete and continuous distributions—for example, a random variable that is 0 with probability 1/2, and takes a random value from a normal distribution with probability 1/2. It can still be studied to some extent by considering it to have a pdf of , where
Dirac delta function In mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities ...
. Other distributions may not even be a mix, for example, the
Cantor distribution The Cantor distribution is the probability distribution In probability theory and statistics Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statis ...
has no positive probability for any single point, neither does it have a density. The modern approach to probability theory solves these problems using
measure theory Measure is a fundamental concept of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contai ...
to define the
probability space In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expres ...
: Given any set $\Omega\,$ (also called ) and a
σ-algebra In mathematical analysis and in probability theory, a σ-algebra (also σ-field) on a set ''X'' is a Family of sets, collection Σ of subsets of ''X'' that includes ''X'' itself, is Closure (mathematics), closed under complement (set theory), comp ...
$\mathcal\,$ on it, a measure $P\,$ defined on $\mathcal\,$ is called a if $P\left(\Omega\right)=1.\,$ If $\mathcal\,$ is the Borel σ-algebra on the set of real numbers, then there is a unique probability measure on $\mathcal\,$ for any cdf, and vice versa. The measure corresponding to a cdf is said to be by the cdf. This measure coincides with the pmf for discrete variables and pdf for continuous variables, making the measure-theoretic approach free of fallacies. The ''probability'' of a set $E\,$ in the σ-algebra $\mathcal\,$ is defined as :$P\left(E\right) = \int_ \mu_F\left(d\omega\right)\,$ where the integration is with respect to the measure $\mu_F\,$ induced by $F\,.$ Along with providing better understanding and unification of discrete and continuous probabilities, measure-theoretic treatment also allows us to work on probabilities outside $\mathbb^n$, as in the theory of
stochastic process In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a Indexed family, family of random variables. Stochastic processes are widely used as mathematical models of systems and phen ... es. For example, to study
Brownian motion Brownian motion, or pedesis (from grc, πήδησις "leaping"), is the random motion of particle In the Outline of physical science, physical sciences, a particle (or corpuscule in older texts) is a small wikt:local, localized physica ... , probability is defined on a space of functions. When it's convenient to work with a dominating measure, the Radon-Nikodym theorem is used to define a density as the Radon-Nikodym derivative of the probability distribution of interest with respect to this dominating measure. Discrete densities are usually defined as this derivative with respect to a
counting measure In mathematics, specifically measure theory, the counting measure is an intuitive way to put a Measure (mathematics), measure on any Set (mathematics), set – the "size" of a subset is taken to be the number of elements in the subset if the subset ...
over the set of all possible outcomes. Densities for
absolutely continuousIn calculus, absolute continuity is a smoothness property of function (mathematics), functions that is stronger than continuous function, continuity and uniform continuity. The notion of absolute continuity allows one to obtain generalizations of the ...
distributions are usually defined as this derivative with respect to the
Lebesgue measure In Measure (mathematics), measure theory, a branch of mathematics, the Lebesgue measure, named after france, French mathematician Henri Lebesgue, is the standard way of assigning a measure (mathematics), measure to subsets of ''n''-dimensional Eucli ...
. If a theorem can be proved in this general setting, it holds for both discrete and continuous distributions as well as others; separate proofs are not required for discrete and continuous distributions.

# Classical probability distributions

Certain random variables occur very often in probability theory because they well describe many natural or physical processes. Their distributions, therefore, have gained ''special importance'' in probability theory. Some fundamental ''discrete distributions'' are the discrete uniform,
BernoulliBernoulli can refer to: People *Bernoulli family of 17th and 18th century Swiss mathematicians: ** Daniel Bernoulli (1700–1782), developer of Bernoulli's principle ** Jacob Bernoulli (1654–1705), also known as Jacques, after whom Bernoulli numbe ... ,
binomial , negative binomial, Poisson and
geometric distribution In probability theory Probability theory is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces i ...
s. Important ''continuous distributions'' include the continuous uniform,
normal ,
exponential Exponential may refer to any of several mathematical topics related to exponentiation Exponentiation is a mathematical Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and ...
,
gamma Gamma (uppercase , lowercase ; ''gámma'') is the third letter of the Greek alphabet The Greek alphabet has been used to write the Greek language since the late ninth or early eighth century BC. It is derived from the earlier Phoenician ... and
beta distribution In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval
, 1 The comma is a punctuation Punctuation (or sometimes interpunction) is the use of spacing, conventional signs (called punctuation marks), and certain typographical devices as aids to the understanding and correct reading of written text, ...
Statistical parameter, parameterized by two positive shape parameters, denoted by ''α'' and ''β'', tha ...
s.

# Convergence of random variables

In probability theory, there are several notions of convergence for
random variable A random variable is a variable whose values depend on outcomes of a random In common parlance, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no ...
s. They are listed below in the order of strength, i.e., any subsequent notion of convergence in the list implies convergence according to all of the preceding notions. ;Weak convergence: A sequence of random variables $X_1,X_2,\dots,\,$ converges to the random variable $X\,$ if their respective cumulative ''distribution functions'' $F_1,F_2,\dots\,$ converge to the cumulative distribution function $F\,$ of $X\,$, wherever $F\,$ is
continuous Continuity or continuous may refer to: Mathematics * Continuity (mathematics), the opposing concept to discreteness; common examples include ** Continuous probability distribution or random variable in probability and statistics ** Continuous ga ...
. Weak convergence is also called . :Most common shorthand notation: $\displaystyle X_n \, \xrightarrow \, X$ ;Convergence in probability: The sequence of random variables $X_1,X_2,\dots\,$ is said to converge towards the random variable $X\,$ if $\lim_P\left\left(\left, X_n-X\\geq\varepsilon\right\right)=0$ for every ε > 0. :Most common shorthand notation: $\displaystyle X_n \, \xrightarrow \, X$ ;Strong convergence: The sequence of random variables $X_1,X_2,\dots\,$ is said to converge towards the random variable $X\,$ if $P\left(\lim_ X_n=X\right)=1$. Strong convergence is also known as . :Most common shorthand notation: $\displaystyle X_n \, \xrightarrow \, X$ As the names indicate, weak convergence is weaker than strong convergence. In fact, strong convergence implies convergence in probability, and convergence in probability implies weak convergence. The reverse statements are not always true.

## Law of large numbers

Common intuition suggests that if a fair coin is tossed many times, then ''roughly'' half of the time it will turn up ''heads'', and the other half it will turn up ''tails''. Furthermore, the more often the coin is tossed, the more likely it should be that the ratio of the number of ''heads'' to the number of ''tails'' will approach unity. Modern probability theory provides a formal version of this intuitive idea, known as the . This law is remarkable because it is not assumed in the foundations of probability theory, but instead emerges from these foundations as a theorem. Since it links theoretically derived probabilities to their actual frequency of occurrence in the real world, the law of large numbers is considered as a pillar in the history of statistical theory and has had widespread influence. The (LLN) states that the sample average :$\overline_n=\frac1n$ of a sequence of independent and identically distributed random variables $X_k$ converges towards their common expectation $\mu$, provided that the expectation of $, X_k,$ is finite. It is in the different forms of
convergence of random variables In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressi ...
that separates the ''weak'' and the ''strong'' law of large numbers :Weak law: $\displaystyle \overline_n \, \xrightarrow \, \mu$ for $n \to \infty$ :Strong law: $\displaystyle \overline_n \, \xrightarrow \, \mu$ for $n \to \infty .$ It follows from the LLN that if an event of probability ''p'' is observed repeatedly during independent experiments, the ratio of the observed frequency of that event to the total number of repetitions converges towards ''p''. For example, if $Y_1,Y_2,...\,$ are independent
Bernoulli random variables taking values 1 with probability ''p'' and 0 with probability 1-''p'', then $\textrm\left(Y_i\right)=p$ for all ''i'', so that $\bar Y_n$ converges to ''p''
almost surely In probability theory Probability theory is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ...
.

## Central limit theorem

"The central limit theorem (CLT) is one of the great results of mathematics." (Chapter 18 in
David Williams , "Probability with martingales", Cambridge 1991/2008
) It explains the ubiquitous occurrence of the
normal distribution In probability theory Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these ... in nature. The theorem states that the
average In colloquial language, an average is a single number taken as representative of a non-empty list of numbers. Different concepts of average are used in different contexts. Often "average" refers to the arithmetic mean, the sum of the numbers divide ... of many independent and identically distributed random variables with finite variance tends towards a normal distribution ''irrespective'' of the distribution followed by the original random variables. Formally, let $X_1,X_2,\dots\,$ be independent random variables with
mean There are several kinds of mean in mathematics, especially in statistics. For a data set, the ''arithmetic mean'', also known as arithmetic average, is a central value of a finite set of numbers: specifically, the sum of the values divided by ...
$\mu$ and
variance In probability theory Probability theory is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ... $\sigma^2 > 0.\,$ Then the sequence of random variables :$Z_n=\frac\,$ converges in distribution to a
standard normal In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by ex ...
random variable. For some classes of random variables the classic central limit theorem works rather fast (see Berry–Esseen theorem), for example the distributions with finite first, second, and third moment from the
exponential family In theory of probability, probability and statistics, an exponential family is a parametric model, parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, based on ...
; on the other hand, for some random variables of the heavy tail and
fat tail A fat-tailed distribution is a probability distribution that exhibits a large skewness or kurtosis, relative to that of either a normal distribution or an exponential distribution. In common usage, the terms fat-tailed and Heavy-tailed distributio ...
variety, it works very slowly or may not work at all: in such cases one may use the Generalized Central Limit Theorem (GCLT).

* Catalog of articles in probability theory *
Expected value In probability theory Probability theory is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and space ...
and
Variance In probability theory Probability theory is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ... *
Fuzzy logic In logic Logic is an interdisciplinary field which studies truth and reasoning. Informal logic seeks to characterize Validity (logic), valid arguments informally, for instance by listing varieties of fallacies. Formal logic represents statem ... and
Fuzzy measure theoryIn mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ha ...
*
Glossary of probability and statistics The following is a glossary __NOTOC__ A glossary (from grc, γλῶσσα / language, speech, wording) also known as a vocabulary or clavis, is an alphabetical list of terms in a particular domain of knowledge with the definitions for those ...
*
Likelihood function The likelihood function (often simply called the likelihood) describes the joint probability Given random variables X,Y,\ldots, that are defined on a probability space, the joint probability distribution for X,Y,\ldots is a probability distribut ...
*
List of probability topics {{ProbabilityTopicsTOC This is a list of probability topics, by Wikipedia page. It overlaps with the (alphabetical) list of statistical topics. There are also the outline of probability and catalog of articles in probability theory. For distribution ...
* List of publications in statistics * List of statistical topics * Notation in probability *
Predictive modelling Predictive modelling uses statistics to predict outcomes. Most often the event one wants to predict is in the future, but predictive modelling can be applied to any type of unknown event, regardless of when it occurred. For example, predictive model ...
*
Probabilistic logic Probabilistic logic (also probability logic and probabilistic reasoning) involves the use of probability and logic to deal with uncertain situations. The result is a richer and more expressive formalism with a broad range of possible application are ...
– A combination of probability theory and logic * Probabilistic proofs of non-probabilistic theorems *
Probability distribution In probability theory Probability theory is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ...
*
Probability axioms The Kolmogorov axioms are the foundations of probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in ...
*
Probability interpretations The word probability has been used in a variety of ways since it was first applied to the mathematical study of games of chance. Does probability measure the real, physical, tendency of something to occur, or is it a measure of how strongly one be ...
*
Probability space In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a space (mathematics), mathematical construct that provides a formal model of a randomness, random process or "experiment". For example, one can define a ...
*
Statistical independence Independence is a fundamental notion in probability theory Probability theory is the branch of mathematics Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related stru ...
*
Statistical physics Statistical physics is a branch of physics Physics (from grc, φυσική (ἐπιστήμη), physikḗ (epistḗmē), knowledge of nature, from ''phýsis'' 'nature'), , is the natural science that studies matter, its Motion (physics), ...
*
Subjective logicSubjective logic is a type of probabilistic logic that explicitly takes epistemic uncertainty and source trust into account. In general, subjective logic is suitable for modeling and analysing situations involving uncertainty and relatively unreliabl ...
* Probability of the union of pairwise independent events

# References

* :: The first major treatise blending calculus with probability theory, originally in French: ''Théorie Analytique des Probabilités''. * :: An English translation by Nathan Morrison appeared under the title ''Foundations of the Theory of Probability'' (Chelsea, New York) in 1950, with a second edition in 1956. * * Olav Kallenberg; ''Foundations of Modern Probability,'' 2nd ed. Springer Series in Statistics. (2002). 650 pp. * :: A lively introduction to probability theory for the beginner. * Olav Kallenberg; ''Probabilistic Symmetries and Invariance Principles''. Springer -Verlag, New York (2005). 510 pp. * {{DEFAULTSORT:Probability Theory id:Peluang (matematika)