Probability theory is the branch of

sample space
In probability theory
Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ...

, which relates to the set of all ''possible outcomes'' in classical sense, denoted by $\backslash Omega$. It is then assumed that for each element $x\; \backslash in\; \backslash Omega\backslash ,$, an intrinsic "probability" value $f(x)\backslash ,$ is attached, which satisfies the following properties:
# $f(x)\backslash in;\; href="/html/ALL/s/,1.html"\; ;"title=",1">,1$
# $\backslash sum\_\; f(x)\; =\; 1\backslash ,.$
That is, the probability function ''f''(''x'') lies between zero and one for every value of ''x'' in the sample space ''Ω'', and the sum of ''f''(''x'') over all values ''x'' in the sample space ''Ω'' is equal to 1. An is defined as any

mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and their changes (cal ...

concerned with probability
Probability is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained ...

. Although there are several different probability interpretations
The word probability has been used in a variety of ways since it was first applied to the mathematical study of games of chance. Does probability measure the real, physical, tendency of something to occur, or is it a measure of how strongly one bel ...

, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of axioms
An axiom, postulate or assumption is a statement that is taken to be truth, true, to serve as a premise or starting point for further reasoning and arguments. The word comes from the Greek ''axíōma'' () 'that which is thought worthy or fit' or ...

. Typically these axioms formalise probability in terms of a probability space
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expres ...

, which assigns a measure taking values between 0 and 1, termed the probability measure
In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies Measure (mathematics), measure properties such as ''countable additivity''. The difference between a probability meas ...

, to a set of outcomes called the sample space
In probability theory
Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ...

. Any specified subset of the sample space is called an event
Event may refer to:
Gatherings of people
* Ceremony
A ceremony (, ) is a unified ritual
A ritual is a sequence of activities involving gestures, words, actions, or objects, performed in a sequestered place and according to a set sequence. Rit ...

.
Central subjects in probability theory include discrete and continuous random variable
A random variable is a variable whose values depend on outcomes of a random
In common parlance, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no ...

s, probability distributions
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...

, and stochastic process
In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a Indexed family, family of random variables. Stochastic processes are widely used as mathematical models of systems and phen ...

es, which provide mathematical abstractions of non-deterministic or uncertain processes or measured quantities
Quantity or amount is a property that can exist as a multitude
Multitude is a term for a group of people who cannot be classed under any other distinct category, except for their shared fact of existence. The term has a history of use reaching ba ...

that may either be single occurrences or evolve over time in a random fashion.
Although it is not possible to perfectly predict random events, much can be said about their behavior. Two major results in probability theory describing such behaviour are the and the central limit theorem
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by express ...

.
As a mathematical foundation for statistics
Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data
Data (; ) are individual facts, statistics, or items of information, often numeric. In a more technical sens ...

, probability theory is essential to many human activities that involve quantitative analysis of data. Methods of probability theory also apply to descriptions of complex systems given only partial knowledge of their state, as in statistical mechanics
In physics
Physics is the that studies , its , its and behavior through , and the related entities of and . "Physical science is that department of knowledge which relates to the order of nature, or, in other words, to the regular ...

or sequential estimation. A great discovery of twentieth-century physics
Physics is the that studies , its , its and behavior through , and the related entities of and . "Physical science is that department of knowledge which relates to the order of nature, or, in other words, to the regular succession of eve ...

was the probabilistic nature of physical phenomena at atomic scales, described in quantum mechanics
Quantum mechanics is a fundamental theory
A theory is a reason, rational type of abstraction, abstract thinking about a phenomenon, or the results of such thinking. The process of contemplative and rational thinking is often associated with ...

.
History of probability

The modern mathematical theory ofprobability
Probability is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained ...

has its roots in attempts to analyze games of chance
A game of chance is a game
with separate sliding drawer, from 1390 to 1353 BC, made of glazed faience, dimensions: 5.5 × 7.7 × 21 cm, in the Brooklyn Museum (New York City)
'', 1560, Pieter Bruegel the Elder
File:Paul Cézanne, ...

by Gerolamo Cardano
Gerolamo (also Girolamo or Geronimo) Cardano (; french: link=no, Jérôme Cardan; la, Hieronymus Cardanus; 24 September 1501 (O. S.)– 21 September 1576 (O. S.)) was an Italian polymath
A polymath ( el, πολυμαθής, ', "having learn ...

in the sixteenth century, and by Pierre de Fermat
Pierre de Fermat (; between 31 October and 6 December 1607 – 12 January 1665) was a French
French (french: français(e), link=no) may refer to:
* Something of, from, or related to France
France (), officially the French Republic (fren ...

and Blaise Pascal
Blaise Pascal ( , , ; ; 19 June 1623 – 19 August 1662) was a French mathematician, physicist, inventor, philosopher, writer and Catholic Church, Catholic theologian.
He was a child prodigy who was educated by his father, a tax collector i ...

in the seventeenth century (for example the "problem of points
The problem of points, also called the problem of division of the stakes, is a classical problem in probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability int ...

"). Christiaan Huygens
Christiaan Huygens ( , also , ; la, Hugenius; 14 April 1629 – 8 July 1695), also spelled Huyghens, was a Dutch mathematician, physicist, astronomer and inventor, who is regarded as one of the greatest scientists of all time and a major fig ...

published a book on the subject in 1657 and in the 19th century, Pierre Laplace
Pierre-Simon, marquis de Laplace (; ; 23 March 1749 – 5 March 1827) was a French scholar
A scholar is a person who pursues academic and intellectual activities, particularly those that develop expertise in an area of Studying, study. A ...

completed what is today considered the classic interpretation.
Initially, probability theory mainly considered events, and its methods were mainly combinatorial
Combinatorics is an area of mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematic ...

. Eventually, analytical considerations compelled the incorporation of variables into the theory.
This culminated in modern probability theory, on foundations laid by Andrey Nikolaevich Kolmogorov
Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...

. Kolmogorov combined the notion of sample space
In probability theory
Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ...

, introduced by Richard von Mises
Richard Edler von Mises (; 19 April 1883 – 14 July 1953) was an Austrian
Austrian may refer to:
* Austrians, someone from Austria or of Austrian descent
** Someone who is considered an Austrian citizen, see Austrian nationality law
* Something ...

, and measure theory
Measure is a fundamental concept of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contai ...

and presented his axiom system
In mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...

for probability theory in 1933. This became the mostly undisputed axiomatic basis for modern probability theory; but, alternatives exist, such as the adoption of finite rather than countable additivity by Bruno de Finetti
Bruno de Finetti (13 June 1906 – 20 July 1985) was an Italian probabilist statistician and actuary
An actuary is a business professional who deals with the measurement and management of risk
In simple terms, risk is the possibility of somet ...

.
Treatment

Most introductions to probability theory treat discrete probability distributions and continuous probability distributions separately. The measure theory-based treatment of probability covers the discrete, continuous, a mix of the two, and more.Motivation

Consider anexperiment
An experiment is a procedure carried out to support or refute a hypothesis, or determine the efficacy or likelihood of something previously untried. Experiments provide insight into Causality, cause-and-effect by demonstrating what outcome oc ...

that can produce a number of outcomes. The set of all outcomes is called the ''sample space
In probability theory
Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these axiom ...

'' of the experiment. The ''power set
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities an ...

'' of the sample space (or equivalently, the event space) is formed by considering all different collections of possible results. For example, rolling an honest die produces one of six possible results. One collection of possible results corresponds to getting an odd number. Thus, the subset is an element of the power set of the sample space of die rolls. These collections are called ''events''. In this case, is the event that the die falls on some odd number. If the results that actually occur fall in a given event, that event is said to have occurred.
Probability is a way of assigning every "event" a value between zero and one, with the requirement that the event made up of all possible results (in our example, the event ) be assigned a value of one. To qualify as a probability distribution
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expre ...

, the assignment of values must satisfy the requirement that if you look at a collection of mutually exclusive events (events that contain no common results, e.g., the events , , and are all mutually exclusive), the probability that any of these events occurs is given by the sum of the probabilities of the events.
The probability that any one of the events , , or will occur is 5/6. This is the same as saying that the probability of event is 5/6. This event encompasses the possibility of any number except five being rolled. The mutually exclusive event has a probability of 1/6, and the event has a probability of 1, that is, absolute certainty.
When doing calculations using the outcomes of an experiment, it is necessary that all those elementary event
Elementary may refer to:
Arts, entertainment, and media Music
* ''Elementary'' (Cindy Morgan album), 2001
* ''Elementary'' (The End album), 2007
* ''Elementary'', a Melvin "Wah-Wah Watson" Ragin album, 1977
Other uses in arts, entertainment, an ...

s have a number assigned to them. This is done using a random variable
A random variable is a variable whose values depend on outcomes of a random
In common parlance, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no ...

. A random variable is a function that assigns to each elementary event in the sample space a real number
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers ( and ), formulas and related structures (), shapes and spaces in which they are contained (), and quantities and their changes ( and ). There is no g ...

. This function is usually denoted by a capital letter. In the case of a die, the assignment of a number to a certain elementary events can be done using the identity function
Graph of the identity function on the real numbers
In mathematics, an identity function, also called an identity relation, identity map or identity transformation, is a function (mathematics), function that always returns the same value that was ...

. This does not always work. For example, when the two possible outcomes are "heads" and "tails". In this example, the random variable ''X'' could assign to the outcome "heads" the number "0" ($X(heads)=0$) and to the outcome "tails" the number "1" ($X(tails)=1$).
Discrete probability distributions

deals with events that occur incountable
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities and ...

sample spaces.
Examples: Throwing dice
Dice (singular die or dice) are small, throwable objects with marked sides that can rest in multiple positions. They are used for generating random numbers, commonly as part of tabletop game
Tabletop games are game
with separate sliding d ...

, experiments with decks of cards, random walk
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities ...

, and tossing coin
A coin is a small, flat, (usually, depending on the country or value) round piece of metal
A metal (from Greek
Greek may refer to:
Greece
Anything of, from, or related to Greece
Greece ( el, Ελλάδα, , ), officially the Hell ...

s
:
Initially the probability of an event to occur was defined as the number of cases favorable for the event, over the number of total outcomes possible in an equiprobable sample space: see Classical definition of probability.
For example, if the event is "occurrence of an even number when a die is rolled", the probability is given by $\backslash tfrac=\backslash tfrac$, since 3 faces out of the 6 have even numbers and each face has the same probability of appearing.
:
The modern definition starts with a finite or countable set called the subset
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities a ...

$E\backslash ,$ of the sample space $\backslash Omega\backslash ,$. The of the event $E\backslash ,$ is defined as
:$P(E)=\backslash sum\_\; f(x)\backslash ,.$
So, the probability of the entire sample space is 1, and the probability of the null event is 0.
The function $f(x)\backslash ,$ mapping a point in the sample space to the "probability" value is called a abbreviated as . The modern definition does not try to answer how probability mass functions are obtained; instead, it builds a theory that assumes their existence.
Continuous probability distributions

deals with events that occur in a continuous sample space. : The classical definition breaks down when confronted with the continuous case. See Bertrand's paradox. : If the sample space of a random variable ''X'' is the set ofreal numbers
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities an ...

($\backslash mathbb$) or a subset thereof, then a function called the (or ) $F\backslash ,$ exists, defined by $F(x)\; =\; P(X\backslash le\; x)\; \backslash ,$. That is, ''F''(''x'') returns the probability that ''X'' will be less than or equal to ''x''.
The cdf necessarily satisfies the following properties.
# $F\backslash ,$ is a monotonically non-decreasing, right-continuous
In mathematics, a continuous function is a function (mathematics), function that does not have any abrupt changes in Value (mathematics), value, known as Classification of discontinuities, discontinuities. More precisely, a function is continuous ...

function;
# $\backslash lim\_\; F(x)=0\backslash ,;$
# $\backslash lim\_\; F(x)=1\backslash ,.$
If $F\backslash ,$ is absolutely continuousIn calculus, absolute continuity is a smoothness property of function (mathematics), functions that is stronger than continuous function, continuity and uniform continuity. The notion of absolute continuity allows one to obtain generalizations of the ...

, i.e., its derivative exists and integrating the derivative gives us the cdf back again, then the random variable ''X'' is said to have a or or simply $f(x)=\backslash frac\backslash ,.$
For a set $E\; \backslash subseteq\; \backslash mathbb$, the probability of the random variable ''X'' being in $E\backslash ,$ is
:$P(X\backslash in\; E)\; =\; \backslash int\_\; dF(x)\backslash ,.$
In case the probability density function exists, this can be written as
:$P(X\backslash in\; E)\; =\; \backslash int\_\; f(x)\backslash ,dx\backslash ,.$
Whereas the ''pdf'' exists only for continuous random variables, the ''cdf'' exists for all random variables (including discrete random variables) that take values in $\backslash mathbb\backslash ,.$
These concepts can be generalized for cases on $\backslash mathbb^n$ and other continuous sample spaces.
Measure-theoretic probability theory

The ''raison d'être
Raison d'être is a French expression commonly used in English, meaning "reason for being" or "reason to be".
Raison d'être may refer to:
Music
* Raison d'être (band), a Swedish dark-ambient-industrial-drone music project
* Raison D'être (albu ...

'' of the measure-theoretic treatment of probability is that it unifies the discrete and the continuous cases, and makes the difference a question of which measure is used. Furthermore, it covers distributions that are neither discrete nor continuous nor mixtures of the two.
An example of such distributions could be a mix of discrete and continuous distributions—for example, a random variable that is 0 with probability 1/2, and takes a random value from a normal distribution with probability 1/2. It can still be studied to some extent by considering it to have a pdf of $(\backslash delta;\; href="/html/ALL/s/.html"\; ;"title="">$, where $\backslash delta;\; href="/html/ALL/s/.html"\; ;"title="">$Dirac delta function
In mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contained (geometry), and quantities ...

.
Other distributions may not even be a mix, for example, the Cantor distribution
The Cantor distribution is the probability distribution
In probability theory and statistics
Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statis ...

has no positive probability for any single point, neither does it have a density. The modern approach to probability theory solves these problems using measure theory
Measure is a fundamental concept of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces in which they are contai ...

to define the probability space
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expres ...

:
Given any set $\backslash Omega\backslash ,$ (also called ) and a σ-algebra
In mathematical analysis and in probability theory, a σ-algebra (also σ-field) on a set ''X'' is a Family of sets, collection Σ of subsets of ''X'' that
includes ''X'' itself, is Closure (mathematics), closed under complement (set theory), comp ...

$\backslash mathcal\backslash ,$ on it, a measure $P\backslash ,$ defined on $\backslash mathcal\backslash ,$ is called a if $P(\backslash Omega)=1.\backslash ,$
If $\backslash mathcal\backslash ,$ is the Borel σ-algebra on the set of real numbers, then there is a unique probability measure on $\backslash mathcal\backslash ,$ for any cdf, and vice versa. The measure corresponding to a cdf is said to be by the cdf. This measure coincides with the pmf for discrete variables and pdf for continuous variables, making the measure-theoretic approach free of fallacies.
The ''probability'' of a set $E\backslash ,$ in the σ-algebra $\backslash mathcal\backslash ,$ is defined as
:$P(E)\; =\; \backslash int\_\; \backslash mu\_F(d\backslash omega)\backslash ,$
where the integration is with respect to the measure $\backslash mu\_F\backslash ,$ induced by $F\backslash ,.$
Along with providing better understanding and unification of discrete and continuous probabilities, measure-theoretic treatment also allows us to work on probabilities outside $\backslash mathbb^n$, as in the theory of stochastic process
In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a Indexed family, family of random variables. Stochastic processes are widely used as mathematical models of systems and phen ...

es. For example, to study Brownian motion
Brownian motion, or pedesis (from grc, πήδησις "leaping"), is the random motion of particle
In the Outline of physical science, physical sciences, a particle (or corpuscule in older texts) is a small wikt:local, localized physica ...

, probability is defined on a space of functions.
When it's convenient to work with a dominating measure, the Radon-Nikodym theorem is used to define a density as the Radon-Nikodym derivative of the probability distribution of interest with respect to this dominating measure. Discrete densities are usually defined as this derivative with respect to a counting measure In mathematics, specifically measure theory, the counting measure is an intuitive way to put a Measure (mathematics), measure on any Set (mathematics), set – the "size" of a subset is taken to be the number of elements in the subset if the subset ...

over the set of all possible outcomes. Densities for absolutely continuousIn calculus, absolute continuity is a smoothness property of function (mathematics), functions that is stronger than continuous function, continuity and uniform continuity. The notion of absolute continuity allows one to obtain generalizations of the ...

distributions are usually defined as this derivative with respect to the Lebesgue measure In Measure (mathematics), measure theory, a branch of mathematics, the Lebesgue measure, named after france, French mathematician Henri Lebesgue, is the standard way of assigning a measure (mathematics), measure to subsets of ''n''-dimensional Eucli ...

. If a theorem can be proved in this general setting, it holds for both discrete and continuous distributions as well as others; separate proofs are not required for discrete and continuous distributions.
Classical probability distributions

Certain random variables occur very often in probability theory because they well describe many natural or physical processes. Their distributions, therefore, have gained ''special importance'' in probability theory. Some fundamental ''discrete distributions'' are the discrete uniform,BernoulliBernoulli can refer to:
People
*Bernoulli family of 17th and 18th century Swiss mathematicians:
** Daniel Bernoulli (1700–1782), developer of Bernoulli's principle
** Jacob Bernoulli (1654–1705), also known as Jacques, after whom Bernoulli numbe ...

, , negative binomial, Poisson and geometric distribution
In probability theory
Probability theory is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces i ...

s. Important ''continuous distributions'' include the continuous uniform, , exponential
Exponential may refer to any of several mathematical topics related to exponentiation
Exponentiation is a mathematical
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and ...

, gamma
Gamma (uppercase , lowercase ; ''gámma'') is the third letter of the Greek alphabet
The Greek alphabet has been used to write the Greek language since the late ninth or early eighth century BC. It is derived from the earlier Phoenician ...

and beta distribution
In probability theory and statistics, the beta distribution is a family of continuous probability distributions defined on the interval

s.
, 1
The comma is a punctuation
Punctuation (or sometimes interpunction) is the use of spacing, conventional signs (called punctuation marks), and certain typographical devices as aids to the understanding and correct reading of written text, ...

Statistical parameter, parameterized by two positive shape parameters, denoted by ''α'' and ''β'', tha ...Convergence of random variables

In probability theory, there are several notions of convergence forrandom variable
A random variable is a variable whose values depend on outcomes of a random
In common parlance, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no ...

s. They are listed below in the order of strength, i.e., any subsequent notion of convergence in the list implies convergence according to all of the preceding notions.
;Weak convergence: A sequence of random variables $X\_1,X\_2,\backslash dots,\backslash ,$ converges to the random variable $X\backslash ,$ if their respective cumulative ''distribution functions'' $F\_1,F\_2,\backslash dots\backslash ,$ converge to the cumulative distribution function $F\backslash ,$ of $X\backslash ,$, wherever $F\backslash ,$ is continuous
Continuity or continuous may refer to:
Mathematics
* Continuity (mathematics), the opposing concept to discreteness; common examples include
** Continuous probability distribution or random variable in probability and statistics
** Continuous ga ...

. Weak convergence is also called .
:Most common shorthand notation: $\backslash displaystyle\; X\_n\; \backslash ,\; \backslash xrightarrow\; \backslash ,\; X$
;Convergence in probability: The sequence of random variables $X\_1,X\_2,\backslash dots\backslash ,$ is said to converge towards the random variable $X\backslash ,$ if $\backslash lim\_P\backslash left(\backslash left,\; X\_n-X\backslash \backslash geq\backslash varepsilon\backslash right)=0$ for every ε > 0.
:Most common shorthand notation: $\backslash displaystyle\; X\_n\; \backslash ,\; \backslash xrightarrow\; \backslash ,\; X$
;Strong convergence: The sequence of random variables $X\_1,X\_2,\backslash dots\backslash ,$ is said to converge towards the random variable $X\backslash ,$ if $P(\backslash lim\_\; X\_n=X)=1$. Strong convergence is also known as .
:Most common shorthand notation: $\backslash displaystyle\; X\_n\; \backslash ,\; \backslash xrightarrow\; \backslash ,\; X$
As the names indicate, weak convergence is weaker than strong convergence. In fact, strong convergence implies convergence in probability, and convergence in probability implies weak convergence. The reverse statements are not always true.
Law of large numbers

Common intuition suggests that if a fair coin is tossed many times, then ''roughly'' half of the time it will turn up ''heads'', and the other half it will turn up ''tails''. Furthermore, the more often the coin is tossed, the more likely it should be that the ratio of the number of ''heads'' to the number of ''tails'' will approach unity. Modern probability theory provides a formal version of this intuitive idea, known as the . This law is remarkable because it is not assumed in the foundations of probability theory, but instead emerges from these foundations as a theorem. Since it links theoretically derived probabilities to their actual frequency of occurrence in the real world, the law of large numbers is considered as a pillar in the history of statistical theory and has had widespread influence. The (LLN) states that the sample average :$\backslash overline\_n=\backslash frac1n$ of a sequence of independent and identically distributed random variables $X\_k$ converges towards their common expectation $\backslash mu$, provided that the expectation of $,\; X\_k,$ is finite. It is in the different forms ofconvergence of random variables
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressi ...

that separates the ''weak'' and the ''strong'' law of large numbers
:Weak law: $\backslash displaystyle\; \backslash overline\_n\; \backslash ,\; \backslash xrightarrow\; \backslash ,\; \backslash mu$ for $n\; \backslash to\; \backslash infty$
:Strong law: $\backslash displaystyle\; \backslash overline\_n\; \backslash ,\; \backslash xrightarrow\; \backslash ,\; \backslash mu$ for $n\; \backslash to\; \backslash infty\; .$
It follows from the LLN that if an event of probability ''p'' is observed repeatedly during independent experiments, the ratio of the observed frequency of that event to the total number of repetitions converges towards ''p''.
For example, if $Y\_1,Y\_2,...\backslash ,$ are independent taking values 1 with probability ''p'' and 0 with probability 1-''p'', then $\backslash textrm(Y\_i)=p$ for all ''i'', so that $\backslash bar\; Y\_n$ converges to ''p'' almost surely
In probability theory
Probability theory is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ...

.
Central limit theorem

"The central limit theorem (CLT) is one of the great results of mathematics." (Chapter 18 in, "Probability with martingales", Cambridge 1991/2008) It explains the ubiquitous occurrence of thenormal distribution
In probability theory
Probability theory is the branch of concerned with . Although there are several different , probability theory treats the concept in a rigorous mathematical manner by expressing it through a set of . Typically these ...

in nature.
The theorem states that the average
In colloquial language, an average is a single number taken as representative of a non-empty list of numbers. Different concepts of average are used in different contexts. Often "average" refers to the arithmetic mean, the sum of the numbers divide ...

of many independent and identically distributed random variables with finite variance tends towards a normal distribution ''irrespective'' of the distribution followed by the original random variables. Formally, let $X\_1,X\_2,\backslash dots\backslash ,$ be independent random variables with mean
There are several kinds of mean in mathematics, especially in statistics.
For a data set, the ''arithmetic mean'', also known as arithmetic average, is a central value of a finite set of numbers: specifically, the sum of the values divided by ...

$\backslash mu$ and variance
In probability theory
Probability theory is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ...

$\backslash sigma^2\; >\; 0.\backslash ,$ Then the sequence of random variables
:$Z\_n=\backslash frac\backslash ,$
converges in distribution to a standard normal
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by ex ...

random variable.
For some classes of random variables the classic central limit theorem works rather fast (see Berry–Esseen theorem), for example the distributions with finite first, second, and third moment from the exponential family
In theory of probability, probability and statistics, an exponential family is a parametric model, parametric set of probability distributions of a certain form, specified below. This special form is chosen for mathematical convenience, based on ...

; on the other hand, for some random variables of the heavy tail and fat tail
A fat-tailed distribution is a probability distribution that exhibits a large skewness or kurtosis, relative to that of either a normal distribution or an exponential distribution. In common usage, the terms fat-tailed and Heavy-tailed distributio ...

variety, it works very slowly or may not work at all: in such cases one may use the Generalized Central Limit Theorem (GCLT).
See also

* Catalog of articles in probability theory *Expected value
In probability theory
Probability theory is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and space ...

and Variance
In probability theory
Probability theory is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ...

* Fuzzy logic
In logic
Logic is an interdisciplinary field which studies truth and reasoning. Informal logic seeks to characterize Validity (logic), valid arguments informally, for instance by listing varieties of fallacies. Formal logic represents statem ...

and Fuzzy measure theoryIn mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ha ...

* Glossary of probability and statistics
The following is a glossary
__NOTOC__
A glossary (from grc, γλῶσσα / language, speech, wording) also known as a vocabulary or clavis, is an alphabetical list of terms in a particular domain of knowledge with the definitions for those ...

* Likelihood function
The likelihood function (often simply called the likelihood) describes the joint probability
Given random variables X,Y,\ldots, that are defined on a probability space, the joint probability distribution for X,Y,\ldots is a probability distribut ...

* List of probability topics
{{ProbabilityTopicsTOC
This is a list of probability topics, by Wikipedia page.
It overlaps with the (alphabetical) list of statistical topics. There are also the outline of probability and catalog of articles in probability theory. For distribution ...

* List of publications in statistics
* List of statistical topics
* Notation in probability
* Predictive modelling
Predictive modelling uses statistics to predict outcomes. Most often the event one wants to predict is in the future, but predictive modelling can be applied to any type of unknown event, regardless of when it occurred. For example, predictive model ...

* Probabilistic logic Probabilistic logic (also probability logic and probabilistic reasoning) involves the use of probability and logic to deal with uncertain situations. The result is a richer and more expressive formalism with a broad range of possible application are ...

– A combination of probability theory and logic
* Probabilistic proofs of non-probabilistic theorems
* Probability distribution
In probability theory
Probability theory is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related structures (algebra), shapes and spaces ...

* Probability axioms
The Kolmogorov axioms are the foundations of probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in ...

* Probability interpretations
The word probability has been used in a variety of ways since it was first applied to the mathematical study of games of chance. Does probability measure the real, physical, tendency of something to occur, or is it a measure of how strongly one be ...

* Probability space
In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a space (mathematics), mathematical construct that provides a formal model of a randomness, random process or "experiment". For example, one can define a ...

* Statistical independence
Independence is a fundamental notion in probability theory
Probability theory is the branch of mathematics
Mathematics (from Greek: ) includes the study of such topics as numbers (arithmetic and number theory), formulas and related stru ...

* Statistical physics
Statistical physics is a branch of physics
Physics (from grc, φυσική (ἐπιστήμη), physikḗ (epistḗmē), knowledge of nature, from ''phýsis'' 'nature'), , is the natural science that studies matter, its Motion (physics), ...

* Subjective logicSubjective logic is a type of probabilistic logic that explicitly takes epistemic uncertainty and source trust into account. In general, subjective logic is suitable for modeling and analysing situations involving uncertainty and relatively unreliabl ...

* Probability of the union of pairwise independent events
Notes

References

* :: The first major treatise blending calculus with probability theory, originally in French: ''Théorie Analytique des Probabilités''. * :: An English translation by Nathan Morrison appeared under the title ''Foundations of the Theory of Probability'' (Chelsea, New York) in 1950, with a second edition in 1956. * * Olav Kallenberg; ''Foundations of Modern Probability,'' 2nd ed. Springer Series in Statistics. (2002). 650 pp. * :: A lively introduction to probability theory for the beginner. * Olav Kallenberg; ''Probabilistic Symmetries and Invariance Principles''. Springer -Verlag, New York (2005). 510 pp. * {{DEFAULTSORT:Probability Theory id:Peluang (matematika)