![Dice Distribution (bar)](https://upload.wikimedia.org/wikipedia/commons/1/12/Dice_Distribution_%28bar%29.svg)
Probability is the branch of
mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
concerning numerical descriptions of how likely an
event
Event may refer to:
Gatherings of people
* Ceremony, an event of ritual significance, performed on a special occasion
* Convention (meeting), a gathering of individuals engaged in some common interest
* Event management, the organization of e ...
is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and 1, where, roughly speaking, 0 indicates impossibility of the event and 1 indicates certainty.
["Kendall's Advanced Theory of Statistics, Volume 1: Distribution Theory", Alan Stuart and Keith Ord, 6th Ed, (2009), .][William Feller, ''An Introduction to Probability Theory and Its Applications'', (Vol 1), 3rd Ed, (1968), Wiley, .] The higher the probability of an event, the more likely it is that the event will occur. A simple example is the tossing of a fair (unbiased) coin. Since the coin is fair, the two outcomes ("heads" and "tails") are both equally probable; the probability of "heads" equals the probability of "tails"; and since no other outcomes are possible, the probability of either "heads" or "tails" is 1/2 (which could also be written as 0.5 or 50%).
These concepts have been given an
axiomatic
An axiom, postulate, or assumption is a statement that is taken to be true, to serve as a premise or starting point for further reasoning and arguments. The word comes from the Ancient Greek word (), meaning 'that which is thought worthy or f ...
mathematical formalization in
probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
, which is used widely in
areas of study
Area is the quantity that expresses the extent of a region on the plane or on a curved surface. The area of a plane region or ''plane area'' refers to the area of a shape or planar lamina, while '' surface area'' refers to the area of an open s ...
such as
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
,
mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
,
science
Science is a systematic endeavor that builds and organizes knowledge in the form of testable explanations and predictions about the universe.
Science may be as old as the human species, and some of the earliest archeological evidence for ...
,
finance
Finance is the study and discipline of money, currency and capital assets. It is related to, but not synonymous with economics, the study of production, distribution, and consumption of money, assets, goods and services (the discipline of fina ...
,
gambling
Gambling (also known as betting or gaming) is the wagering of something of value ("the stakes") on a random event with the intent of winning something else of value, where instances of strategy are discounted. Gambling thus requires three el ...
,
artificial intelligence
Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech re ...
,
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
,
computer science
Computer science is the study of computation, automation, and information. Computer science spans theoretical disciplines (such as algorithms, theory of computation, information theory, and automation) to Applied science, practical discipli ...
,
game theory
Game theory is the study of mathematical models of strategic interactions among rational agents. Myerson, Roger B. (1991). ''Game Theory: Analysis of Conflict,'' Harvard University Press, p.&nbs1 Chapter-preview links, ppvii–xi It has appli ...
, and
philosophy
Philosophy (from , ) is the systematized study of general and fundamental questions, such as those about existence, reason, knowledge, values, mind, and language. Such questions are often posed as problems to be studied or resolved. Some ...
to, for example, draw inferences about the expected frequency of events. Probability theory is also used to describe the underlying mechanics and regularities of
complex systems
A complex system is a system composed of many components which may interact with each other. Examples of complex systems are Earth's global climate, organisms, the human brain, infrastructure such as power grid, transportation or communication s ...
.
Interpretations
When dealing with
experiment
An experiment is a procedure carried out to support or refute a hypothesis, or determine the efficacy or likelihood of something previously untried. Experiments provide insight into Causality, cause-and-effect by demonstrating what outcome oc ...
s that are
random
In common usage, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no :wikt:order, order and does not follow an intelligible pattern or combination. Ind ...
and
well-defined
In mathematics, a well-defined expression or unambiguous expression is an expression whose definition assigns it a unique interpretation or value. Otherwise, the expression is said to be ''not well defined'', ill defined or ''ambiguous''. A funct ...
in a purely theoretical setting (like tossing a coin), probabilities can be numerically described by the number of desired outcomes, divided by the total number of all outcomes. For example, tossing a coin twice will yield "head-head", "head-tail", "tail-head", and "tail-tail" outcomes. The probability of getting an outcome of "head-head" is 1 out of 4 outcomes, or, in numerical terms, 1/4, 0.25 or 25%. However, when it comes to practical application, there are two major competing categories of probability interpretations, whose adherents hold different views about the fundamental nature of probability:
*
Objectivists
Objectivism is a philosophical system developed by Russian-American writer and philosopher Ayn Rand. She described it as "the concept of man as a heroic being, with his own happiness as the moral purpose of his life, with productive achievement ...
assign numbers to describe some objective or physical state of affairs. The most popular version of objective probability is
frequentist probability
Frequentist probability or frequentism is an interpretation of probability; it defines an event's probability as the limit of its relative frequency in many trials (the long-run probability). Probabilities can be found (in principle) by a repea ...
, which claims that the probability of a random event denotes the ''relative frequency of occurrence'' of an experiment's outcome when the experiment is repeated indefinitely. This interpretation considers probability to be the relative frequency "in the long run" of outcomes. A modification of this is
propensity probability
The propensity theory of probability is a probability interpretation in which the probability is thought of as a physical propensity, disposition, or tendency of a given type of situation to yield an Outcome (probability), outcome of a certain kin ...
, which interprets probability as the tendency of some experiment to yield a certain outcome, even if it is performed only once.
*
Subjectivists assign numbers per subjective probability, that is, as a degree of belief. The degree of belief has been interpreted as "the price at which you would buy or sell a bet that pays 1 unit of utility if E, 0 if not E", although that interpretation is not universally agreed upon. The most popular version of subjective probability is
Bayesian probability
Bayesian probability is an Probability interpretations, interpretation of the concept of probability, in which, instead of frequentist probability, frequency or propensity probability, propensity of some phenomenon, probability is interpreted as re ...
, which includes expert knowledge as well as experimental data to produce probabilities. The expert knowledge is represented by some (subjective)
prior probability distribution
In Bayesian statistical inference, a prior probability distribution, often simply called the prior, of an uncertain quantity is the probability distribution that would express one's beliefs about this quantity before some evidence is taken int ...
. These data are incorporated in a
likelihood function
The likelihood function (often simply called the likelihood) represents the probability of random variable realizations conditional on particular values of the statistical parameters. Thus, when evaluated on a given sample, the likelihood funct ...
. The product of the prior and the likelihood, when normalized, results in a
posterior probability distribution
The posterior probability is a type of conditional probability that results from updating the prior probability with information summarized by the likelihood via an application of Bayes' rule. From an epistemological perspective, the posterior ...
that incorporates all the information known to date. By
Aumann's agreement theorem
Aumann's agreement theorem was stated and proved by Robert Aumann in a paper titled "Agreeing to Disagree", which introduced the set theoretic description of common knowledge. The theorem concerns agents who share a common prior and update their ...
, Bayesian agents whose prior beliefs are similar will end up with similar posterior beliefs. However, sufficiently different priors can lead to different conclusions, regardless of how much information the agents share.
Etymology
The word ''probability''
derives from the Latin , which can also mean "
probity", a measure of the
authority
In the fields of sociology and political science, authority is the legitimate power of a person or group over other people. In a civil state, ''authority'' is practiced in ways such a judicial branch or an executive branch of government.''The N ...
of a
witness
In law, a witness is someone who has knowledge about a matter, whether they have sensed it or are testifying on another witnesses' behalf. In law a witness is someone who, either voluntarily or under compulsion, provides testimonial evidence, e ...
in a
legal case
A legal case is in a general sense a dispute between opposing parties which may be resolved by a court, or by some equivalent legal process. A legal case is typically based on either civil or criminal law. In most legal cases there are one or mor ...
in
Europe
Europe is a large peninsula conventionally considered a continent in its own right because of its great physical size and the weight of its history and traditions. Europe is also considered a Continent#Subcontinents, subcontinent of Eurasia ...
, and often correlated with the witness's
nobility
Nobility is a social class found in many societies that have an aristocracy (class), aristocracy. It is normally ranked immediately below Royal family, royalty. Nobility has often been an Estates of the realm, estate of the realm with many e ...
. In a sense, this differs much from the modern meaning of ''probability'', which in contrast is a measure of the weight of
empirical evidence
Empirical evidence for a proposition is evidence, i.e. what supports or counters this proposition, that is constituted by or accessible to sense experience or experimental procedure. Empirical evidence is of central importance to the sciences and ...
, and is arrived at from
inductive reasoning
Inductive reasoning is a method of reasoning in which a general principle is derived from a body of observations. It consists of making broad generalizations based on specific observations. Inductive reasoning is distinct from ''deductive'' re ...
and
statistical inference
Statistical inference is the process of using data analysis to infer properties of an underlying probability distribution, distribution of probability.Upton, G., Cook, I. (2008) ''Oxford Dictionary of Statistics'', OUP. . Inferential statistical ...
.
[ Hacking, I. (2006) ''The Emergence of Probability: A Philosophical Study of Early Ideas about Probability, Induction and Statistical Inference'', Cambridge University Press, ]
History
The scientific study of probability is a modern development of
mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
.
Gambling shows that there has been an interest in quantifying the ideas of probability for millennia, but exact mathematical descriptions arose much later. There are reasons for the slow development of the mathematics of probability. Whereas games of chance provided the impetus for the mathematical study of probability, fundamental issues are still obscured by the superstitions of gamblers.
According to
Richard Jeffrey
Richard Carl Jeffrey (August 5, 1926 – November 9, 2002) was an American philosopher, logician, and probability theorist. He is best known for developing and championing the philosophy of radical probabilism and the associated heuristic of pr ...
, "Before the middle of the seventeenth century, the term 'probable' (Latin ''probabilis'') meant ''approvable'', and was applied in that sense, univocally, to opinion and to action. A probable action or opinion was one such as sensible people would undertake or hold, in the circumstances."
[Jeffrey, R.C., ''Probability and the Art of Judgment,'' Cambridge University Press. (1992). pp. 54–55 . ] However, in legal contexts especially, 'probable' could also apply to propositions for which there was good evidence.
[Franklin, J. (2001) ''The Science of Conjecture: Evidence and Probability Before Pascal,'' Johns Hopkins University Press. (pp. 22, 113, 127)]
![Christiaan Huygens-painting](https://upload.wikimedia.org/wikipedia/commons/a/a4/Christiaan_Huygens-painting.jpeg)
The sixteenth-century
Italian
Italian(s) may refer to:
* Anything of, from, or related to the people of Italy over the centuries
** Italians, an ethnic group or simply a citizen of the Italian Republic or Italian Kingdom
** Italian language, a Romance language
*** Regional Ita ...
polymath
Gerolamo Cardano
Gerolamo Cardano (; also Girolamo or Geronimo; french: link=no, Jérôme Cardan; la, Hieronymus Cardanus; 24 September 1501– 21 September 1576) was an Italian polymath, whose interests and proficiencies ranged through those of mathematician, ...
demonstrated the efficacy of defining
odds
Odds provide a measure of the likelihood of a particular outcome. They are calculated as the ratio of the number of events that produce that outcome to the number that do not. Odds are commonly used in gambling and statistics.
Odds also have ...
as the ratio of favourable to unfavourable outcomes (which implies that the probability of an event is given by the ratio of favourable outcomes to the total number of possible outcomes).
Aside from the elementary work by Cardano, the doctrine of probabilities dates to the correspondence of
Pierre de Fermat
Pierre de Fermat (; between 31 October and 6 December 1607 – 12 January 1665) was a French mathematician who is given credit for early developments that led to infinitesimal calculus, including his technique of adequality. In particular, he ...
and
Blaise Pascal
Blaise Pascal ( , , ; ; 19 June 1623 – 19 August 1662) was a French mathematician, physicist, inventor, philosopher, and Catholic Church, Catholic writer.
He was a child prodigy who was educated by his father, a tax collector in Rouen. Pa ...
(1654).
Christiaan Huygens
Christiaan Huygens, Lord of Zeelhem, ( , , ; also spelled Huyghens; la, Hugenius; 14 April 1629 – 8 July 1695) was a Dutch mathematician, physicist, engineer, astronomer, and inventor, who is regarded as one of the greatest scientists of ...
(1657) gave the earliest known scientific treatment of the subject.
Jakob Bernoulli
Jacob Bernoulli (also known as James or Jacques; – 16 August 1705) was one of the many prominent mathematicians in the Bernoulli family. He was an early proponent of Leibnizian calculus and sided with Gottfried Wilhelm Leibniz during the Leib ...
's ''
Ars Conjectandi
(Latin for "The Art of Conjecturing") is a book on combinatorics and mathematical probability written by Jacob Bernoulli and published in 1713, eight years after his death, by his nephew, Niklaus Bernoulli. The seminal work consolidated, apa ...
'' (posthumous, 1713) and
Abraham de Moivre
Abraham de Moivre FRS (; 26 May 166727 November 1754) was a French mathematician known for de Moivre's formula, a formula that links complex numbers and trigonometry, and for his work on the normal distribution and probability theory.
He moved ...
's ''
Doctrine of Chances
In law, the doctrine of chances is a rule of evidence that allows evidence to show that it is unlikely a defendant would be repeatedly, innocently involved in similar, suspicious circumstances.
Normally, under Federal Rule of Evidence
First ...
'' (1718) treated the subject as a branch of mathematics. See
Ian Hacking
Ian MacDougall Hacking (born February 18, 1936) is a Canadian philosopher specializing in the philosophy of science. Throughout his career, he has won numerous awards, such as the Killam Prize for the Humanities and the Balzan Prize, and been ...
's ''The Emergence of Probability''
[ and James Franklin's ''The Science of Conjecture'' for histories of the early development of the very concept of mathematical probability.
The ]theory of errors
In statistics, propagation of uncertainty (or propagation of error) is the effect of variables' uncertainties (or errors, more specifically random errors) on the uncertainty of a function based on them. When the variables are the values of exp ...
may be traced back to Roger Cotes
Roger Cotes (10 July 1682 – 5 June 1716) was an English mathematician, known for working closely with Isaac Newton by proofreading the second edition of his famous book, the '' Principia'', before publication. He also invented the quadratur ...
's ''Opera Miscellanea'' (posthumous, 1722), but a memoir prepared by Thomas Simpson
Thomas Simpson FRS (20 August 1710 – 14 May 1761) was a British mathematician and inventor known for the eponymous Simpson's rule to approximate definite integrals. The attribution, as often in mathematics, can be debated: this rule had been ...
in 1755 (printed 1756) first applied the theory to the discussion of errors of observation. The reprint (1757) of this memoir lays down the axioms that positive and negative errors are equally probable, and that certain assignable limits define the range of all errors. Simpson also discusses continuous errors and describes a probability curve.
The first two laws of error that were proposed both originated with Pierre-Simon Laplace
Pierre-Simon, marquis de Laplace (; ; 23 March 1749 – 5 March 1827) was a French scholar and polymath whose work was important to the development of engineering, mathematics, statistics, physics, astronomy, and philosophy. He summarized ...
. The first law was published in 1774, and stated that the frequency of an error could be expressed as an exponential function of the numerical magnitude of the error—disregarding sign. The second law of error was proposed in 1778 by Laplace, and stated that the frequency of the error is an exponential function of the square of the error.[Wilson EB (1923) "First and second laws of error". ]Journal of the American Statistical Association
The ''Journal of the American Statistical Association (JASA)'' is the primary journal published by the American Statistical Association, the main professional body for statisticians in the United States. It is published four times a year in March, ...
, 18, 143 The second law of error is called the normal distribution or the Gauss law. "It is difficult historically to attribute that law to Gauss, who in spite of his well-known precocity had probably not made this discovery before he was two years old."
Daniel Bernoulli
Daniel Bernoulli FRS (; – 27 March 1782) was a Swiss mathematician and physicist and was one of the many prominent mathematicians in the Bernoulli family from Basel. He is particularly remembered for his applications of mathematics to mechan ...
(1778) introduced the principle of the maximum product of the probabilities of a system of concurrent errors.
Adrien-Marie Legendre
Adrien-Marie Legendre (; ; 18 September 1752 – 9 January 1833) was a French mathematician who made numerous contributions to mathematics. Well-known and important concepts such as the Legendre polynomials and Legendre transformation are named ...
(1805) developed the method of least squares
The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the res ...
, and introduced it in his ''Nouvelles méthodes pour la détermination des orbites des comètes'' (''New Methods for Determining the Orbits of Comets''). In ignorance of Legendre's contribution, an Irish-American writer, Robert Adrain
Robert Adrain (30 September 1775 – 10 August 1843) was an Irish political exile who won renown as a mathematician in the United States. He left Ireland after leading republican insurgents in the Rebellion of 1798, and settled in New Jersey an ...
, editor of "The Analyst" (1808), first deduced the law of facility of error,
:
where is a constant depending on precision of observation, and is a scale factor ensuring that the area under the curve equals 1. He gave two proofs, the second being essentially the same as John Herschel
Sir John Frederick William Herschel, 1st Baronet (; 7 March 1792 – 11 May 1871) was an English polymath active as a mathematician, astronomer, chemist, inventor, experimental photographer who invented the blueprint and did botanical wor ...
's (1850). Gauss
Johann Carl Friedrich Gauss (; german: Gauß ; la, Carolus Fridericus Gauss; 30 April 177723 February 1855) was a German mathematician and physicist who made significant contributions to many fields in mathematics and science. Sometimes refer ...
gave the first proof that seems to have been known in Europe (the third after Adrain's) in 1809. Further proofs were given by Laplace (1810, 1812), Gauss (1823), James Ivory
James Francis Ivory (born June 7, 1928) is an American film director, producer, and screenwriter. For many years, he worked extensively with Indian-born film producer Ismail Merchant, his domestic as well as professional partner, and with screen ...
(1825, 1826), Hagen (1837), Friedrich Bessel
Friedrich Wilhelm Bessel (; 22 July 1784 – 17 March 1846) was a German astronomer, mathematician, physicist, and geodesist. He was the first astronomer who determined reliable values for the distance from the sun to another star by the method ...
(1838), W.F. Donkin (1844, 1856), and Morgan Crofton
Morgan Crofton (1826, Dublin, Ireland – 1915, Brighton, England) was an Irish mathematician who contributed to the field of geometric probability theory. He also worked with James Joseph Sylvester and contributed an article on probability to ...
(1870). Other contributors were Ellis
Ellis is a surname of Welsh and English origin. Retrieved 21 January 2014 An independent French origin of the surname is said to derive from the phrase fleur-de-lis.
Surname
A
* Abe Ellis (Stargate), a fictional character in the TV series ' ...
(1844), De Morgan De Morgan or de Morgan is a surname, and may refer to:
* Augustus De Morgan (1806–1871), British mathematician and logician.
** De Morgan's laws (or De Morgan's theorem), a set of rules from propositional logic.
** The De Morgan Medal, a trien ...
(1864), Glaisher Glaisher is a surname, and may refer to:
*Cecilia Glaisher (1828–1892), photographer and illustrator
*James Glaisher (1809–1903), English meteorologist and astronomer
*James Whitbread Lee Glaisher (1848–1928), English mathematician and astron ...
(1872), and Giovanni Schiaparelli
Giovanni Virginio Schiaparelli ( , also , ; 14 March 1835 – 4 July 1910) was an Italian astronomer and science historian.
Biography
He studied at the University of Turin, graduating in 1854, and later did research at Berlin Observatory, ...
(1875). Peters Peters may refer to:
People
* Peters (surname)
* Peters Band, a First Nations band in British Columbia, Canada
Places United States
* Peters, California, a census-designated place
* Peters, Florida, a town
* Peters Township, Kingman County, Kan ...
's (1856) formula for ''r'', the probable error In statistics, probable error defines the half-range of an interval about a central point for the distribution, such that half of the values from the distribution will lie within the interval and half outside.Dodge, Y. (2006) ''The Oxford Dictiona ...
of a single observation, is well known.
In the nineteenth century, authors on the general theory included Laplace, Sylvestre Lacroix Sylvestre can refer to:
People Surname
Given name
Middle name
* Carlos Sylvestre Begnis (1903–1980), Argentine medical doctor and politician
* Philippe Sylvestre Dufour (1622–1687), French Protestant apothecary, banker, collector, a ...
(1816), Littrow (1833), Adolphe Quetelet
Lambert Adolphe Jacques Quetelet FRSF or FRSE (; 22 February 1796 – 17 February 1874) was a Belgian astronomer, mathematician, statistician and sociologist who founded and directed the Brussels Observatory and was influential in introduc ...
(1853), Richard Dedekind
Julius Wilhelm Richard Dedekind (6 October 1831 – 12 February 1916) was a German mathematician who made important contributions to number theory, abstract algebra (particularly ring theory), and
the axiomatic foundations of arithmetic. His ...
(1860), Helmert (1872), Hermann Laurent
Paul Matthieu Hermann Laurent (2 September 1841, in Luxembourg City – 19 February 1908, in Paris, France) was a French mathematician. Despite his large body of works, Laurent series expansions for complex functions were ''not'' named after him, ...
(1873), Liagre, Didion and Karl Pearson
Karl Pearson (; born Carl Pearson; 27 March 1857 – 27 April 1936) was an English mathematician and biostatistician. He has been credited with establishing the discipline of mathematical statistics. He founded the world's first university st ...
. Augustus De Morgan and George Boole
George Boole (; 2 November 1815 – 8 December 1864) was a largely self-taught English mathematician, philosopher, and logician, most of whose short career was spent as the first professor of mathematics at Queen's College, Cork in Ire ...
improved the exposition of the theory.
In 1906, Andrey Markov
Andrey Andreyevich Markov, first name also spelled "Andrei", in older works also spelled Markoff) (14 June 1856 – 20 July 1922) was a Russian mathematician best known for his work on stochastic processes. A primary subject of his research lat ...
introduced the notion of Markov chains
A Markov chain or Markov process is a stochastic process, stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought ...
, which played an important role in stochastic process
In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that appea ...
es theory and its applications. The modern theory of probability based on the measure theory
In mathematics, the concept of a measure is a generalization and formalization of geometrical measures ( length, area, volume) and other common notions, such as mass and probability of events. These seemingly distinct concepts have many simil ...
was developed by Andrey Kolmogorov
Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
in 1931.
On the geometric side, contributors to ''The Educational Times'' included Miller, Crofton, McColl, Wolstenholme, Watson, and Artemas Martin
Artemas Martin (August 3, 1835 – November 7, 1918) was a self-educated American mathematician.
Biography
Martin was born on August 3, 1835, in Steuben County, New York, grew up in Venango County, Pennsylvania, and spent most of his life in E ...
. See integral geometry In mathematics, integral geometry is the theory of measures on a geometrical space invariant under the symmetry group of that space. In more recent times, the meaning has been broadened to include a view of invariant (or equivariant) transformation ...
for more information.
Theory
Like other theories
A theory is a rational type of abstract thinking about a phenomenon, or the results of such thinking. The process of contemplative and rational thinking is often associated with such processes as observational study or research. Theories may be s ...
, the theory of probability
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
is a representation of its concepts in formal terms—that is, in terms that can be considered separately from their meaning. These formal terms are manipulated by the rules of mathematics and logic, and any results are interpreted or translated back into the problem domain.
There have been at least two successful attempts to formalize probability, namely the Kolmogorov
Andrey Nikolaevich Kolmogorov ( rus, Андре́й Никола́евич Колмого́ров, p=ɐnˈdrʲej nʲɪkɐˈlajɪvʲɪtɕ kəlmɐˈɡorəf, a=Ru-Andrey Nikolaevich Kolmogorov.ogg, 25 April 1903 – 20 October 1987) was a Sovi ...
formulation and the Cox
Cox may refer to:
* Cox (surname), including people with the name
Companies
* Cox Enterprises, a media and communications company
** Cox Communications, cable provider
** Cox Media Group, a company that owns television and radio stations
** ...
formulation. In Kolmogorov's formulation (see also probability space
In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a mathematical construct that provides a formal model of a random process or "experiment". For example, one can define a probability space which models t ...
), sets are interpreted as events
Event may refer to:
Gatherings of people
* Ceremony, an event of ritual significance, performed on a special occasion
* Convention (meeting), a gathering of individuals engaged in some common interest
* Event management, the organization of ev ...
and probability as a measure
Measure may refer to:
* Measurement, the assignment of a number to a characteristic of an object or event
Law
* Ballot measure, proposed legislation in the United States
* Church of England Measure, legislation of the Church of England
* Mea ...
on a class of sets. In Cox's theorem
Cox's theorem, named after the physicist Richard Threlkeld Cox, is a derivation of the laws of probability theory from a certain set of postulates. This derivation justifies the so-called "logical" interpretation of probability, as the laws of p ...
, probability is taken as a primitive (i.e., not further analyzed), and the emphasis is on constructing a consistent assignment of probability values to propositions. In both cases, the laws of probability
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
are the same, except for technical details.
There are other methods for quantifying uncertainty, such as the Dempster–Shafer theory
The theory of belief functions, also referred to as evidence theory or Dempster–Shafer theory (DST), is a general framework for reasoning with uncertainty, with understood connections to other frameworks such as probability, possibility and i ...
or possibility theory
Possibility theory is a mathematical theory for dealing with certain types of uncertainty and is an alternative to probability theory. It uses measures of possibility and necessity between 0 and 1, ranging from impossible to possible and unnecessa ...
, but those are essentially different and not compatible with the usually-understood laws of probability.
Applications
Probability theory is applied in everyday life in risk
In simple terms, risk is the possibility of something bad happening. Risk involves uncertainty about the effects/implications of an activity with respect to something that humans value (such as health, well-being, wealth, property or the environme ...
assessment and modeling
A model is an informative representation of an object, person or system. The term originally denoted the plans of a building in late 16th-century English, and derived via French and Italian ultimately from Latin ''modulus'', a measure.
Models c ...
. The insurance industry and markets
Market is a term used to describe concepts such as:
* Market (economics), system in which parties engage in transactions according to supply and demand
* Market economy
*Marketplace, a physical marketplace or public market
Geography
*Märket, a ...
use actuarial science to determine pricing and make trading decisions. Governments apply probabilistic methods in environmental regulation
Environmental law is a collective term encompassing aspects of the law that provide protection to the environment. A related but distinct set of regulatory regimes, now strongly influenced by environmental Legal doctrine, legal principles, focu ...
, entitlement analysis, and financial regulation
Financial regulation is a form of regulation or supervision, which subjects financial institutions to certain requirements, restrictions and guidelines, aiming to maintain the stability and integrity of the financial system. This may be handled ...
.
An example of the use of probability theory in equity trading is the effect of the perceived probability of any widespread Middle East conflict on oil prices, which have ripple effects in the economy as a whole. An assessment by a commodity trader that a war is more likely can send that commodity's prices up or down, and signals other traders of that opinion. Accordingly, the probabilities are neither assessed independently nor necessarily rationally. The theory of behavioral finance
Behavioral economics studies the effects of psychological, cognitive, emotional, cultural and social factors on the decisions of individuals or institutions, such as how those decisions vary from those implied by classical economic theory.
...
emerged to describe the effect of such groupthink
Groupthink is a psychological phenomenon that occurs within a group of people in which the desire for harmony or conformity in the group results in an irrational or dysfunctional decision-making outcome. Cohesiveness, or the desire for cohesiveness ...
on pricing, on policy, and on peace and conflict.
In addition to financial assessment, probability can be used to analyze trends in biology (e.g., disease spread) as well as ecology (e.g., biological Punnett squares). As with finance, risk assessment can be used as a statistical tool to calculate the likelihood of undesirable events occurring, and can assist with implementing protocols to avoid encountering such circumstances. Probability is used to design games of chance
A game of chance is in contrast with a game of skill. It is a game whose outcome is strongly influenced by some randomizing device. Common devices used include dice, spinning tops, playing cards, roulette wheels, or numbered balls drawn from ...
so that casinos can make a guaranteed profit, yet provide payouts to players that are frequent enough to encourage continued play.
Another significant application of probability theory in everyday life is reliability
Reliability, reliable, or unreliable may refer to:
Science, technology, and mathematics Computing
* Data reliability (disambiguation), a property of some disk arrays in computer storage
* High availability
* Reliability (computer networking), a ...
. Many consumer products, such as automobiles
A car or automobile is a motor vehicle with wheels. Most definitions of ''cars'' say that they run primarily on roads, seat one to eight people, have four wheels, and mainly transport people instead of goods.
The year 1886 is regarded as ...
and consumer electronics, use reliability theory in product design to reduce the probability of failure. Failure probability may influence a manufacturer's decisions on a product's warranty
In contract law, a warranty is a promise which is not a condition of the contract or an innominate term: (1) it is a term "not going to the root of the contract",Hogg M. (2011). ''Promises and Contract Law: Comparative Perspectives''p. 48 Cambrid ...
.
The cache language model and other statistical language models that are used in natural language processing
Natural language processing (NLP) is an interdisciplinary subfield of linguistics, computer science, and artificial intelligence concerned with the interactions between computers and human language, in particular how to program computers to pro ...
are also examples of applications of probability theory.
Mathematical treatment
Consider an experiment that can produce a number of results. The collection of all possible results is called the sample space
In probability theory, the sample space (also called sample description space, possibility space, or outcome space) of an experiment or random trial is the set of all possible outcomes or results of that experiment. A sample space is usually den ...
of the experiment, sometimes denoted as . The power set
In mathematics, the power set (or powerset) of a set is the set of all subsets of , including the empty set and itself. In axiomatic set theory (as developed, for example, in the ZFC axioms), the existence of the power set of any set is po ...
of the sample space is formed by considering all different collections of possible results. For example, rolling a die can produce six possible results. One collection of possible results gives an odd number on the die. Thus, the subset is an element of the power set
In mathematics, the power set (or powerset) of a set is the set of all subsets of , including the empty set and itself. In axiomatic set theory (as developed, for example, in the ZFC axioms), the existence of the power set of any set is po ...
of the sample space of dice rolls. These collections are called "events". In this case, is the event that the die falls on some odd number. If the results that actually occur fall in a given event, the event is said to have occurred.
A probability is a way of assigning every event a value between zero and one, with the requirement that the event made up of all possible results (in our example, the event ) is assigned a value of one. To qualify as a probability, the assignment of values must satisfy the requirement that for any collection of mutually exclusive events (events with no common results, such as the events , , and ), the probability that at least one of the events will occur is given by the sum of the probabilities of all the individual events.
The probability of an event
Event may refer to:
Gatherings of people
* Ceremony, an event of ritual significance, performed on a special occasion
* Convention (meeting), a gathering of individuals engaged in some common interest
* Event management, the organization of e ...
''A'' is written as , , or . This mathematical definition of probability can extend to infinite sample spaces, and even uncountable sample spaces, using the concept of a measure.
The ''opposite'' or ''complement'' of an event ''A'' is the event ot ''A''(that is, the event of ''A'' not occurring), often denoted as , , or ; its probability is given by . As an example, the chance of not rolling a six on a six-sided die is . For a more comprehensive treatment, see Complementary event
In probability theory, the complement of any event ''A'' is the event ot ''A'' i.e. the event that ''A'' does not occur.Robert R. Johnson, Patricia J. Kuby: ''Elementary Statistics''. Cengage Learning 2007, , p. 229 () The event ''A'' and ...
.
If two events ''A'' and ''B'' occur on a single performance of an experiment, this is called the intersection or joint probability
Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered ...
of ''A'' and ''B'', denoted as .
Independent events
If two events, ''A'' and ''B'' are independent
Independent or Independents may refer to:
Arts, entertainment, and media Artist groups
* Independents (artist group), a group of modernist painters based in the New Hope, Pennsylvania, area of the United States during the early 1930s
* Independ ...
then the joint probability is
:
For example, if two coins are flipped, then the chance of both being heads is .
Mutually exclusive events
If either event ''A'' or event ''B'' can occur but never both simultaneously, then they are called mutually exclusive events.
If two events are mutually exclusive
In logic and probability theory, two events (or propositions) are mutually exclusive or disjoint if they cannot both occur at the same time. A clear example is the set of outcomes of a single coin toss, which can result in either heads or tails ...
, then the probability of ''both'' occurring is denoted as and
:
If two events are mutually exclusive
In logic and probability theory, two events (or propositions) are mutually exclusive or disjoint if they cannot both occur at the same time. A clear example is the set of outcomes of a single coin toss, which can result in either heads or tails ...
, then the probability of ''either'' occurring is denoted as and
:
For example, the chance of rolling a 1 or 2 on a six-sided is
Not mutually exclusive events
If the events are not mutually exclusive then
:
For example, when drawing a card from a deck of cards, the chance of getting a heart or a face card (J,Q,K) (or both) is , since among the 52 cards of a deck, 13 are hearts, 12 are face cards, and 3 are both: here the possibilities included in the "3 that are both" are included in each of the "13 hearts" and the "12 face cards", but should only be counted once.
Conditional probability
''Conditional probability
In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occur ...
'' is the probability of some event ''A'', given the occurrence of some other event ''B''.
Conditional probability is written , and is read "the probability of ''A'', given ''B''". It is defined by
:
If then is formally undefined
Undefined may refer to:
Mathematics
* Undefined (mathematics), with several related meanings
** Indeterminate form, in calculus
Computing
* Undefined behavior, computer code whose behavior is not specified under certain conditions
* Undefined ...
by this expression. In this case and are independent, since . However, it is possible to define a conditional probability for some zero-probability events using a σ-algebra
In mathematical analysis and in probability theory, a σ-algebra (also σ-field) on a set ''X'' is a collection Σ of subsets of ''X'' that includes the empty subset, is closed under complement, and is closed under countable unions and countabl ...
of such events (such as those arising from a continuous random variable
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon ...
).
For example, in a bag of 2 red balls and 2 blue balls (4 balls in total), the probability of taking a red ball is ; however, when taking a second ball, the probability of it being either a red ball or a blue ball depends on the ball previously taken. For example, if a red ball was taken, then the probability of picking a red ball again would be , since only 1 red and 2 blue balls would have been remaining. And if a blue ball was taken previously, the probability of taking a red ball will be .
Inverse probability
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
and applications, ''Bayes' rule'' relates the odds
Odds provide a measure of the likelihood of a particular outcome. They are calculated as the ratio of the number of events that produce that outcome to the number that do not. Odds are commonly used in gambling and statistics.
Odds also have ...
of event to event , before (prior to) and after (posterior to) conditioning Conditioning may refer to:
Science, computing, and technology
* Air conditioning, the removal of heat from indoor air for thermal comfort
** Automobile air conditioning, air conditioning in a vehicle
** Ice storage air conditioning, air condition ...
on another event . The odds on to event is simply the ratio of the probabilities of the two events. When arbitrarily many events are of interest, not just two, the rule can be rephrased as ''posterior is proportional to prior times likelihood'', where the proportionality symbol means that the left hand side is proportional to (i.e., equals a constant times) the right hand side as varies, for fixed or given (Lee, 2012; Bertsch McGrayne, 2012). In this form it goes back to Laplace (1774) and to Cournot (1843); see Fienberg (2005). See Inverse probability
In probability theory, inverse probability is an obsolete term for the probability distribution of an unobserved variable.
Today, the problem of determining an unobserved variable (by whatever method) is called inferential statistics, the method o ...
and Bayes' rule
In probability theory and statistics, Bayes' theorem (alternatively Bayes' law or Bayes' rule), named after Thomas Bayes, describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For examp ...
.
Summary of probabilities
Relation to randomness and probability in quantum mechanics
In a deterministic
Determinism is a philosophical view, where all events are determined completely by previously existing causes. Deterministic theories throughout the history of philosophy have developed from diverse and sometimes overlapping motives and consi ...
universe, based on Newtonian concepts, there would be no probability if all conditions were known (Laplace's demon
In the history of science, Laplace's demon was a notable published articulation of causal determinism on a scientific basis by Pierre-Simon Laplace in 1814. According to determinism, if someone (the demon) knows the precise location and moment ...
), (but there are situations in which sensitivity to initial conditions exceeds our ability to measure them, i.e. know them). In the case of a roulette
Roulette is a casino game named after the French word meaning ''little wheel'' which was likely developed from the Italian game Biribi''.'' In the game, a player may choose to place a bet on a single number, various groupings of numbers, the ...
wheel, if the force of the hand and the period of that force are known, the number on which the ball will stop would be a certainty (though as a practical matter, this would likely be true only of a roulette wheel that had not been exactly levelled – as Thomas A. Bass' Newtonian Casino revealed). This also assumes knowledge of inertia and friction of the wheel, weight, smoothness, and roundness of the ball, variations in hand speed during the turning, and so forth. A probabilistic description can thus be more useful than Newtonian mechanics for analyzing the pattern of outcomes of repeated rolls of a roulette wheel. Physicists face the same situation in the kinetic theory of gases
Kinetic (Ancient Greek: κίνησις “kinesis”, movement or to move) may refer to:
* Kinetic theory, describing a gas as particles in random motion
* Kinetic energy, the energy of an object that it possesses due to its motion
Art and enter ...
, where the system, while deterministic ''in principle'', is so complex (with the number of molecules typically the order of magnitude of the Avogadro constant
The Avogadro constant, commonly denoted or , is the proportionality factor that relates the number of constituent particles (usually molecules, atoms or ions) in a sample with the amount of substance in that sample. It is an SI defining con ...
) that only a statistical description of its properties is feasible.
Probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
is required to describe quantum phenomena. A revolutionary discovery of early 20th century physics
Physics is the natural science that studies matter, its fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force. "Physical science is that department of knowledge which r ...
was the random character of all physical processes that occur at sub-atomic scales and are governed by the laws of quantum mechanics
Quantum mechanics is a fundamental theory in physics that provides a description of the physical properties of nature at the scale of atoms and subatomic particles. It is the foundation of all quantum physics including quantum chemistry, ...
. The objective wave function
A wave function in quantum physics is a mathematical description of the quantum state of an isolated quantum system. The wave function is a complex-valued probability amplitude, and the probabilities for the possible results of measurements mad ...
evolves deterministically but, according to the Copenhagen interpretation
The Copenhagen interpretation is a collection of views about the meaning of quantum mechanics, principally attributed to Niels Bohr and Werner Heisenberg. It is one of the oldest of numerous proposed interpretations of quantum mechanics, as featu ...
, it deals with probabilities of observing, the outcome being explained by a wave function collapse
In quantum mechanics, wave function collapse occurs when a wave function—initially in a quantum superposition, superposition of several eigenstates—reduces to a single eigenstate due to interaction with the external world. This interaction is ...
when an observation is made. However, the loss of determinism
Determinism is a philosophical view, where all events are determined completely by previously existing causes. Deterministic theories throughout the history of philosophy have developed from diverse and sometimes overlapping motives and consi ...
for the sake of instrumentalism
In philosophy of science and in epistemology, instrumentalism is a methodological view that ideas are useful instruments, and that the worth of an idea is based on how effective it is in explaining and predicting phenomena.
According to instrumenta ...
did not meet with universal approval. Albert Einstein
Albert Einstein ( ; ; 14 March 1879 – 18 April 1955) was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. Einstein is best known for developing the theory ...
famously remarked in a letter to Max Born
Max Born (; 11 December 1882 – 5 January 1970) was a German physicist and mathematician who was instrumental in the development of quantum mechanics. He also made contributions to solid-state physics and optics and supervised the work of a n ...
: "I am convinced that God does not play dice". Like Einstein, Erwin Schrödinger
Erwin Rudolf Josef Alexander Schrödinger (, ; ; 12 August 1887 – 4 January 1961), sometimes written as or , was a Nobel Prize-winning Austrian physicist with Irish citizenship who developed a number of fundamental results in quantum theory ...
, who discovered the wave function, believed quantum mechanics is a statistical
Statistics (from German: ''Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industria ...
approximation of an underlying deterministic reality
Reality is the sum or aggregate of all that is real or existent within a system, as opposed to that which is only imaginary. The term is also used to refer to the ontological status of things, indicating their existence. In physical terms, r ...
. In some modern interpretations of the statistical mechanics of measurement, quantum decoherence
Quantum decoherence is the loss of quantum coherence. In quantum mechanics, particles such as electrons are described by a wave function, a mathematical representation of the quantum state of a system; a probabilistic interpretation of the wave ...
is invoked to account for the appearance of subjectively probabilistic experimental outcomes.
See also
* Chance (disambiguation)
Chance may refer to:
Mathematics and Science
* In mathematics, likelihood of something (by way of the Likelihood function and/or Probability density function).
* Chance (statistics magazine), ''Chance'' (statistics magazine)
Places
* Chance, ...
* Class membership probabilities
In machine learning, a probabilistic classifier is a classifier that is able to predict, given an observation of an input, a probability distribution over a set of classes, rather than only outputting the most likely class that the observation sh ...
* Contingency
* Equiprobability
Equiprobability is a property for a collection of events that each have the same probability of occurring. In statistics and probability theory it is applied in the discrete uniform distribution and the equidistribution theorem for rational num ...
* Heuristics in judgment and decision-making
Heuristics is the process by which humans use mental short cuts to arrive at decisions. Heuristics are simple strategies that humans, animals, organizations, and even machines use to quickly form judgments, make decisions, and find solutions to c ...
* Probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
* Randomness
In common usage, randomness is the apparent or actual lack of pattern or predictability in events. A random sequence of events, symbols or steps often has no order and does not follow an intelligible pattern or combination. Individual rand ...
* Statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
* Estimators
In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule (the estimator), the quantity of interest (the estimand) and its result (the estimate) are distinguished. For example, the ...
* Estimation theory
Estimation theory is a branch of statistics that deals with estimating the values of parameters based on measured empirical data that has a random component. The parameters describe an underlying physical setting in such a way that their valu ...
* Probability density estimation
In statistics, probability density estimation or simply density estimation is the construction of an estimate, based on observed data, of an unobservable underlying probability density function. The unobservable density function is thought of ...
* Probability density function
In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
* Pairwise independence In probability theory, a pairwise independent collection of random variables is a set of random variables any two of which are independent. Any collection of mutually independent random variables is pairwise independent, but some pairwise independe ...
;In law
* Balance of probabilities
In a legal dispute, one party has the burden of proof to show that they are correct, while the other party had no such burden and is presumed to be correct. The burden of proof requires a party to produce evidence to establish the truth of facts ...
Notes
References
Bibliography
* Kallenberg, O. (2005) ''Probabilistic Symmetries and Invariance Principles''. Springer-Verlag, New York. 510 pp.
* Kallenberg, O. (2002) ''Foundations of Modern Probability,'' 2nd ed. Springer Series in Statistics. 650 pp.
* Olofsson, Peter (2005) ''Probability, Statistics, and Stochastic Processes'', Wiley-Interscience. 504 pp .
External links
Virtual Laboratories in Probability and Statistics (Univ. of Ala.-Huntsville)
*
Probability and Statistics EBook
* Edwin Thompson Jaynes
Edwin Thompson Jaynes (July 5, 1922 – April 30, 1998) was the Wayman Crow Distinguished Professor of Physics at Washington University in St. Louis. He wrote extensively on statistical mechanics and on foundations of probability and statistic ...
. ''Probability Theory: The Logic of Science''. Preprint: Washington University, (1996).
HTML index with links to PostScript files
an
PDF
(first three chapters)
* ttp://www.economics.soton.ac.uk/staff/aldrich/Probability%20Earliest%20Uses.htm Probability and Statistics on the Earliest Uses Pages (Univ. of Southampton)
Earliest Uses of Symbols in Probability and Statistics
o
A tutorial on probability and Bayes' theorem devised for first-year Oxford University students
pdf file of An Anthology of Chance Operations
''An Anthology of Chance Operations'' (An Anthology) was an artist's book publication from the early 1960s of experimental neodada art and music composition that used John Cage–inspired indeterminacy. It was edited by La Monte Young and DIY ...
(1963) at UbuWeb
UbuWeb is a web-based educational resource for avant-garde material available on the internet, founded in 1996 by poet Kenneth Goldsmith. It offers visual, concrete and sound poetry, expanding to include film and sound art mp3 archives.
Philosop ...
Introduction to Probability – eBook
, by Charles Grinstead, Laurie Snel
Source
''(GNU Free Documentation License
The GNU Free Documentation License (GNU FDL or simply GFDL) is a copyleft license for free documentation, designed by the Free Software Foundation (FSF) for the GNU Project. It is similar to the GNU General Public License, giving readers the r ...
)''
* Bruno de Finetti
Bruno de Finetti (13 June 1906 – 20 July 1985) was an Italian probabilist statistician and actuary, noted for the "operational subjective" conception of probability. The classic exposition of his distinctive theory is the 1937 "La prévision: ...
,
Probabilità e induzione
', Bologna, CLUEB, 1993. (digital version)
{{Authority control
Dimensionless numbers