In
probability
Probability is the branch of mathematics concerning numerical descriptions of how likely an Event (probability theory), event is to occur, or how likely it is that a proposition is true. The probability of an event is a number between 0 and ...
theory, the chain rule (also called the general product rule
) permits the calculation of any member of the
joint distribution
Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered ...
of a set of
random variables using only
conditional probabilities
In probability theory, conditional probability is a measure of the probability of an event occurring, given that another event (by assumption, presumption, assertion or evidence) has already occurred. This particular method relies on event B occu ...
. The rule is useful in the study of
Bayesian network
A Bayesian network (also known as a Bayes network, Bayes net, belief network, or decision network) is a probabilistic graphical model that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG). Bay ...
s, which describe a probability distribution in terms of conditional probabilities.
Chain rule for events
Two events
The chain rule for two random
events
Event may refer to:
Gatherings of people
* Ceremony, an event of ritual significance, performed on a special occasion
* Convention (meeting), a gathering of individuals engaged in some common interest
* Event management, the organization of ev ...
and
says
Example
This rule is illustrated in the following example. Urn 1 has 1 black ball and 2 white balls and Urn 2 has 1 black ball and 3 white balls. Suppose we pick an urn at random and then select a ball from that urn. Let event
be choosing the first urn:
Let event
be the chance we choose a white ball. The chance of choosing a white ball, given that we have chosen the first urn, is
Event
would be their intersection: choosing the first urn and a white ball from it. The probability can be found by the chain rule for probability:
More than two events
For more than two events
the chain rule extends to the formula
which by induction may be turned into
Example
With four events (
), the chain rule is
Chain rule for random variables
Two random variables
For two random variables
, to find the joint distribution, we can apply the definition of conditional probability to obtain:
for any possible values
of
and
of
in the discrete case or, in general,
for any possible measurable sets
and
.
If one desires a notation for the probability distribution of
, one can use
, so that
in the discrete case or, in general,
for a measurable set
.
Note: in the examples below, it is meaningless to write
for a single random variable
or multiple random variables. We have left them as an earlier editor wrote them to provide an example to warn against this incomplete notation. It is particularly egregious to write intersections of random variables.
More than two random variables
Consider an indexed collection of random variables
. To find the value of this member of the joint distribution, we can apply the definition of conditional probability to obtain:
Repeating this process with each final term creates the product:
Example
With four variables (
), the chain rule produces this product of conditional probabilities:
See also
*
References
* {{Russell Norvig 2003, p. 496.
"The Chain Rule of Probability" ''
developerWorks'', Nov 3, 2012.
Bayesian inference
Bayesian statistics
Mathematical identities
Probability theory