In
probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set ...
, particularly
information theory, the conditional mutual information
is, in its most basic form, the
expected value of the
mutual information
In probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two variables. More specifically, it quantifies the " amount of information" (in units such ...
of two random variables given the value of a third.
Definition
For random variables
,
, and
with
support sets ,
and
, we define the conditional mutual information as
This may be written in terms of the expectation operator: