HOME

TheInfoList



OR:

In
probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...
and
statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, a conditional variance is the
variance In probability theory and statistics, variance is the expectation of the squared deviation of a random variable from its population mean or sample mean. Variance is a measure of dispersion, meaning it is a measure of how far a set of numbers ...
of a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
given the value(s) of one or more other variables. Particularly in
econometrics Econometrics is the application of Statistics, statistical methods to economic data in order to give Empirical evidence, empirical content to economic relationships.M. Hashem Pesaran (1987). "Econometrics," ''The New Palgrave: A Dictionary of ...
, the conditional variance is also known as the scedastic function or skedastic function. Conditional variances are important parts of
autoregressive conditional heteroskedasticity In econometrics, the autoregressive conditional heteroskedasticity (ARCH) model is a statistical model for time series data that describes the variance of the current error term or innovation as a function of the actual sizes of the previous ti ...
(ARCH) models.


Definition

The conditional variance of a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
''Y'' given another random variable ''X'' is :\operatorname(Y, X) = \operatorname\Big(\big(Y - \operatorname(Y\mid X)\big)^\mid X\Big). The conditional variance tells us how much variance is left if we use \operatorname(Y\mid X) to "predict" ''Y''. Here, as usual, \operatorname(Y\mid X) stands for the
conditional expectation In probability theory, the conditional expectation, conditional expected value, or conditional mean of a random variable is its expected value – the value it would take “on average” over an arbitrarily large number of occurrences – give ...
of ''Y'' given ''X'', which we may recall, is a random variable itself (a function of ''X'', determined up to probability one). As a result, \operatorname(Y, X) itself is a random variable (and is a function of ''X'').


Explanation, relation to least-squares

Recall that variance is the expected squared deviation between a random variable (say, ''Y'') and its expected value. The expected value can be thought of as a reasonable prediction of the outcomes of the random experiment (in particular, the expected value is the best constant prediction when predictions are assessed by expected squared prediction error). Thus, one interpretation of variance is that it gives the smallest possible expected squared prediction error. If we have the knowledge of another random variable (''X'') that we can use to predict ''Y'', we can potentially use this knowledge to reduce the expected squared error. As it turns out, the best prediction of ''Y'' given ''X'' is the conditional expectation. In particular, for any f: \mathbb \to \mathbb measurable, : \begin \operatorname (Y-f(X))^2 &= \operatorname X)\,\,+\,\, \operatorname(Y, X)-f(X) )^2 \\ &= \operatorname \operatorname\ \\ &= \operatorname X )+ \operatorname X)-f(X))^2,. \end By selecting f(X)=\operatorname(Y, X), the second, nonnegative term becomes zero, showing the claim. Here, the second equality used the
law of total expectation The proposition in probability theory known as the law of total expectation, the law of iterated expectations (LIE), Adam's law, the tower rule, and the smoothing theorem, among other names, states that if X is a random variable whose expected v ...
. We also see that the expected conditional variance of ''Y'' given ''X'' shows up as the irreducible error of predicting ''Y'' given only the knowledge of ''X''.


Special cases, variations


Conditioning on discrete random variables

When ''X'' takes on countable many values S = \ with positive probability, i.e., it is a
discrete random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
, we can introduce \operatorname(Y, X=x), the conditional variance of ''Y'' given that ''X=x'' for any ''x'' from ''S'' as follows: :\operatorname(Y, X=x) = \operatorname((Y - \operatorname(Y\mid X=x))^\mid X=x), where recall that \operatorname(Z\mid X=x) is the conditional expectation of ''Z'' given that ''X=x'', which is well-defined for x\in S. An alternative notation for \operatorname(Y, X=x) is \operatorname_(Y, x). Note that here \operatorname(Y, X=x) defines a constant for possible values of ''x'', and in particular, \operatorname(Y, X=x), is ''not'' a random variable. The connection of this definition to \operatorname(Y, X) is as follows: Let ''S'' be as above and define the function v: S \to \mathbb as v(x) = \operatorname(Y, X=x). Then, v(X) = \operatorname(Y, X)
almost surely In probability theory, an event is said to happen almost surely (sometimes abbreviated as a.s.) if it happens with probability 1 (or Lebesgue measure 1). In other words, the set of possible exceptions may be non-empty, but it has probability 0. ...
.


Definition using conditional distributions

The "conditional expectation of ''Y'' given ''X=x''" can also be defined more generally using the
conditional distribution In probability theory and statistics, given two jointly distributed random variables X and Y, the conditional probability distribution of Y given X is the probability distribution of Y when X is known to be a particular value; in some cases the ...
of ''Y'' given ''X'' (this exists in this case, as both here ''X'' and ''Y'' are real-valued). In particular, letting P_ be the (regular)
conditional distribution In probability theory and statistics, given two jointly distributed random variables X and Y, the conditional probability distribution of Y given X is the probability distribution of Y when X is known to be a particular value; in some cases the ...
P_ of ''Y'' given ''X'', i.e., P_:\mathcal \times \mathbb\to ,1/math> (the intention is that P_(U,x) = P(Y\in U, X=x) almost surely over the support of ''X''), we can define \operatorname(Y, X=x) = \int \left(y- \int y' P_(dy', x)\right)^2 P_(dy, x). This can, of course, be specialized to when ''Y'' is discrete itself (replacing the integrals with sums), and also when the conditional density of ''Y'' given ''X=x'' with respect to some underlying distribution exists.


Components of variance

The
law of total variance In probability theory, the law of total variance or variance decomposition formula or conditional variance formulas or law of iterated variances also known as Eve's law, states that if X and Y are random variables on the same probability space, and ...
says \operatorname(Y) = \operatorname(\operatorname(Y\mid X))+\operatorname(\operatorname(Y\mid X)). In words: the variance of ''Y'' is the sum of the expected conditional variance of ''Y'' given ''X'' and the variance of the conditional expectation of ''Y'' given ''X''. The first term captures the variation left after "using ''X'' to predict ''Y''", while the second term captures the variation due to the mean of the prediction of ''Y'' due to the randomness of ''X''.


See also

*
Mixed model A mixed model, mixed-effects model or mixed error-component model is a statistical model containing both fixed effects and random effects. These models are useful in a wide variety of disciplines in the physical, biological and social sciences. ...
*
Random effects model In statistics, a random effects model, also called a variance components model, is a statistical model where the model parameters are random variables. It is a kind of hierarchical linear model, which assumes that the data being analysed are dra ...


References


Further reading

* Statistical deviation and dispersion Theory of probability distributions Conditional probability {{probability-stub