probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...

and

statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...

, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous

probability distribution In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...

s. It is the

conjugate prior In Bayesian probability theory, if the posterior distribution p(\theta \mid x) is in the same probability distribution family as the prior probability distribution p(\theta), the prior and posterior are then called conjugate distributions, and th ...

of a

multivariate normal distribution In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One d ...

with unknown

mean There are several kinds of mean in mathematics, especially in statistics. Each mean serves to summarize a given group of data, often to better understand the overall value (magnitude and sign) of a given data set. For a data set, the ''arithme ...

and

precision matrix In statistics, the precision matrix or concentration matrix is the matrix inverse of the covariance matrix or dispersion matrix, P = \Sigma^. For univariate distributions, the precision matrix degenerates into a scalar precision, defined as the r ...

(the inverse of the

covariance matrix In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of ...

).Bishop, Christopher M. (2006). ''Pattern Recognition and Machine Learning.'' Springer Science+Business Media. Page 690.

Definition

Suppose :

\boldsymbol\mu, \boldsymbol\mu_0,\lambda,\boldsymbol\Lambda \sim \mathcal(\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^)

has a

with

\boldsymbol\mu_0

and

(\lambda\boldsymbol\Lambda)^

, where :

\boldsymbol\Lambda, \mathbf,\nu \sim \mathcal(\boldsymbol\Lambda, \mathbf,\nu)

has a

Wishart distribution In statistics, the Wishart distribution is a generalization to multiple dimensions of the gamma distribution. It is named in honor of John Wishart, who first formulated the distribution in 1928. It is a family of probability distributions define ...

. Then

(\boldsymbol\mu,\boldsymbol\Lambda)

has a normal-Wishart distribution, denoted as :

(\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm(\boldsymbol\mu_0,\lambda,\mathbf,\nu) .

Characterization

Probability density function

f(\boldsymbol\mu,\boldsymbol\Lambda, \boldsymbol\mu_0,\lambda,\mathbf,\nu) = \mathcal(\boldsymbol\mu, \boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^)\ \mathcal(\boldsymbol\Lambda, \mathbf,\nu)

Properties

Scaling

Marginal distributions

By construction, the

marginal distribution In probability theory and statistics, the marginal distribution of a subset of a collection of random variables is the probability distribution of the variables contained in the subset. It gives the probabilities of various values of the variables ...

over

\boldsymbol\Lambda

is a

, and the

conditional distribution In probability theory and statistics, given two jointly distributed random variables X and Y, the conditional probability distribution of Y given X is the probability distribution of Y when X is known to be a particular value; in some cases the co ...

over

\boldsymbol\mu

given

\boldsymbol\Lambda

is a

. The

over

\boldsymbol\mu

is a multivariate ''t''-distribution.

Posterior distribution of the parameters

After making

n

observations

\boldsymbol_1, \dots, \boldsymbol_n

, the posterior distribution of the parameters is :

(\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm(\boldsymbol\mu_n,\lambda_n,\mathbf_n,\nu_n),

where :

\lambda_n = \lambda + n,

\boldsymbol\mu_n = \frac,

\nu_n = \nu + n,

\mathbf_n^ = \mathbf^ + \sum_^n (\boldsymbol_i - \boldsymbol)(\boldsymbol_i - \boldsymbol)^T + \frac (\boldsymbol - \boldsymbol\mu_0)(\boldsymbol - \boldsymbol\mu_0)^T.

Cross Validated, https://stats.stackexchange.com/q/324925

Generating normal-Wishart random variates

Generation of random variates is straightforward: # Sample

\boldsymbol\Lambda

from a

with parameters

\mathbf

and

\nu

# Sample

\boldsymbol\mu

from a

with mean

\boldsymbol\mu_0

and variance

(\lambda\boldsymbol\Lambda)^

Related distributions

* The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision. * The

normal-gamma distribution In probability theory and statistics, the normal-gamma distribution (or Gaussian-gamma distribution) is a bivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean ...

is the one-dimensional equivalent. * The

and

are the component distributions out of which this distribution is made.

Notes

References

* Bishop, Christopher M. (2006). ''Pattern Recognition and Machine Learning.'' Springer Science+Business Media. {{DEFAULTSORT:Normal-Wishart Distribution Multivariate continuous distributions Conjugate prior distributions Normal distribution