In
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the matrix normal distribution or matrix Gaussian distribution is a
probability distribution
In probability theory and statistics, a probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It is a mathematical description of a random phenomenon i ...
that is a generalization of the
multivariate normal distribution
In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One d ...
to matrix-valued random variables.
Definition
The
probability density function
In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can ...
for the random matrix X (''n'' × ''p'') that follows the matrix normal distribution
has the form:
:
where
denotes
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
and M is ''n'' × ''p'', U is ''n'' × ''n'' and V is ''p'' × ''p'', and the density is understood as the probability density function with respect to the standard Lebesgue measure in
, i.e.: the measure corresponding to integration with respect to
.
The matrix normal is related to the
multivariate normal distribution
In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One d ...
in the following way:
:
if and only if
:
where
denotes the
Kronecker product
In mathematics, the Kronecker product, sometimes denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a block matrix. It is a generalization of the outer product (which is denoted by the same symbol) from vectors to ...
and
denotes the
vectorization
Vectorization may refer to:
Computing
* Array programming, a style of computer programming where operations are applied to whole arrays instead of individual elements
* Automatic vectorization, a compiler optimization that transforms loops to vec ...
of
.
Proof
The equivalence between the above ''matrix normal'' and ''multivariate normal'' density functions can be shown using several properties of the
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
and
Kronecker product
In mathematics, the Kronecker product, sometimes denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a block matrix. It is a generalization of the outer product (which is denoted by the same symbol) from vectors to ...
, as follows. We start with the argument of the exponent of the matrix normal PDF:
:
which is the argument of the exponent of the multivariate normal PDF with respect to Lebesgue measure in
. The proof is completed by using the determinant property:
Properties
If
, then we have the following properties:
Expected values
The mean, or
expected value
In probability theory, the expected value (also called expectation, expectancy, mathematical expectation, mean, average, or first moment) is a generalization of the weighted average. Informally, the expected value is the arithmetic mean of a l ...
is:
:
and we have the following second-order expectations:
:
:
where
denotes
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
.
More generally, for appropriately dimensioned matrices A,B,C:
:
Transformation
Transpose
In linear algebra, the transpose of a matrix is an operator which flips a matrix over its diagonal;
that is, it switches the row and column indices of the matrix by producing another matrix, often denoted by (among other notations).
The tr ...
transform:
:
Linear transform: let D (''r''-by-''n''), be of full
rank
Rank is the relative position, value, worth, complexity, power, importance, authority, level, etc. of a person or object within a ranking, such as:
Level or position in a hierarchical organization
* Academic rank
* Diplomatic rank
* Hierarchy
* H ...
''r ≤ n'' and C (''p''-by-''s''), be of full rank ''s ≤ p'', then:
:
Example
Let's imagine a sample of ''n'' independent ''p''-dimensional random variables identically distributed according to a
multivariate normal distribution
In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One d ...
:
:
.
When defining the ''n'' × ''p'' matrix
for which the ''i''th row is
, we obtain:
:
where each row of
is equal to
, that is
,
is the ''n'' × ''n'' identity matrix, that is the rows are independent, and
.
Maximum likelihood parameter estimation
Given ''k'' matrices, each of size ''n'' × ''p'', denoted
, which we assume have been sampled
i.i.d.
In probability theory and statistics, a collection of random variables is independent and identically distributed if each random variable has the same probability distribution as the others and all are mutually independent. This property is us ...
from a matrix normal distribution, the
maximum likelihood estimate
In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statisti ...
of the parameters can be obtained by maximizing:
:
The solution for the mean has a closed form, namely
:
but the covariance parameters do not. However, these parameters can be iteratively maximized by zero-ing their gradients at:
:
and
:
See for example
and references therein. The covariance parameters are non-identifiable in the sense that for any scale factor, ''s''>0, we have:
:
Drawing values from the distribution
Sampling from the matrix normal distribution is a special case of the sampling procedure for the
multivariate normal distribution
In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One d ...
. Let
be an ''n'' by ''p'' matrix of ''np'' independent samples from the standard normal distribution, so that
:
Then let
:
so that
:
where A and B can be chosen by
Cholesky decomposition
In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
or a similar matrix square root operation.
Relation to other distributions
Dawid (1981) provides a discussion of the relation of the matrix-valued normal distribution to other distributions, including the
Wishart distribution
In statistics, the Wishart distribution is a generalization to multiple dimensions of the gamma distribution. It is named in honor of John Wishart, who first formulated the distribution in 1928.
It is a family of probability distributions define ...
,
inverse-Wishart distribution
In statistics, the inverse Wishart distribution, also called the inverted Wishart distribution, is a probability distribution defined on real-valued positive-definite matrices. In Bayesian statistics it is used as the conjugate prior for the co ...
and
matrix t-distribution
In statistics, the matrix ''t''-distribution (or matrix variate ''t''-distribution) is the generalization of the multivariate ''t''-distribution from vectors to matrices. The matrix ''t''-distribution shares the same relationship with the multi ...
, but uses different notation from that employed here.
See also
*
Multivariate normal distribution
In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One d ...
References
*
*
*
{{ProbDistributions, multivariate
Random matrices
Continuous distributions
Multivariate continuous distributions