In
multivariate statistics
Multivariate statistics is a subdivision of statistics encompassing the simultaneous observation and analysis of more than one outcome variable.
Multivariate statistics concerns understanding the different aims and background of each of the dif ...
, if
is a
vector
Vector most often refers to:
*Euclidean vector, a quantity with a magnitude and a direction
*Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism
Vector may also refer to:
Mathematic ...
of
random variables, and
is an
-dimensional
symmetric matrix
In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally,
Because equal matrices have equal dimensions, only square matrices can be symmetric.
The entries of a symmetric matrix are symmetric with ...
, then the
scalar quantity
is known as a quadratic form in
.
Expectation
It can be shown that
:
where
and
are the
expected value and
variance-covariance matrix of
, respectively, and tr denotes the
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
of a matrix. This result only depends on the existence of
and
; in particular,
normality of
is ''not'' required.
A book treatment of the topic of quadratic forms in random variables is that of Mathai and Provost.
Proof
Since the quadratic form is a scalar quantity,
.
Next, by the cyclic property of the
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
operator,
:
Since the trace operator is a
linear combination of the components of the matrix, it therefore follows from the linearity of the expectation operator that
:
A standard property of variances then tells us that this is
:
Applying the cyclic property of the trace operator again, we get
:
Variance in the Gaussian case
In general, the variance of a quadratic form depends greatly on the distribution of
. However, if
''does'' follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that
is a symmetric matrix. Then,
:
.
In fact, this can be generalized to find the
covariance
In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the ...
between two quadratic forms on the same
(once again,
and
must both be symmetric):
:
.
In addition, a quadratic form such as this follows a
generalized chi-squared distribution.
Computing the variance in the non-symmetric case
Some texts incorrectly state that the above variance or covariance results hold without requiring
to be symmetric. The case for general
can be derived by noting that
:
so
:
is a quadratic form in the symmetric matrix
, so the mean and variance expressions are the same, provided
is replaced by
therein.
Examples of quadratic forms
In the setting where one has a set of observations
and an
operator matrix , then the
residual sum of squares can be written as a quadratic form in
:
:
For procedures where the matrix
is
symmetric
Symmetry (from grc, συμμετρία "agreement in dimensions, due proportion, arrangement") in everyday language refers to a sense of harmonious and beautiful proportion and balance. In mathematics, "symmetry" has a more precise definiti ...
and
idempotent
Idempotence (, ) is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. The concept of idempotence arises in a number of pl ...
, and the
errors are
Gaussian
Carl Friedrich Gauss (1777–1855) is the eponym of all of the topics listed below.
There are over 100 topics all named after this German mathematician and scientist, all in the fields of mathematics, physics, and astronomy. The English eponymo ...
with covariance matrix
,
has a
chi-squared distribution
In probability theory and statistics, the chi-squared distribution (also chi-square or \chi^2-distribution) with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables. The chi-squar ...
with
degrees of freedom and noncentrality parameter
, where
: