In
mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
, a
symmetric matrix
In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally,
Because equal matrices have equal dimensions, only square matrices can be symmetric.
The entries of a symmetric matrix are symmetric with re ...
with
real
Real may refer to:
Currencies
* Brazilian real (R$)
* Central American Republic real
* Mexican real
* Portuguese real
* Spanish real
* Spanish colonial real
Music Albums
* ''Real'' (L'Arc-en-Ciel album) (2000)
* ''Real'' (Bright album) (2010)
...
entries is positive-definite if the real number
is positive for every nonzero real
column vector
In linear algebra, a column vector with m elements is an m \times 1 matrix consisting of a single column of m entries, for example,
\boldsymbol = \begin x_1 \\ x_2 \\ \vdots \\ x_m \end.
Similarly, a row vector is a 1 \times n matrix for some n, c ...
where
is the
transpose
In linear algebra, the transpose of a matrix is an operator which flips a matrix over its diagonal;
that is, it switches the row and column indices of the matrix by producing another matrix, often denoted by (among other notations).
The tr ...
of More generally, a
Hermitian matrix
In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
(that is, a
complex matrix
In mathematics, a matrix (plural matrices) is a rectangular array or table of numbers, symbols, or expressions, arranged in rows and columns, which is used to represent a mathematical object or a property of such an object.
For example,
\begin ...
equal to its
conjugate transpose
In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex con ...
) is
positive-definite if the real number
is positive for every nonzero complex column vector
where
denotes the conjugate transpose of
Positive semi-definite matrices are defined similarly, except that the scalars
and
are required to be positive ''or zero'' (that is, nonnegative). Negative-definite and negative semi-definite matrices are defined analogously. A matrix that is not positive semi-definite and not negative semi-definite is sometimes called indefinite.
A matrix is thus positive-definite if and only if it is the matrix of a
positive-definite quadratic form In mathematics, positive definiteness is a property of any object to which a bilinear form or a sesquilinear form may be naturally associated, which is positive-definite. See, in particular:
* Positive-definite bilinear form
* Positive-definite fu ...
or
Hermitian form
In mathematics, a sesquilinear form is a generalization of a bilinear form that, in turn, is a generalization of the concept of the dot product of Euclidean space. A bilinear form is linear in each of its arguments, but a sesquilinear form allows o ...
. In other words, a matrix is positive-definite if and only if it defines an
inner product
In mathematics, an inner product space (or, rarely, a Hausdorff space, Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation (mathematics), operation called an inner product. The inner product of two ve ...
.
Positive-definite and positive-semidefinite matrices can be characterized in many ways, which may explain the importance of the concept in various parts of mathematics. A matrix is positive-definite if and only if it satisfies any of the following equivalent conditions.
* is
congruent
Congruence may refer to:
Mathematics
* Congruence (geometry), being the same size and shape
* Congruence or congruence relation, in abstract algebra, an equivalence relation on an algebraic structure that is compatible with the structure
* In mod ...
with a
diagonal matrix
In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal ma ...
with positive real entries.
* is symmetric or Hermitian, and all its
eigenvalue
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
s are real and positive.
* is symmetric or Hermitian, and all its leading
principal minor
In linear algebra, a minor of a matrix A is the determinant of some smaller square matrix, cut down from A by removing one or more of its rows and columns. Minors obtained by removing just one row and one column from square matrices (first minors ...
s are positive.
* There exists an
invertible matrix
In linear algebra, an -by- square matrix is called invertible (also nonsingular or nondegenerate), if there exists an -by- square matrix such that
:\mathbf = \mathbf = \mathbf_n \
where denotes the -by- identity matrix and the multiplicati ...
with conjugate transpose
such that
A matrix is positive semi-definite if it satisfies similar equivalent conditions where "positive" is replaced by "nonnegative", "invertible matrix" is replaced by "matrix", and the word "leading" is removed.
Positive-definite and positive-semidefinite real matrices are at the basis of
convex optimization
Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets). Many classes of convex optimization probl ...
, since, given a
function of several real variables
In mathematical analysis and its applications, a function of several real variables or real multivariate function is a function with more than one argument, with all arguments being real variables. This concept extends the idea of a function o ...
that is twice
differentiable
In mathematics, a differentiable function of one real variable is a function whose derivative exists at each point in its domain. In other words, the graph of a differentiable function has a non-vertical tangent line at each interior point in its ...
, then if its
Hessian matrix
In mathematics, the Hessian matrix or Hessian is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field. It describes the local curvature of a function of many variables. The Hessian matrix was developed ...
(matrix of its second partial derivatives) is positive-definite at a point , then the function is
convex
Convex or convexity may refer to:
Science and technology
* Convex lens, in optics
Mathematics
* Convex set, containing the whole line segment that joins points
** Convex polygon, a polygon which encloses a convex set of points
** Convex polytope ...
near , and, conversely, if the function is convex near , then the Hessian matrix is positive-semidefinite at .
Some authors use more general definitions of definiteness, including some non-symmetric real matrices, or non-Hermitian complex ones.
Definitions
In the following definitions,
is the transpose of
,
is the
conjugate transpose
In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex con ...
of
and
denotes the ''n''-dimensional zero-vector.
Definitions for real matrices
An
symmetric real matrix
is said to be positive-definite if
for all non-zero
in
. Formally,
An
symmetric real matrix
is said to be positive-semidefinite or non-negative-definite if
for all
in
. Formally,
An
symmetric real matrix
is said to be negative-definite if
for all non-zero
in
. Formally,
An
symmetric real matrix
is said to be negative-semidefinite or non-positive-definite if
for all
in
. Formally,
An
symmetric real matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.
Definitions for complex matrices
The following definitions all involve the term
. Notice that this is always a real number for any Hermitian square matrix
.
An
Hermitian complex matrix
is said to be positive-definite if
for all non-zero
in
. Formally,
An
Hermitian complex matrix
is said to be positive semi-definite or non-negative-definite if
for all
in
. Formally,
An
Hermitian complex matrix
is said to be negative-definite if
for all non-zero
in
. Formally,
An
Hermitian complex matrix
is said to be negative semi-definite or non-positive-definite if
for all
in
. Formally,
An
Hermitian complex matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.
Consistency between real and complex definitions
Since every real matrix is also a complex matrix, the definitions of "definiteness" for the two classes must agree.
For complex matrices, the most common definition says that "
is positive-definite if and only if
is real and positive for all non-zero ''complex'' column vectors
". This condition implies that
is Hermitian (i.e. its transpose is equal to its conjugate). To see this, consider the matrices
and
, so that
and
. The matrices
and
are Hermitian, therefore
and
are individually real. If
is real, then
must be zero for all
. Then
is the zero matrix and
, proving that
is Hermitian.
By this definition, a positive-definite ''real'' matrix
is Hermitian, hence symmetric; and
is positive for all non-zero ''real'' column vectors
. However the last condition alone is not sufficient for
to be positive-definite. For example, if
then for any real vector
with entries
and
we have
, which is always positive if
is not zero. However, if
is the complex vector with entries
and
, one gets
which is not real. Therefore,
is not positive-definite.
On the other hand, for a ''symmetric'' real matrix
, the condition "
for all nonzero real vectors
" ''does'' imply that
is positive-definite in the complex sense.
Notation
If a Hermitian matrix
is positive semi-definite, one sometimes writes
and if
is positive-definite one writes
. To denote that
is negative semi-definite one writes
and to denote that
is negative-definite one writes
.
The notion comes from
functional analysis
Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure (e.g. Inner product space#Definition, inner product, Norm (mathematics)#Defini ...
where positive semidefinite matrices define
positive operator In mathematics (specifically linear algebra, operator theory, and functional analysis) as well as physics, a linear operator A acting on an inner product space is called positive-semidefinite (or ''non-negative'') if, for every x \in \mathop(A), \la ...
s. If two matrices
and
satisfy
, we can define a
non-strict partial order that is
reflexive,
antisymmetric, and
transitive; It is not a
total order
In mathematics, a total or linear order is a partial order in which any two elements are comparable. That is, a total order is a binary relation \leq on some set X, which satisfies the following for all a, b and c in X:
# a \leq a ( reflexive) ...
, however, as
in general may be indefinite.
A common alternative notation is
,
,
and
for positive semi-definite and positive-definite, negative semi-definite and negative-definite matrices, respectively. This may be confusing, as sometimes
nonnegative matrices
In mathematics, the sign of a real number is its property of being either positive, negative, or zero. Depending on local conventions, zero may be considered as being neither positive nor negative (having no sign or a unique third sign), or it ...
(respectively, nonpositive matrices) are also denoted in this way.
Examples
Eigenvalues
Let
be an
Hermitian matrix
In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
(this includes real
symmetric matrices
In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally,
Because equal matrices have equal dimensions, only square matrices can be symmetric.
The entries of a symmetric matrix are symmetric with re ...
). All eigenvalues of
are real, and their sign characterize its definiteness:
*
is positive definite if and only if all of its eigenvalues are positive.
*
is positive semi-definite if and only if all of its eigenvalues are non-negative.
*
is negative definite if and only if all of its eigenvalues are negative
*
is negative semi-definite if and only if all of its eigenvalues are non-positive.
*
is indefinite if and only if it has both positive and negative eigenvalues.
Let
be an
eigendecomposition
In linear algebra, eigendecomposition is the factorization of a matrix into a canonical form, whereby the matrix is represented in terms of its eigenvalues and eigenvectors. Only diagonalizable matrices can be factorized in this way. When the matri ...
of
, where
is a
unitary complex matrix whose columns comprise an
orthonormal basis
In mathematics, particularly linear algebra, an orthonormal basis for an inner product space ''V'' with finite dimension is a basis for V whose vectors are orthonormal, that is, they are all unit vectors and orthogonal to each other. For example, ...
of
eigenvector
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
s of
, and
is a ''real''
diagonal matrix
In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal ma ...
whose
main diagonal
In linear algebra, the main diagonal (sometimes principal diagonal, primary diagonal, leading diagonal, major diagonal, or good diagonal) of a matrix A is the list of entries a_ where i = j. All off-diagonal elements are zero in a diagonal matrix. ...
contains the corresponding
eigenvalue
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
s. The matrix
may be regarded as a diagonal matrix
that has been re-expressed in coordinates of the (eigenvectors) basis
. Put differently, applying
to some vector , giving , is the same as
changing the basis to the eigenvector coordinate system using , giving , applying the
stretching transformation to the result, giving , and then changing the basis back using , giving .
With this in mind, the one-to-one change of variable
shows that
is real and positive for any complex vector
if and only if
is real and positive for any
; in other words, if
is positive definite. For a diagonal matrix, this is true only if each element of the main diagonal—that is, every eigenvalue of
—is positive. Since the
spectral theorem
In mathematics, particularly linear algebra and functional analysis, a spectral theorem is a result about when a linear operator or matrix (mathematics), matrix can be Diagonalizable matrix, diagonalized (that is, represented as a diagonal matrix i ...
guarantees all eigenvalues of a Hermitian matrix to be real, the positivity of eigenvalues can be checked using
Descartes' rule of alternating signs when the
characteristic polynomial
In linear algebra, the characteristic polynomial of a square matrix is a polynomial which is invariant under matrix similarity and has the eigenvalues as roots. It has the determinant and the trace of the matrix among its coefficients. The chara ...
of a real, symmetric matrix
is available.
Decomposition
Let
be an
Hermitian matrix
In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
.
is positive semidefinite if and only if it can be decomposed as a product
of a matrix
with its
conjugate transpose
In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex con ...
.
When
is real,
can be real as well and the decomposition can be written as
is positive definite if and only if such a decomposition exists with
invertible
In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers.
Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
.
More generally,
is positive semidefinite with rank
if and only if a decomposition exists with a
matrix
of full row rank (i.e. of rank
).
Moreover, for any decomposition
,
.
The columns
of
can be seen as vectors in the
complex
Complex commonly refers to:
* Complexity, the behaviour of a system whose components interact in multiple ways so possible interactions are difficult to describe
** Complex system, a system composed of many components which may interact with each ...
or
real vector space
Real may refer to:
Currencies
* Brazilian real (R$)
* Central American Republic real
* Mexican real
* Portuguese real
* Spanish real
* Spanish colonial real
Music Albums
* ''Real'' (L'Arc-en-Ciel album) (2000)
* ''Real'' (Bright album) (2010) ...
, respectively.
Then the entries of
are
inner product
In mathematics, an inner product space (or, rarely, a Hausdorff space, Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation (mathematics), operation called an inner product. The inner product of two ve ...
s (that is
dot product
In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidean space. is an algebra ...
s, in the real case) of these vectors
In other words, a Hermitian matrix
is positive semidefinite if and only if it is the
Gram matrix
In linear algebra, the Gram matrix (or Gramian matrix, Gramian) of a set of vectors v_1,\dots, v_n in an inner product space is the Hermitian matrix of inner products, whose entries are given by the inner product G_ = \left\langle v_i, v_j \right\r ...
of some vectors
.
It is positive definite if and only if it is the Gram matrix of some
linearly independent
In the theory of vector spaces, a set of vectors is said to be if there is a nontrivial linear combination of the vectors that equals the zero vector. If no such linear combination exists, then the vectors are said to be . These concepts are ...
vectors.
In general, the rank of the Gram matrix of vectors
equals the dimension of the space
spanned by these vectors.
Uniqueness up to unitary transformations
The decomposition is not unique:
if
for some
matrix
and if
is any
unitary
Unitary may refer to:
Mathematics
* Unitary divisor
* Unitary element
* Unitary group
* Unitary matrix
* Unitary morphism
* Unitary operator
* Unitary transformation
* Unitary representation
* Unitarity (physics)
* ''E''-unitary inverse semigrou ...
matrix (meaning
),
then
for
.
However, this is the only way in which two decompositions can differ: the decomposition is unique up to
unitary transformation
In mathematics, a unitary transformation is a transformation that preserves the inner product: the inner product of two vectors before the transformation is equal to their inner product after the transformation.
Formal definition
More precisely, ...
s.
More formally, if
is a
matrix and
is a
matrix such that
,
then there is a
matrix
with orthonormal columns (meaning
) such that
.
When
this means
is
unitary
Unitary may refer to:
Mathematics
* Unitary divisor
* Unitary element
* Unitary group
* Unitary matrix
* Unitary morphism
* Unitary operator
* Unitary transformation
* Unitary representation
* Unitarity (physics)
* ''E''-unitary inverse semigrou ...
.
This statement has an intuitive geometric interpretation in the real case:
let the columns of
and
be the vectors
and
in
.
A real unitary matrix is an
orthogonal matrix
In linear algebra, an orthogonal matrix, or orthonormal matrix, is a real square matrix whose columns and rows are orthonormal vectors.
One way to express this is
Q^\mathrm Q = Q Q^\mathrm = I,
where is the transpose of and is the identity ma ...
, which describes a
rigid transformation
In mathematics, a rigid transformation (also called Euclidean transformation or Euclidean isometry) is a geometric transformation of a Euclidean space that preserves the Euclidean distance between every pair of points.
The rigid transformations ...
(an isometry of Euclidean space
) preserving the 0 point (i.e.
rotations and
reflections, without translations).
Therefore, the dot products
and
are equal if and only if some rigid transformation of
transforms the vectors
to
(and 0 to 0).
Square root
A matrix
is positive semidefinite if and only if there is a positive semidefinite matrix
(in particular
is Hermitian, so
) satisfying
. This matrix
is unique, is called the ''non-negative
square root
In mathematics, a square root of a number is a number such that ; in other words, a number whose ''square'' (the result of multiplying the number by itself, or ⋅ ) is . For example, 4 and −4 are square roots of 16, because .
E ...
'' of
, and is denoted with
.
When
is positive definite, so is
, hence it is also called the ''positive square root'' of
.
The non-negative square root should not be confused with other decompositions
.
Some authors use the name ''square root'' and
for any such decomposition, or specifically for the
Cholesky decomposition
In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
,
or any decomposition of the form
;
other only use it for the non-negative square root.
If
then
.
Cholesky decomposition
A positive semidefinite matrix
can be written as
, where
is lower triangular with non-negative diagonal (equivalently
where
is upper triangular); this is the
Cholesky decomposition
In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
.
If
is positive definite, then the diagonal of
is positive and the Cholesky decomposition is unique. Conversely if
is lower triangular with nonnegative diagonal then
is positive semidefinite.
The Cholesky decomposition is especially useful for efficient numerical calculations.
A closely related decomposition is the
LDL decomposition
In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
,
, where
is diagonal and
is
lower unitriangular.
Other characterizations
Let
be an
real symmetric matrix, and let
be the "unit ball" defined by
. Then we have the following
*
is a solid slab sandwiched between
.
*
if and only if
is an ellipsoid, or an ellipsoidal cylinder.
*
if and only if
is bounded, that is, it is an ellipsoid.
* If
, then
if and only if
;
if and only if
.
* If
, then
for all
if and only if
. So, since the polar dual of an ellipsoid is also an ellipsoid with the same principal axes, with inverse lengths, we have
That is, if
is positive-definite, then
for all
if and only if
Let
be an
Hermitian matrix
In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
. The following properties are equivalent to
being positive definite:
; The associated sesquilinear form is an inner product: The
sesquilinear form
In mathematics, a sesquilinear form is a generalization of a bilinear form that, in turn, is a generalization of the concept of the dot product of Euclidean space. A bilinear form is linear in each of its arguments, but a sesquilinear form allows o ...
defined by
is the function
from
to
such that
for all
and
in
, where
is the conjugate transpose of
. For any complex matrix
, this form is linear in
and semilinear in
. Therefore, the form is an
inner product
In mathematics, an inner product space (or, rarely, a Hausdorff space, Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation (mathematics), operation called an inner product. The inner product of two ve ...
on
if and only if
is real and positive for all nonzero
; that is if and only if
is positive definite. (In fact, every inner product on
arises in this fashion from a Hermitian positive definite matrix.)
; Its leading principal minors are all positive: The ''k''th
leading principal minor of a matrix
is the
determinant
In mathematics, the determinant is a scalar value that is a function of the entries of a square matrix. It characterizes some properties of the matrix and the linear map represented by the matrix. In particular, the determinant is nonzero if and ...
of its upper-left
sub-matrix. It turns out that a matrix is positive definite if and only if all these determinants are positive. This condition is known as
Sylvester's criterion In mathematics, Sylvester’s criterion is a necessary and sufficient criterion to determine whether a Hermitian matrix is positive-definite. It is named after James Joseph Sylvester.
Sylvester's criterion states that a ''n'' × ''n'' Hermitian ma ...
, and provides an efficient test of positive definiteness of a symmetric real matrix. Namely, the matrix is reduced to an
upper triangular matrix
In mathematics, a triangular matrix is a special kind of square matrix. A square matrix is called if all the entries ''above'' the main diagonal are zero. Similarly, a square matrix is called if all the entries ''below'' the main diagonal are ...
by using
elementary row operations In mathematics, an elementary matrix is a matrix which differs from the identity matrix by one single elementary row operation. The elementary matrices generate the general linear group GL''n''(F) when F is a field. Left multiplication (pre-multi ...
, as in the first part of the
Gaussian elimination
In mathematics, Gaussian elimination, also known as row reduction, is an algorithm for solving systems of linear equations. It consists of a sequence of operations performed on the corresponding matrix of coefficients. This method can also be used ...
method, taking care to preserve the sign of its determinant during
pivoting process. Since the ''k''th leading principal minor of a triangular matrix is the product of its diagonal elements up to row
, Sylvester's criterion is equivalent to checking whether its diagonal elements are all positive. This condition can be checked each time a new row
of the triangular matrix is obtained.
A positive semidefinite matrix is positive definite if and only if it is
invertible
In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers.
Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
.
A matrix
is negative (semi)definite if and only if
is positive (semi)definite.
Quadratic forms
The (purely)
quadratic form
In mathematics, a quadratic form is a polynomial with terms all of degree two ("form" is another name for a homogeneous polynomial). For example,
:4x^2 + 2xy - 3y^2
is a quadratic form in the variables and . The coefficients usually belong to a ...
associated with a real
matrix
is the function
such that
for all
.
can be assumed symmetric by replacing it with
.
A symmetric matrix
is positive definite if and only if its quadratic form is a
strictly convex function
In mathematics, a real-valued function is called convex if the line segment between any two points on the graph of the function lies above the graph between the two points. Equivalently, a function is convex if its epigraph (the set of poin ...
.
More generally, any
quadratic function
In mathematics, a quadratic polynomial is a polynomial of degree two in one or more variables. A quadratic function is the polynomial function defined by a quadratic polynomial. Before 20th century, the distinction was unclear between a polynomial ...
from
to
can be written as
where
is a symmetric
matrix,
is a real
-vector, and
a real constant. In the
case, this is a parabola, and just like in the
case, we have
Theorem: This quadratic function is strictly convex, and hence has a unique finite global minimum, if and only if
is positive definite.
Proof: If
is positive definite, then the function is strictly convex. Its gradient is zero at the unique point of
, which must be the global minimum since the function is strictly convex. If
is not positive definite, then there exists some vector
such that
, so the function
is a line or a downward parabola, thus not strictly convex and not having a global minimum.
For this reason, positive definite matrices play an important role in
optimization
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
problems.
Simultaneous diagonalization
One symmetric matrix and another matrix that is both symmetric and positive definite can be
simultaneously diagonalized. This is so although simultaneous diagonalization is not necessarily performed with a
similarity transformation. This result does not extend to the case of three or more matrices. In this section we write for the real case. Extension to the complex case is immediate.
Let
be a symmetric and
a symmetric and positive definite matrix. Write the generalized eigenvalue equation as
where we impose that
be normalized, i.e.
. Now we use
Cholesky decomposition
In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
to write the inverse of
as
. Multiplying by
and letting
, we get
, which can be rewritten as
where
. Manipulation now yields
where
is a matrix having as columns the generalized eigenvectors and
is a diagonal matrix of the generalized eigenvalues. Now premultiplication with
gives the final result:
and
, but note that this is no longer an orthogonal diagonalization with respect to the inner product where
. In fact, we diagonalized
with respect to the inner product induced by
.
Note that this result does not contradict what is said on simultaneous diagonalization in the article
Diagonalizable matrix
In linear algebra, a square matrix A is called diagonalizable or non-defective if it is similar to a diagonal matrix, i.e., if there exists an invertible matrix P and a diagonal matrix D such that or equivalently (Such D are not unique.) ...
, which refers to simultaneous diagonalization by a similarity transformation. Our result here is more akin to a simultaneous diagonalization of two quadratic forms, and is useful for optimization of one form under conditions on the other.
Properties
Induced partial ordering
For arbitrary square matrices
,
we write
if
i.e.,
is positive semi-definite. This defines a
partial ordering
In mathematics, especially order theory, a partially ordered set (also poset) formalizes and generalizes the intuitive concept of an ordering, sequencing, or arrangement of the elements of a set. A poset consists of a set together with a binary r ...
on the set of all square matrices. One can similarly define a strict partial ordering
. The ordering is called the
Loewner order In mathematics, Loewner order is the partial order defined by the convex cone of positive semi-definite matrices. This order is usually employed to generalize the definitions of monotone and concave/convex scalar functions to monotone and concav ...
.
Inverse of positive definite matrix
Every positive definite matrix is
invertible
In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers.
Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
and its inverse is also positive definite. If
then
. Moreover, by the
min-max theorem
In linear algebra and functional analysis, the min-max theorem, or variational theorem, or Courant–Fischer–Weyl min-max principle, is a result that gives a variational characterization of eigenvalues of compact Hermitian operators o ...
, the ''k''th largest eigenvalue of
is greater than the ''k''th largest eigenvalue of
.
Scaling
If
is positive definite and
is a real number, then
is positive definite.
[, p. 430, Observation 7.1.3]
Addition
* If
and
are positive-definite, then the sum
is also positive-definite.
* If
and
are positive-semidefinite, then the sum
is also positive-semidefinite.
* If
is positive-definite and
is positive-semidefinite, then the sum
is also positive-definite.
Multiplication
* If
and
are positive definite, then the products
and
are also positive definite. If
, then
is also positive definite.
* If
is positive semidefinite, then
is positive semidefinite for any (possibly rectangular) matrix
. If
is positive definite and
has full column rank, then
is positive definite.
Trace
The diagonal entries
of a positive-semidefinite matrix are real and non-negative. As a consequence the
trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band), a Dutch progressive rock band
* ''The Trace'' (album)
Other uses in arts and entertainment
* ''Trace'' ...
,
. Furthermore, since every principal sub-matrix (in particular, 2-by-2) is positive semidefinite,
and thus, when
,
An
Hermitian matrix
is positive definite if it satisfies the following trace inequalities:
Another important result is that for any
and
positive-semidefinite matrices,
Hadamard product
If
, although
is not necessary positive semidefinite, the
Hadamard product is,
(this result is often called the
Schur product theorem In mathematics, particularly in linear algebra, the Schur product theorem states that the Hadamard product of two positive definite matrices is also a positive definite matrix.
The result is named after Issai Schur (Schur 1911, p. 14, Theor ...
).
Regarding the Hadamard product of two positive semidefinite matrices
,
, there are two notable inequalities:
* Oppenheim's inequality:
*
.
[, Corollary 3.6, p. 227]
Kronecker product
If
, although
is not necessary positive semidefinite, the
Kronecker product
In mathematics, the Kronecker product, sometimes denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a block matrix. It is a generalization of the outer product (which is denoted by the same symbol) from vectors to ...
.
Frobenius product
If
, although
is not necessary positive semidefinite, the
Frobenius inner product
In mathematics, the Frobenius inner product is a binary operation that takes two matrices and returns a scalar. It is often denoted \langle \mathbf,\mathbf \rangle_\mathrm. The operation is a component-wise inner product of two matrices as though t ...
(Lancaster–Tismenetsky, ''The Theory of Matrices'', p. 218).
Convexity
The set of positive semidefinite symmetric matrices is
convex
Convex or convexity may refer to:
Science and technology
* Convex lens, in optics
Mathematics
* Convex set, containing the whole line segment that joins points
** Convex polygon, a polygon which encloses a convex set of points
** Convex polytope ...
. That is, if
and
are positive semidefinite, then for any
between 0 and 1,
is also positive semidefinite. For any vector
:
This property guarantees that
semidefinite programming Semidefinite programming (SDP) is a subfield of convex optimization concerned with the optimization of a linear objective function (a user-specified function that the user wants to minimize or maximize)
over the intersection of the cone of positive ...
problems converge to a globally optimal solution.
Relation with cosine
The positive-definiteness of a matrix
expresses that the angle
between any vector
and its image
is always
:
Further properties
# If
is a symmetric
Toeplitz matrix In linear algebra, a Toeplitz matrix or diagonal-constant matrix, named after Otto Toeplitz, is a matrix in which each descending diagonal from left to right is constant. For instance, the following matrix is a Toeplitz matrix:
:\qquad\begin
a & b ...
, i.e. the entries
are given as a function of their absolute index differences:
, and the ''strict'' inequality
holds, then
is ''strictly'' positive definite.
# Let
and
Hermitian. If
(resp.,
) then
(resp.,
).
[ ]
# If
is real, then there is a
such that
, where
is the
identity matrix
In linear algebra, the identity matrix of size n is the n\times n square matrix with ones on the main diagonal and zeros elsewhere.
Terminology and notation
The identity matrix is often denoted by I_n, or simply by I if the size is immaterial o ...
.
# If
denotes the leading
minor,
is the ''k''th pivot during
LU decomposition
In numerical analysis and linear algebra, lower–upper (LU) decomposition or factorization factors a matrix as the product of a lower triangular matrix and an upper triangular matrix (see matrix decomposition). The product sometimes includes a pe ...
.
# A matrix is negative definite if its ''k-''th order leading
principal minor
In linear algebra, a minor of a matrix A is the determinant of some smaller square matrix, cut down from A by removing one or more of its rows and columns. Minors obtained by removing just one row and one column from square matrices (first minors ...
is negative when
is odd, and positive when
is even.
A Hermitian matrix is positive semidefinite if and only if all of its principal minors are nonnegative. It is however not enough to consider the leading principal minors only, as is checked on the diagonal matrix with entries 0 and −1.
Block matrices and submatrices
A positive
matrix may also be defined by
blocks:
where each block is
. By applying the positivity condition, it immediately follows that
and
are hermitian, and
.
We have that
for all complex
, and in particular for
. Then
A similar argument can be applied to
, and thus we conclude that both
and
must be positive definite. The argument can be extended to show that any
principal submatrix of
is itself positive definite.
Converse results can be proved with stronger conditions on the blocks, for instance using the
Schur complement In linear algebra and the theory of matrices, the Schur complement of a block matrix is defined as follows.
Suppose ''p'', ''q'' are nonnegative integers, and suppose ''A'', ''B'', ''C'', ''D'' are respectively ''p'' × ''p'', ''p'' × ''q'', ''q'' ...
.
Local extrema
A general
quadratic form
In mathematics, a quadratic form is a polynomial with terms all of degree two ("form" is another name for a homogeneous polynomial). For example,
:4x^2 + 2xy - 3y^2
is a quadratic form in the variables and . The coefficients usually belong to a ...
on
real variables
can always be written as
where
is the column vector with those variables, and
is a symmetric real matrix. Therefore, the matrix being positive definite means that
has a unique minimum (zero) when
is zero, and is strictly positive for any other
.
More generally, a twice-differentiable real function
on
real variables has local minimum at arguments
if its
gradient
In vector calculus, the gradient of a scalar-valued differentiable function of several variables is the vector field (or vector-valued function) \nabla f whose value at a point p is the "direction and rate of fastest increase". If the gradi ...
is zero and its
Hessian
A Hessian is an inhabitant of the German state of Hesse.
Hessian may also refer to:
Named from the toponym
*Hessian (soldier), eighteenth-century German regiments in service with the British Empire
**Hessian (boot), a style of boot
**Hessian f ...
(the matrix of all second derivatives) is positive semi-definite at that point. Similar statements can be made for negative definite and semi-definite matrices.
Covariance
In
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the
covariance matrix
In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of ...
of a
multivariate probability distribution
Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considere ...
is always positive semi-definite; and it is positive definite unless one variable is an exact linear function of the others. Conversely, every positive semi-definite matrix is the covariance matrix of some multivariate distribution.
Extension for non-Hermitian square matrices
The definition of positive definite can be generalized by designating any complex matrix
(e.g. real non-symmetric) as positive definite if
for all non-zero complex vectors
, where
denotes the real part of a complex number
.
[Weisstein, Eric W. ]
Positive Definite Matrix.
' From ''MathWorld--A Wolfram Web Resource''. Accessed on 2012-07-26 Only the Hermitian part
determines whether the matrix is positive definite, and is assessed in the narrower sense above. Similarly, if
and
are real, we have
for all real nonzero vectors
if and only if the symmetric part
is positive definite in the narrower sense. It is immediately clear that
is insensitive to transposition of ''M''.
Consequently, a non-symmetric real matrix with only positive eigenvalues does not need to be positive definite. For example, the matrix