HOME

TheInfoList



OR:

In
mathematics Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
, a
symmetric matrix In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally, Because equal matrices have equal dimensions, only square matrices can be symmetric. The entries of a symmetric matrix are symmetric with re ...
M with
real Real may refer to: Currencies * Brazilian real (R$) * Central American Republic real * Mexican real * Portuguese real * Spanish real * Spanish colonial real Music Albums * ''Real'' (L'Arc-en-Ciel album) (2000) * ''Real'' (Bright album) (2010) ...
entries is positive-definite if the real number z^\textsfMz is positive for every nonzero real
column vector In linear algebra, a column vector with m elements is an m \times 1 matrix consisting of a single column of m entries, for example, \boldsymbol = \begin x_1 \\ x_2 \\ \vdots \\ x_m \end. Similarly, a row vector is a 1 \times n matrix for some n, c ...
z, where z^\textsf is the
transpose In linear algebra, the transpose of a matrix is an operator which flips a matrix over its diagonal; that is, it switches the row and column indices of the matrix by producing another matrix, often denoted by (among other notations). The tr ...
of More generally, a
Hermitian matrix In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
(that is, a
complex matrix In mathematics, a matrix (plural matrices) is a rectangular array or table of numbers, symbols, or expressions, arranged in rows and columns, which is used to represent a mathematical object or a property of such an object. For example, \begin ...
equal to its
conjugate transpose In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex con ...
) is positive-definite if the real number z^* Mz is positive for every nonzero complex column vector z, where z^* denotes the conjugate transpose of z. Positive semi-definite matrices are defined similarly, except that the scalars z^\textsfMz and z^* Mz are required to be positive ''or zero'' (that is, nonnegative). Negative-definite and negative semi-definite matrices are defined analogously. A matrix that is not positive semi-definite and not negative semi-definite is sometimes called indefinite. A matrix is thus positive-definite if and only if it is the matrix of a
positive-definite quadratic form In mathematics, positive definiteness is a property of any object to which a bilinear form or a sesquilinear form may be naturally associated, which is positive-definite. See, in particular: * Positive-definite bilinear form * Positive-definite fu ...
or
Hermitian form In mathematics, a sesquilinear form is a generalization of a bilinear form that, in turn, is a generalization of the concept of the dot product of Euclidean space. A bilinear form is linear in each of its arguments, but a sesquilinear form allow ...
. In other words, a matrix is positive-definite if and only if it defines an
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
. Positive-definite and positive-semidefinite matrices can be characterized in many ways, which may explain the importance of the concept in various parts of mathematics. A matrix is positive-definite if and only if it satisfies any of the following equivalent conditions. * is
congruent Congruence may refer to: Mathematics * Congruence (geometry), being the same size and shape * Congruence or congruence relation, in abstract algebra, an equivalence relation on an algebraic structure that is compatible with the structure * In mod ...
with a
diagonal matrix In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal ma ...
with positive real entries. * is symmetric or Hermitian, and all its
eigenvalue In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
s are real and positive. * is symmetric or Hermitian, and all its leading
principal minor In linear algebra, a minor of a matrix A is the determinant of some smaller square matrix, cut down from A by removing one or more of its rows and columns. Minors obtained by removing just one row and one column from square matrices (first minors ...
s are positive. * There exists an
invertible matrix In linear algebra, an -by- square matrix is called invertible (also nonsingular or nondegenerate), if there exists an -by- square matrix such that :\mathbf = \mathbf = \mathbf_n \ where denotes the -by- identity matrix and the multiplicati ...
B with conjugate transpose B^* such that M=B^*B. A matrix is positive semi-definite if it satisfies similar equivalent conditions where "positive" is replaced by "nonnegative", "invertible matrix" is replaced by "matrix", and the word "leading" is removed. Positive-definite and positive-semidefinite real matrices are at the basis of
convex optimization Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets). Many classes of convex optimization probl ...
, since, given a
function of several real variables In mathematical analysis and its applications, a function of several real variables or real multivariate function is a function with more than one argument, with all arguments being real variables. This concept extends the idea of a function o ...
that is twice
differentiable In mathematics, a differentiable function of one real variable is a function whose derivative exists at each point in its domain. In other words, the graph of a differentiable function has a non-vertical tangent line at each interior point in its ...
, then if its
Hessian matrix In mathematics, the Hessian matrix or Hessian is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field. It describes the local curvature of a function of many variables. The Hessian matrix was developed ...
(matrix of its second partial derivatives) is positive-definite at a point , then the function is
convex Convex or convexity may refer to: Science and technology * Convex lens, in optics Mathematics * Convex set, containing the whole line segment that joins points ** Convex polygon, a polygon which encloses a convex set of points ** Convex polytope ...
near , and, conversely, if the function is convex near , then the Hessian matrix is positive-semidefinite at . Some authors use more general definitions of definiteness, including some non-symmetric real matrices, or non-Hermitian complex ones.


Definitions

In the following definitions, \mathbf^\textsf is the transpose of \mathbf x, \mathbf^* is the
conjugate transpose In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex con ...
of \mathbf x and \mathbf denotes the ''n''-dimensional zero-vector.


Definitions for real matrices

An n \times n symmetric real matrix M is said to be positive-definite if \mathbf^\textsf M\mathbf > 0 for all non-zero \mathbf in \R^n. Formally, An n \times n symmetric real matrix M is said to be positive-semidefinite or non-negative-definite if \mathbf^\textsf M\mathbf \geq 0 for all \mathbf in \R^n. Formally, An n \times n symmetric real matrix M is said to be negative-definite if \mathbf^\textsf M\mathbf < 0 for all non-zero \mathbf in \R^n. Formally, An n \times n symmetric real matrix M is said to be negative-semidefinite or non-positive-definite if x^\textsf Mx \leq 0 for all x in \R^n. Formally, An n \times n symmetric real matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.


Definitions for complex matrices

The following definitions all involve the term \mathbf^* M\mathbf. Notice that this is always a real number for any Hermitian square matrix M. An n \times n Hermitian complex matrix M is said to be positive-definite if \mathbf^* M\mathbf > 0 for all non-zero \mathbf in \Complex^n. Formally, An n \times n Hermitian complex matrix M is said to be positive semi-definite or non-negative-definite if x^* Mx \geq 0 for all x in \Complex^n. Formally, An n \times n Hermitian complex matrix M is said to be negative-definite if \mathbf^* M\mathbf < 0 for all non-zero \mathbf in \Complex^n. Formally, An n \times n Hermitian complex matrix M is said to be negative semi-definite or non-positive-definite if \mathbf^* M\mathbf \leq 0 for all \mathbf in \Complex^n. Formally, An n \times n Hermitian complex matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.


Consistency between real and complex definitions

Since every real matrix is also a complex matrix, the definitions of "definiteness" for the two classes must agree. For complex matrices, the most common definition says that "M is positive-definite if and only if \mathbf^* M\mathbf is real and positive for all non-zero ''complex'' column vectors \mathbf z". This condition implies that M is Hermitian (i.e. its transpose is equal to its conjugate). To see this, consider the matrices A = \frac \left(M + M^*\right) and B = \frac \left(M - M^*\right), so that M = A + iB and \mathbf^* M\mathbf = \mathbf^* A\mathbf + i\mathbf^* B\mathbf. The matrices A and B are Hermitian, therefore \mathbf^* A\mathbf and \mathbf^* B\mathbf are individually real. If \mathbf^* M\mathbf is real, then \mathbf^* B\mathbf must be zero for all \mathbf z. Then B is the zero matrix and M = A, proving that M is Hermitian. By this definition, a positive-definite ''real'' matrix M is Hermitian, hence symmetric; and \mathbf^\textsf M\mathbf is positive for all non-zero ''real'' column vectors \mathbf. However the last condition alone is not sufficient for M to be positive-definite. For example, if M = \begin 1 & 1 \\ -1 & 1 \end, then for any real vector \mathbf with entries a and b we have \mathbf^\textsf M\mathbf = \left(a + b\right)a + \left(-a + b\right)b = a^2 + b^2, which is always positive if \mathbf z is not zero. However, if \mathbf z is the complex vector with entries 1 and i, one gets \mathbf^* M \mathbf = \begin1 & -i\end M \begin1 \\ i\end = \begin1 + i & 1 - i\end \begin1 \\ i\end = 2+2i which is not real. Therefore, M is not positive-definite. On the other hand, for a ''symmetric'' real matrix M, the condition "\mathbf^\textsf M\mathbf > 0 for all nonzero real vectors \mathbf z" ''does'' imply that M is positive-definite in the complex sense.


Notation

If a Hermitian matrix M is positive semi-definite, one sometimes writes M \succeq 0 and if M is positive-definite one writes M \succ 0. To denote that M is negative semi-definite one writes M \preceq 0 and to denote that M is negative-definite one writes M \prec 0. The notion comes from
functional analysis Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure (e.g. Inner product space#Definition, inner product, Norm (mathematics)#Defini ...
where positive semidefinite matrices define
positive operator In mathematics (specifically linear algebra, operator theory, and functional analysis) as well as physics, a linear operator A acting on an inner product space is called positive-semidefinite (or ''non-negative'') if, for every x \in \mathop(A), \l ...
s. If two matrices A and B satisfy B - A \succeq 0, we can define a non-strict partial order B \succeq A that is reflexive, antisymmetric, and transitive; It is not a
total order In mathematics, a total or linear order is a partial order in which any two elements are comparable. That is, a total order is a binary relation \leq on some set X, which satisfies the following for all a, b and c in X: # a \leq a ( reflexive) ...
, however, as B - A in general may be indefinite. A common alternative notation is M \geq 0, M > 0, M \leq 0 and M < 0 for positive semi-definite and positive-definite, negative semi-definite and negative-definite matrices, respectively. This may be confusing, as sometimes
nonnegative matrices In mathematics, the sign of a real number is its property of being either positive, negative, or zero. Depending on local conventions, zero may be considered as being neither positive nor negative (having no sign or a unique third sign), or it ...
(respectively, nonpositive matrices) are also denoted in this way.


Examples


Eigenvalues

Let M be an n \times n
Hermitian matrix In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
(this includes real
symmetric matrices In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally, Because equal matrices have equal dimensions, only square matrices can be symmetric. The entries of a symmetric matrix are symmetric with re ...
). All eigenvalues of M are real, and their sign characterize its definiteness: * M is positive definite if and only if all of its eigenvalues are positive. * M is positive semi-definite if and only if all of its eigenvalues are non-negative. * M is negative definite if and only if all of its eigenvalues are negative * M is negative semi-definite if and only if all of its eigenvalues are non-positive. * M is indefinite if and only if it has both positive and negative eigenvalues. Let PD P^ be an
eigendecomposition In linear algebra, eigendecomposition is the factorization of a matrix into a canonical form, whereby the matrix is represented in terms of its eigenvalues and eigenvectors. Only diagonalizable matrices can be factorized in this way. When the matr ...
of M, where P is a unitary complex matrix whose columns comprise an
orthonormal basis In mathematics, particularly linear algebra, an orthonormal basis for an inner product space ''V'' with finite dimension is a basis for V whose vectors are orthonormal, that is, they are all unit vectors and orthogonal to each other. For example, ...
of
eigenvector In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
s of M, and D is a ''real''
diagonal matrix In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal ma ...
whose
main diagonal In linear algebra, the main diagonal (sometimes principal diagonal, primary diagonal, leading diagonal, major diagonal, or good diagonal) of a matrix A is the list of entries a_ where i = j. All off-diagonal elements are zero in a diagonal matrix. ...
contains the corresponding
eigenvalue In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
s. The matrix M may be regarded as a diagonal matrix D that has been re-expressed in coordinates of the (eigenvectors) basis P. Put differently, applying M to some vector , giving , is the same as changing the basis to the eigenvector coordinate system using , giving , applying the stretching transformation to the result, giving , and then changing the basis back using , giving . With this in mind, the one-to-one change of variable \mathbf = P\mathbf shows that \mathbf^* M\mathbf is real and positive for any complex vector \mathbf z if and only if \mathbf^* D\mathbf is real and positive for any y; in other words, if D is positive definite. For a diagonal matrix, this is true only if each element of the main diagonal—that is, every eigenvalue of M—is positive. Since the
spectral theorem In mathematics, particularly linear algebra and functional analysis, a spectral theorem is a result about when a linear operator or matrix (mathematics), matrix can be Diagonalizable matrix, diagonalized (that is, represented as a diagonal matrix i ...
guarantees all eigenvalues of a Hermitian matrix to be real, the positivity of eigenvalues can be checked using Descartes' rule of alternating signs when the
characteristic polynomial In linear algebra, the characteristic polynomial of a square matrix is a polynomial which is invariant under matrix similarity and has the eigenvalues as roots. It has the determinant and the trace of the matrix among its coefficients. The chara ...
of a real, symmetric matrix M is available.


Decomposition

Let M be an n \times n
Hermitian matrix In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
. M is positive semidefinite if and only if it can be decomposed as a product M = B^* B of a matrix B with its
conjugate transpose In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex con ...
. When M is real, B can be real as well and the decomposition can be written as M = B^\textsf B. M is positive definite if and only if such a decomposition exists with B
invertible In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers. Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
. More generally, M is positive semidefinite with rank k if and only if a decomposition exists with a k \times n matrix B of full row rank (i.e. of rank k). Moreover, for any decomposition M = B^* B, \operatorname(M) = \operatorname(B). The columns b_1,\dots,b_n of B can be seen as vectors in the
complex Complex commonly refers to: * Complexity, the behaviour of a system whose components interact in multiple ways so possible interactions are difficult to describe ** Complex system, a system composed of many components which may interact with each ...
or
real vector space Real may refer to: Currencies * Brazilian real (R$) * Central American Republic real * Mexican real * Portuguese real * Spanish real * Spanish colonial real Music Albums * ''Real'' (L'Arc-en-Ciel album) (2000) * ''Real'' (Bright album) (2010) ...
\mathbb^k, respectively. Then the entries of M are
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
s (that is
dot product In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidean space. is an algebra ...
s, in the real case) of these vectors M_ = \langle b_i, b_j\rangle. In other words, a Hermitian matrix M is positive semidefinite if and only if it is the
Gram matrix In linear algebra, the Gram matrix (or Gramian matrix, Gramian) of a set of vectors v_1,\dots, v_n in an inner product space is the Hermitian matrix of inner products, whose entries are given by the inner product G_ = \left\langle v_i, v_j \right\r ...
of some vectors b_1,\dots,b_n. It is positive definite if and only if it is the Gram matrix of some
linearly independent In the theory of vector spaces, a set of vectors is said to be if there is a nontrivial linear combination of the vectors that equals the zero vector. If no such linear combination exists, then the vectors are said to be . These concepts are ...
vectors. In general, the rank of the Gram matrix of vectors b_1,\dots,b_n equals the dimension of the space spanned by these vectors.


Uniqueness up to unitary transformations

The decomposition is not unique: if M = B^* B for some k \times n matrix B and if Q is any
unitary Unitary may refer to: Mathematics * Unitary divisor * Unitary element * Unitary group * Unitary matrix * Unitary morphism * Unitary operator * Unitary transformation * Unitary representation * Unitarity (physics) * ''E''-unitary inverse semigrou ...
k \times k matrix (meaning Q^* Q = Q Q^* = I), then M = B^* B = B^* Q^* Q B =A^* A for A=Q B. However, this is the only way in which two decompositions can differ: the decomposition is unique up to
unitary transformation In mathematics, a unitary transformation is a transformation that preserves the inner product: the inner product of two vectors before the transformation is equal to their inner product after the transformation. Formal definition More precisely, ...
s. More formally, if A is a k \times n matrix and B is a \ell \times n matrix such that A^* A = B^* B, then there is a \ell \times k matrix Q with orthonormal columns (meaning Q^* Q = I_) such that B = Q A. When \ell = k this means Q is
unitary Unitary may refer to: Mathematics * Unitary divisor * Unitary element * Unitary group * Unitary matrix * Unitary morphism * Unitary operator * Unitary transformation * Unitary representation * Unitarity (physics) * ''E''-unitary inverse semigrou ...
. This statement has an intuitive geometric interpretation in the real case: let the columns of A and B be the vectors a_1,\dots,a_n and b_1,\dots,b_n in \mathbb^k. A real unitary matrix is an
orthogonal matrix In linear algebra, an orthogonal matrix, or orthonormal matrix, is a real square matrix whose columns and rows are orthonormal vectors. One way to express this is Q^\mathrm Q = Q Q^\mathrm = I, where is the transpose of and is the identity ma ...
, which describes a
rigid transformation In mathematics, a rigid transformation (also called Euclidean transformation or Euclidean isometry) is a geometric transformation of a Euclidean space that preserves the Euclidean distance between every pair of points. The rigid transformations ...
(an isometry of Euclidean space \mathbb^k) preserving the 0 point (i.e. rotations and reflections, without translations). Therefore, the dot products a_i \cdot a_j and b_i \cdot b_j are equal if and only if some rigid transformation of \mathbb^k transforms the vectors a_1,\dots,a_n to b_1,\dots,b_n (and 0 to 0).


Square root

A matrix M is positive semidefinite if and only if there is a positive semidefinite matrix B (in particular B is Hermitian, so B^* = B) satisfying M = B B. This matrix B is unique, is called the ''non-negative
square root In mathematics, a square root of a number is a number such that ; in other words, a number whose ''square'' (the result of multiplying the number by itself, or  ⋅ ) is . For example, 4 and −4 are square roots of 16, because . E ...
'' of M, and is denoted with B = M^\frac. When M is positive definite, so is M^\frac, hence it is also called the ''positive square root'' of M. The non-negative square root should not be confused with other decompositions M = B^*B. Some authors use the name ''square root'' and M^\frac for any such decomposition, or specifically for the
Cholesky decomposition In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
, or any decomposition of the form M = B B; other only use it for the non-negative square root. If M > N > 0 then M^\frac > N^\frac > 0.


Cholesky decomposition

A positive semidefinite matrix M can be written as M = LL^*, where L is lower triangular with non-negative diagonal (equivalently M = B^*B where B=L^* is upper triangular); this is the
Cholesky decomposition In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
. If M is positive definite, then the diagonal of L is positive and the Cholesky decomposition is unique. Conversely if L is lower triangular with nonnegative diagonal then L is positive semidefinite. The Cholesky decomposition is especially useful for efficient numerical calculations. A closely related decomposition is the
LDL decomposition In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
, M = L D L^*, where D is diagonal and L is lower unitriangular.


Other characterizations

Let M be an n \times n real symmetric matrix, and let B_1(M) := \ be the "unit ball" defined by M. Then we have the following * B_1(vv^\mathsf) is a solid slab sandwiched between \pm \. * M\succeq 0 if and only if B_1(M) is an ellipsoid, or an ellipsoidal cylinder. * M\succ 0 if and only if B_1(M) is bounded, that is, it is an ellipsoid. * If N\succ 0, then M \succeq N if and only if B_1(M) \subseteq B_1(N); M \succ N if and only if B_1(M) \subseteq \operatorname(B_1(N)). * If N\succ 0 , then M \succeq \frac for all v \neq 0 if and only if B_1(M) \subset \bigcap_ B_1(vv^\mathsf). So, since the polar dual of an ellipsoid is also an ellipsoid with the same principal axes, with inverse lengths, we have B_1(N^) = \bigcap_ B_1(vv^\mathsf) = \bigcap_\ That is, if N is positive-definite, then M \succeq \frac for all v\neq 0 if and only if M \succeq N^ Let M be an n \times n
Hermitian matrix In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
. The following properties are equivalent to M being positive definite: ; The associated sesquilinear form is an inner product: The
sesquilinear form In mathematics, a sesquilinear form is a generalization of a bilinear form that, in turn, is a generalization of the concept of the dot product of Euclidean space. A bilinear form is linear in each of its arguments, but a sesquilinear form allows o ...
defined by M is the function \langle \cdot, \cdot\rangle from \Complex^n \times \Complex^n to \Complex^n such that \langle x, y \rangle := y^*M x for all x and y in \Complex^n, where y^* is the conjugate transpose of y. For any complex matrix M, this form is linear in x and semilinear in y. Therefore, the form is an
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
on \Complex^n if and only if \langle z, z \rangle is real and positive for all nonzero z; that is if and only if M is positive definite. (In fact, every inner product on \Complex^n arises in this fashion from a Hermitian positive definite matrix.) ; Its leading principal minors are all positive: The ''k''th leading principal minor of a matrix M is the
determinant In mathematics, the determinant is a scalar value that is a function of the entries of a square matrix. It characterizes some properties of the matrix and the linear map represented by the matrix. In particular, the determinant is nonzero if and ...
of its upper-left k \times k sub-matrix. It turns out that a matrix is positive definite if and only if all these determinants are positive. This condition is known as
Sylvester's criterion In mathematics, Sylvester’s criterion is a necessary and sufficient criterion to determine whether a Hermitian matrix is positive-definite. It is named after James Joseph Sylvester. Sylvester's criterion states that a ''n'' × ''n'' Hermitian ma ...
, and provides an efficient test of positive definiteness of a symmetric real matrix. Namely, the matrix is reduced to an
upper triangular matrix In mathematics, a triangular matrix is a special kind of square matrix. A square matrix is called if all the entries ''above'' the main diagonal are zero. Similarly, a square matrix is called if all the entries ''below'' the main diagonal are ...
by using
elementary row operations In mathematics, an elementary matrix is a matrix which differs from the identity matrix by one single elementary row operation. The elementary matrices generate the general linear group GL''n''(F) when F is a field. Left multiplication (pre-multi ...
, as in the first part of the
Gaussian elimination In mathematics, Gaussian elimination, also known as row reduction, is an algorithm for solving systems of linear equations. It consists of a sequence of operations performed on the corresponding matrix of coefficients. This method can also be used ...
method, taking care to preserve the sign of its determinant during pivoting process. Since the ''k''th leading principal minor of a triangular matrix is the product of its diagonal elements up to row k, Sylvester's criterion is equivalent to checking whether its diagonal elements are all positive. This condition can be checked each time a new row k of the triangular matrix is obtained. A positive semidefinite matrix is positive definite if and only if it is
invertible In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers. Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
. A matrix M is negative (semi)definite if and only if -M is positive (semi)definite.


Quadratic forms

The (purely)
quadratic form In mathematics, a quadratic form is a polynomial with terms all of degree two ("form" is another name for a homogeneous polynomial). For example, :4x^2 + 2xy - 3y^2 is a quadratic form in the variables and . The coefficients usually belong to a ...
associated with a real n \times n matrix M is the function Q : \mathbb^n \to \mathbb such that Q(x) = x^\textsf Mx for all x. M can be assumed symmetric by replacing it with \tfrac \left(M + M^\textsf\right). A symmetric matrix M is positive definite if and only if its quadratic form is a
strictly convex function In mathematics, a real-valued function is called convex if the line segment between any two points on the graph of the function lies above the graph between the two points. Equivalently, a function is convex if its epigraph (the set of poin ...
. More generally, any
quadratic function In mathematics, a quadratic polynomial is a polynomial of degree two in one or more variables. A quadratic function is the polynomial function defined by a quadratic polynomial. Before 20th century, the distinction was unclear between a polynomial ...
from \mathbb^n to \mathbb can be written as x^\textsf Mx + x^\textsf b + c where M is a symmetric n \times n matrix, b is a real n-vector, and c a real constant. In the n=1 case, this is a parabola, and just like in the n=1 case, we have Theorem: This quadratic function is strictly convex, and hence has a unique finite global minimum, if and only if M is positive definite. Proof: If M is positive definite, then the function is strictly convex. Its gradient is zero at the unique point of M^b, which must be the global minimum since the function is strictly convex. If M is not positive definite, then there exists some vector v such that v^T M v \leq 0, so the function f(t) := (vt)^T M (vt) + b^t (vt) + c is a line or a downward parabola, thus not strictly convex and not having a global minimum. For this reason, positive definite matrices play an important role in
optimization Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
problems.


Simultaneous diagonalization

One symmetric matrix and another matrix that is both symmetric and positive definite can be simultaneously diagonalized. This is so although simultaneous diagonalization is not necessarily performed with a similarity transformation. This result does not extend to the case of three or more matrices. In this section we write for the real case. Extension to the complex case is immediate. Let M be a symmetric and N a symmetric and positive definite matrix. Write the generalized eigenvalue equation as \left(M - \lambda N\right)\mathbf = 0 where we impose that x be normalized, i.e. \mathbf^\textsf N\mathbf = 1. Now we use
Cholesky decomposition In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
to write the inverse of N as Q^\textsf Q. Multiplying by Q and letting \mathbf = Q^\textsf \mathbf, we get Q\left(M - \lambda N\right)Q^\textsf \mathbf = 0, which can be rewritten as \left(QMQ^\textsf\right)\mathbf = \lambda \mathbf where \mathbf^\textsf \mathbf = 1. Manipulation now yields MX = NX\Lambda where X is a matrix having as columns the generalized eigenvectors and \Lambda is a diagonal matrix of the generalized eigenvalues. Now premultiplication with X^\textsf gives the final result: X^\textsf MX = \Lambda and X^\textsf NX = I, but note that this is no longer an orthogonal diagonalization with respect to the inner product where \mathbf^\textsf \mathbf = 1. In fact, we diagonalized M with respect to the inner product induced by N. Note that this result does not contradict what is said on simultaneous diagonalization in the article
Diagonalizable matrix In linear algebra, a square matrix A is called diagonalizable or non-defective if it is similar to a diagonal matrix, i.e., if there exists an invertible matrix P and a diagonal matrix D such that or equivalently (Such D are not unique.) ...
, which refers to simultaneous diagonalization by a similarity transformation. Our result here is more akin to a simultaneous diagonalization of two quadratic forms, and is useful for optimization of one form under conditions on the other.


Properties


Induced partial ordering

For arbitrary square matrices M, N we write M \ge N if M - N \ge 0 i.e., M - N is positive semi-definite. This defines a
partial ordering In mathematics, especially order theory, a partially ordered set (also poset) formalizes and generalizes the intuitive concept of an ordering, sequencing, or arrangement of the elements of a set. A poset consists of a set together with a binary r ...
on the set of all square matrices. One can similarly define a strict partial ordering M > N. The ordering is called the
Loewner order In mathematics, Loewner order is the partial order defined by the convex cone of positive semi-definite matrices. This order is usually employed to generalize the definitions of monotone and concave/convex scalar functions to monotone and concav ...
.


Inverse of positive definite matrix

Every positive definite matrix is
invertible In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers. Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
and its inverse is also positive definite. If M \geq N > 0 then N^ \geq M^ > 0. Moreover, by the
min-max theorem In linear algebra and functional analysis, the min-max theorem, or variational theorem, or Courant–Fischer–Weyl min-max principle, is a result that gives a variational characterization of eigenvalues of compact Hermitian operators o ...
, the ''k''th largest eigenvalue of M is greater than the ''k''th largest eigenvalue of N.


Scaling

If M is positive definite and r > 0 is a real number, then r M is positive definite., p. 430, Observation 7.1.3


Addition

* If M and N are positive-definite, then the sum M + N is also positive-definite. * If M and N are positive-semidefinite, then the sum M + N is also positive-semidefinite. * If M is positive-definite and N is positive-semidefinite, then the sum M + N is also positive-definite.


Multiplication

* If M and N are positive definite, then the products MNM and NMN are also positive definite. If MN = NM, then MN is also positive definite. * If M is positive semidefinite, then A^* MA is positive semidefinite for any (possibly rectangular) matrix A. If M is positive definite and A has full column rank, then A^* M A is positive definite.


Trace

The diagonal entries m_ of a positive-semidefinite matrix are real and non-negative. As a consequence the
trace Trace may refer to: Arts and entertainment Music * ''Trace'' (Son Volt album), 1995 * ''Trace'' (Died Pretty album), 1993 * Trace (band), a Dutch progressive rock band * ''The Trace'' (album) Other uses in arts and entertainment * ''Trace'' ...
, \operatorname(M)\ge0. Furthermore, since every principal sub-matrix (in particular, 2-by-2) is positive semidefinite, \left, m_\ \leq \sqrt \quad \forall i, j and thus, when n \ge 1, \max_ \left, m_\ \leq \max_i m_ An n \times n Hermitian matrix M is positive definite if it satisfies the following trace inequalities: \operatorname(M) > 0 \quad \mathrm \quad \frac > n-1. Another important result is that for any M and N positive-semidefinite matrices, \operatorname(MN) \ge 0


Hadamard product

If M, N \geq 0, although MN is not necessary positive semidefinite, the Hadamard product is, M \circ N \geq 0 (this result is often called the
Schur product theorem In mathematics, particularly in linear algebra, the Schur product theorem states that the Hadamard product of two positive definite matrices is also a positive definite matrix. The result is named after Issai Schur (Schur 1911, p. 14, Theor ...
). Regarding the Hadamard product of two positive semidefinite matrices M = (m_) \geq 0, N \geq 0, there are two notable inequalities: * Oppenheim's inequality: \det(M \circ N) \geq \det (N) \prod\nolimits_i m_. * \det(M \circ N) \geq \det(M) \det(N)., Corollary 3.6, p. 227


Kronecker product

If M, N \geq 0, although MN is not necessary positive semidefinite, the
Kronecker product In mathematics, the Kronecker product, sometimes denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a block matrix. It is a generalization of the outer product (which is denoted by the same symbol) from vectors ...
M \otimes N \geq 0.


Frobenius product

If M, N \geq 0, although MN is not necessary positive semidefinite, the
Frobenius inner product In mathematics, the Frobenius inner product is a binary operation that takes two matrices and returns a scalar. It is often denoted \langle \mathbf,\mathbf \rangle_\mathrm. The operation is a component-wise inner product of two matrices as though ...
M : N \geq 0 (Lancaster–Tismenetsky, ''The Theory of Matrices'', p. 218).


Convexity

The set of positive semidefinite symmetric matrices is
convex Convex or convexity may refer to: Science and technology * Convex lens, in optics Mathematics * Convex set, containing the whole line segment that joins points ** Convex polygon, a polygon which encloses a convex set of points ** Convex polytope ...
. That is, if M and N are positive semidefinite, then for any \alpha between 0 and 1, \alpha M + \left(1 - \alpha\right) N is also positive semidefinite. For any vector \mathbf x: \mathbf^\textsf \left(\alpha M + \left(1 - \alpha\right)N\right)\mathbf = \alpha \mathbf^\textsf M\mathbf + (1 - \alpha) \mathbf^\textsf N\mathbf \geq 0. This property guarantees that
semidefinite programming Semidefinite programming (SDP) is a subfield of convex optimization concerned with the optimization of a linear objective function (a user-specified function that the user wants to minimize or maximize) over the intersection of the cone of positive ...
problems converge to a globally optimal solution.


Relation with cosine

The positive-definiteness of a matrix A expresses that the angle \theta between any vector \mathbf x and its image A\mathbf is always -\pi / 2 < \theta < +\pi / 2 : \cos\theta = \frac=\frac , \theta=\theta(\mathbf,A\mathbf)=\widehat= \text \mathbf \text A\mathbf


Further properties

# If M is a symmetric
Toeplitz matrix In linear algebra, a Toeplitz matrix or diagonal-constant matrix, named after Otto Toeplitz, is a matrix in which each descending diagonal from left to right is constant. For instance, the following matrix is a Toeplitz matrix: :\qquad\begin a & b ...
, i.e. the entries m_ are given as a function of their absolute index differences: m_ = h(, i-j, ), and the ''strict'' inequality \sum_ \left, h(j)\ < h(0) holds, then M is ''strictly'' positive definite. # Let M > 0 and N Hermitian. If MN + NM \ge 0 (resp., MN + NM > 0) then N \ge 0 (resp., N > 0). # If M > 0 is real, then there is a \delta > 0 such that M>\delta I, where I is the
identity matrix In linear algebra, the identity matrix of size n is the n\times n square matrix with ones on the main diagonal and zeros elsewhere. Terminology and notation The identity matrix is often denoted by I_n, or simply by I if the size is immaterial o ...
. # If M_k denotes the leading k \times k minor, \det\left(M_k\right)/\det\left(M_\right) is the ''k''th pivot during
LU decomposition In numerical analysis and linear algebra, lower–upper (LU) decomposition or factorization factors a matrix as the product of a lower triangular matrix and an upper triangular matrix (see matrix decomposition). The product sometimes includes a pe ...
. # A matrix is negative definite if its ''k-''th order leading
principal minor In linear algebra, a minor of a matrix A is the determinant of some smaller square matrix, cut down from A by removing one or more of its rows and columns. Minors obtained by removing just one row and one column from square matrices (first minors ...
is negative when k is odd, and positive when k is even. A Hermitian matrix is positive semidefinite if and only if all of its principal minors are nonnegative. It is however not enough to consider the leading principal minors only, as is checked on the diagonal matrix with entries 0 and −1.


Block matrices and submatrices

A positive 2n \times 2n matrix may also be defined by blocks: M = \begin A & B \\ C & D \end where each block is n \times n. By applying the positivity condition, it immediately follows that A and D are hermitian, and C = B^*. We have that \mathbf^* M\mathbf \ge 0 for all complex \mathbf z, and in particular for \mathbf = mathbf, 0\textsf. Then \begin \mathbf^* & 0 \end \begin A & B \\ B^* & D \end \begin \mathbf \\ 0 \end = \mathbf^* A\mathbf \ge 0. A similar argument can be applied to D, and thus we conclude that both A and D must be positive definite. The argument can be extended to show that any principal submatrix of M is itself positive definite. Converse results can be proved with stronger conditions on the blocks, for instance using the
Schur complement In linear algebra and the theory of matrices, the Schur complement of a block matrix is defined as follows. Suppose ''p'', ''q'' are nonnegative integers, and suppose ''A'', ''B'', ''C'', ''D'' are respectively ''p'' × ''p'', ''p'' × ''q'', ''q'' ...
.


Local extrema

A general
quadratic form In mathematics, a quadratic form is a polynomial with terms all of degree two ("form" is another name for a homogeneous polynomial). For example, :4x^2 + 2xy - 3y^2 is a quadratic form in the variables and . The coefficients usually belong to a ...
f(\mathbf) on n real variables x_1, \ldots, x_n can always be written as \mathbf^\textsf M \mathbf where \mathbf is the column vector with those variables, and M is a symmetric real matrix. Therefore, the matrix being positive definite means that f has a unique minimum (zero) when \mathbf is zero, and is strictly positive for any other \mathbf. More generally, a twice-differentiable real function f on n real variables has local minimum at arguments x_1, \ldots, x_n if its
gradient In vector calculus, the gradient of a scalar-valued differentiable function of several variables is the vector field (or vector-valued function) \nabla f whose value at a point p is the "direction and rate of fastest increase". If the gradi ...
is zero and its
Hessian A Hessian is an inhabitant of the German state of Hesse. Hessian may also refer to: Named from the toponym *Hessian (soldier), eighteenth-century German regiments in service with the British Empire **Hessian (boot), a style of boot **Hessian f ...
(the matrix of all second derivatives) is positive semi-definite at that point. Similar statements can be made for negative definite and semi-definite matrices.


Covariance

In
statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the
covariance matrix In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of ...
of a
multivariate probability distribution Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considere ...
is always positive semi-definite; and it is positive definite unless one variable is an exact linear function of the others. Conversely, every positive semi-definite matrix is the covariance matrix of some multivariate distribution.


Extension for non-Hermitian square matrices

The definition of positive definite can be generalized by designating any complex matrix M (e.g. real non-symmetric) as positive definite if \Re\left(\mathbf^* M\mathbf\right) > 0 for all non-zero complex vectors \mathbf z, where \Re(c) denotes the real part of a complex number c.Weisstein, Eric W.
Positive Definite Matrix.
' From ''MathWorld--A Wolfram Web Resource''. Accessed on 2012-07-26
Only the Hermitian part \frac\left(M + M^*\right) determines whether the matrix is positive definite, and is assessed in the narrower sense above. Similarly, if \mathbf x and M are real, we have \mathbf^\textsf M \mathbf > 0 for all real nonzero vectors \mathbf x if and only if the symmetric part \frac\left(M + M^\textsf\right) is positive definite in the narrower sense. It is immediately clear that \mathbf^\textsf M \mathbf = \sum_ x_i M_ x_jis insensitive to transposition of ''M''. Consequently, a non-symmetric real matrix with only positive eigenvalues does not need to be positive definite. For example, the matrix M = \left begin 4 & 9 \\ 1 & 4 \end\right/math> has positive eigenvalues yet is not positive definite; in particular a negative value of \mathbf^\textsf M\mathbf is obtained with the choice \mathbf = \left begin -1 \\ 1 \end\right (which is the eigenvector associated with the negative eigenvalue of the symmetric part of In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on a complex Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the
polarization identity In linear algebra, a branch of mathematics, the polarization identity is any one of a family of formulas that express the inner product of two vectors in terms of the norm of a normed vector space. If a norm arises from an inner product then t ...
. That is no longer true in the real case.


Applications


Heat conductivity matrix

Fourier's law of heat conduction, giving heat flux \mathbf q in terms of the temperature gradient \mathbf g = \nabla T is written for anisotropic media as \mathbf = -K\mathbf, in which K is the symmetric
thermal conductivity The thermal conductivity of a material is a measure of its ability to conduct heat. It is commonly denoted by k, \lambda, or \kappa. Heat transfer occurs at a lower rate in materials of low thermal conductivity than in materials of high thermal ...
matrix. The negative is inserted in Fourier's law to reflect the expectation that heat will always flow from hot to cold. In other words, since the temperature gradient \mathbf g always points from cold to hot, the heat flux \mathbf q is expected to have a negative inner product with \mathbf g so that \mathbf^\textsf\mathbf < 0. Substituting Fourier's law then gives this expectation as \mathbf^\textsfK\mathbf > 0, implying that the conductivity matrix should be positive definite.


See also

*
Covariance matrix In probability theory and statistics, a covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of ...
*
M-matrix In mathematics, especially linear algebra, an ''M''-matrix is a ''Z''-matrix with eigenvalues whose real parts are nonnegative. The set of non-singular ''M''-matrices are a subset of the class of ''P''-matrices, and also of the class of inverse-p ...
*
Positive-definite function In mathematics, a positive-definite function is, depending on the context, either of two types of function. Most common usage A ''positive-definite function'' of a real variable ''x'' is a complex-valued function f: \mathbb \to \mathbb such ...
*
Positive-definite kernel In operator theory, a branch of mathematics, a positive-definite kernel is a generalization of a positive-definite function or a positive-definite matrix. It was first introduced by James Mercer in the early 20th century, in the context of solving ...
*
Schur complement In linear algebra and the theory of matrices, the Schur complement of a block matrix is defined as follows. Suppose ''p'', ''q'' are nonnegative integers, and suppose ''A'', ''B'', ''C'', ''D'' are respectively ''p'' × ''p'', ''p'' × ''q'', ''q'' ...
*
Sylvester's criterion In mathematics, Sylvester’s criterion is a necessary and sufficient criterion to determine whether a Hermitian matrix is positive-definite. It is named after James Joseph Sylvester. Sylvester's criterion states that a ''n'' × ''n'' Hermitian ma ...
*
Numerical range In the mathematical field of linear algebra and convex analysis, the numerical range or field of values of a complex n \times n matrix ''A'' is the set :W(A) = \left\ where \mathbf^* denotes the conjugate transpose of the vector \mathbf. The nume ...


Notes


References

* * *


External links

*
Wolfram MathWorld: Positive Definite Matrix
{{DEFAULTSORT:Positive-Definite Matrix Matrices de:Definitheit#Definitheit von Matrizen