Gramian matrix

In linear algebra, the Gram matrix (or Gramian matrix, Gramian) of a set of vectors $v_1,\dots, v_n$ in an inner product space is the Hermitian matrix of inner products, whose entries are given by the inner product $G_ = \left\langle v_i, v_j \right\rangle$., p.441, Theorem 7.2.10 If the vectors $v_1,\dots, v_n$ are the columns of matrix $X$ then the Gram matrix is $X^* X$ in the general case that the vector coordinates are complex numbers, which simplifies to $X^\top X$ for the case that the vector coordinates are real numbers.

An important application is to compute linear independence: a set of vectors are linearly independent if and only if the Gram determinant (the determinant of the Gram matrix) is non-zero.

It is named after Jørgen Pedersen Gram.

Examples

For finite-dimensional real vectors in $\mathbb^n$ with the usual Euclidean dot product, the Gram matrix is $G = V^\top V$, where $V$ is a matrix whose columns are the vectors $v_k$ and $V^\top$ is its transpose whose rows are the vectors $v_k^\top$. For complex vectors in $\mathbb^n$, $G = V^\dagger V$, where $V^\dagger$ is the conjugate transpose of $V$.

Given square-integrable functions $\$ on the interval \left _0, t_f\right/math>, the Gram matrix G = \left _\right/math> is:

: $G_ = \int_^ \ell_i^*(\tau)\ell_j(\tau)\, d\tau.$
where $\ell_i^*(\tau)$ is the complex conjugate of $\ell_i(\tau)$.

For any bilinear form $B$ on a finite-dimensional vector space over any field we can define a Gram matrix $G$ attached to a set of vectors $v_1, \dots, v_n$ by $G_ = B\left(v_i, v_j\right)$. The matrix will be symmetric if the bilinear form $B$ is symmetric.

Applications

* In Riemannian geometry, given an embedded $k$-dimensional Riemannian manifold $M\subset \mathbb^n$ and a parametrization $\phi: U\to M$ for the volume form $\omega$ on $M$ induced by the embedding may be computed using the Gramian of the coordinate tangent vectors: \omega = \sqrt\ dx_1 \cdots dx_k,\quad G = \left left\langle \frac,\frac\right\rangle\right This generalizes the classical surface integral of a parametrized surface $\phi:U\to S\subset \mathbb^3$ for $(x, y)\in U\subset\mathbb^2$: $\int_S f\ dA = \iint_U f(\phi(x, y))\, \left, \frac\,\,\frac\\, dx\, dy.$
* If the vectors are centered random variables, the Gramian is approximately proportional to the covariance matrix, with the scaling determined by the number of elements in the vector.
* In quantum chemistry, the Gram matrix of a set of basis vectors is the overlap matrix.
* In control theory (or more generally systems theory), the controllability Gramian and observability Gramian determine properties of a linear system.
* Gramian matrices arise in covariance structure model fitting (see e.g., Jamshidian and Bentler, 1993, Applied Psychological Measurement, Volume 18, pp. 79–94).
* In the finite element method, the Gram matrix arises from approximating a function from a finite dimensional space; the Gram matrix entries are then the inner products of the basis functions of the finite dimensional subspace.
* In machine learning, kernel functions are often represented as Gram matrices. (Also see kernel PCA)
* Since the Gram matrix over the reals is a symmetric matrix, it is diagonalizable and its eigenvalues are non-negative. The diagonalization of the Gram matrix is the singular value decomposition.

Properties

Positive-semidefiniteness

The Gram matrix is symmetric in the case the real product is real-valued; it is Hermitian in the general, complex case by definition of an inner product.

The Gram matrix is positive semidefinite, and every positive semidefinite matrix is the Gramian matrix for some set of vectors. The fact that the Gramian matrix is positive-semidefinite can be seen from the following simple derivation:
: $x^\dagger \mathbf x = \sum_x_i^* x_j\left\langle v_i, v_j \right\rangle = \sum_\left\langle x_i v_i, x_j v_j \right\rangle = \left\langle \sum_i x_i v_i, \sum_j x_j v_j \right\rangle = \left\, \sum_i x_i v_i \right\, ^2 \geq 0 .$

The first equality follows from the definition of matrix multiplication, the second and third from the bi-linearity of the inner-product, and the last from the positive definiteness of the inner product.
Note that this also shows that the Gramian matrix is positive definite if and only if the vectors $v_i$ are linearly independent (that is, $\sum_i x_i v_i \neq 0$ for all $x$).

Finding a vector realization

Given any positive semidefinite matrix $M$, one can decompose it as:
: $M = B^\dagger B$,

where $B^\dagger$ is the conjugate transpose of $B$ (or $M = B^\textsf B$