Singular value decomposition

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complexmatrix into a rotation, followed by a scaling, followed by another rotation. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any ⁠${\displaystyle m\times n}$⁠ matrix. It is related to the polar decomposition, and is a common means for implementing low-rank approximation for matrices.

Specifically, the singular value decomposition of an ${\displaystyle m\times n}$ complex matrix ⁠${\displaystyle \mathbf {M} }$⁠ is a factorization of the form ⁠${\displaystyle \mathbf {M} =\mathbf {U} \mathbf {\Sigma } \mathbf {V} ^{*}}$⁠, where ⁠${\displaystyle \mathbf {U} }$⁠ is an ⁠${\displaystyle m\times m}$⁠ complex unitary matrix, ${\displaystyle \mathbf {\Sigma } }$ is an ${\displaystyle m\times n}$rectangular diagonal matrix with non-negative real numbers on the diagonal, ⁠${\displaystyle \mathbf {V} }$⁠ is an ${\displaystyle n\times n}$ complex unitary matrix, and ⁠${\displaystyle \mathbf {V} ^{*}}$⁠ is the conjugate transpose of ⁠${\displaystyle \mathbf {V} }$⁠. Such decompositions always exist for any complex matrix. If ⁠${\displaystyle \mathbf {M} }$⁠ is real, then some ⁠${\displaystyle \mathbf {U} }$⁠ and ⁠${\displaystyle \mathbf {V} }$⁠ can be found which are real (orthogonal) matrices; a real-valued SVD is often denoted ⁠${\displaystyle \mathbf {U} \mathbf {\Sigma } \mathbf {V} ^{\mathsf {T}}}$⁠, where ⁠${\displaystyle \mathbf {V} ^{\mathsf {T}}}$⁠ is the transpose of ⁠${\displaystyle \mathbf {V} }$⁠.

The diagonal entries ${\displaystyle \sigma _{i}=\mathbf {\Sigma } _{i,i}}$ of ${\displaystyle \mathbf {\Sigma } }$ are uniquely determined by ⁠${\displaystyle \mathbf {M} }$⁠, up to reordering, and are known as the singular values of ⁠${\displaystyle \mathbf {M} }$⁠. Conventionally they are arranged in descending order (from largest to smallest), which uniquely determines ⁠${\displaystyle \mathbf {\Sigma } }$⁠. The number of non-zero singular values, allowing repetitions, is equal to ⁠${\displaystyle r}$⁠, the rank of ⁠${\displaystyle \mathbf {M} }$⁠.

The columns of ⁠${\displaystyle \mathbf {U} }$⁠ and the columns of ⁠${\displaystyle \mathbf {V} }$⁠ are called left-singular vectors and right-singular vectors of ⁠${\displaystyle \mathbf {M} }$⁠, respectively. They form two orthonormal bases, ⁠${\displaystyle \{\mathbf {u} _{1},\ldots ,\mathbf {u} _{m}\}}$⁠ and ⁠${\displaystyle \{\mathbf {v} _{1},\ldots ,\mathbf {v} _{n}\}}$⁠. In general the SVD is not unique, with certain unitary transformations of ⁠${\displaystyle \mathbf {U} }$⁠ and ⁠${\displaystyle \mathbf {V} }$⁠ producing valid alternative decompositions.

The term SVD sometimes refers to the compact SVD, a similar decomposition ⁠${\displaystyle \mathbf {M} =\mathbf {U} _{r}\mathbf {\Sigma } _{r}\mathbf {V} _{r}^{*}}$⁠, in which ⁠${\displaystyle \mathbf {\Sigma } _{r}}$⁠ is an ⁠${\displaystyle r\times r}$⁠ matrix with only the non-zero singular values (allowing repetitions) on its main diagonal. In this variant, ⁠${\displaystyle \mathbf {U} _{r}}$⁠ is an ⁠${\displaystyle m\times r}$⁠semi-unitary matrix whose columns ⁠${\displaystyle \{\mathbf {u} _{1},\ldots ,\mathbf {u} _{r}\}}$⁠span the columns of ⁠${\displaystyle \mathbf {M} }$⁠, and ${\displaystyle \mathbf {V} _{r}}$ is an ⁠${\displaystyle n\times r}$⁠ semi-unitary matrix whose columns ⁠${\displaystyle \{\mathbf {v} _{1},\ldots ,\mathbf {v} _{r}\}}$⁠ span the columns of ⁠${\displaystyle \mathbf {M} ^{*}\!}$⁠.

The SVD (with sorted singular values) splits ⁠${\displaystyle \mathbf {M} }$⁠ into a sum of ⁠${\displaystyle r}$⁠ rank-⁠${\displaystyle 1}$⁠ matrices, ${\displaystyle \textstyle \mathbf {M} =\sigma _{1}\mathbf {u} _{1}\mathbf {v} _{1}^{*}+\sigma _{2}\mathbf {u} _{2}\mathbf {v} _{2}^{*}+\cdots +\sigma _{r}\mathbf {u} _{r}\mathbf {v} _{r}^{*}\!}$.

Mathematical applications of the SVD include computing the pseudoinverse, matrix approximation, and determining the rank, range, and null space of a matrix. The SVD is also extremely useful in many areas of science, engineering, and statistics, such as signal processing, least squares fitting of data, and process control.