In
mathematics
Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
, and in particular
linear algebra
Linear algebra is the branch of mathematics concerning linear equations such as:
:a_1x_1+\cdots +a_nx_n=b,
linear maps such as:
:(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n,
and their representations in vector spaces and through matrices.
...
, the Moore–Penrose inverse of a
matrix
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** ''The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
is the most widely known
generalization
A generalization is a form of abstraction whereby common properties of specific instances are formulated as general concepts or claims. Generalizations posit the existence of a domain or set of elements, as well as one or more common characte ...
of the
inverse matrix
In linear algebra, an -by- square matrix is called invertible (also nonsingular or nondegenerate), if there exists an -by- square matrix such that
:\mathbf = \mathbf = \mathbf_n \
where denotes the -by- identity matrix and the multiplicati ...
. It was independently described by
E. H. Moore
Eliakim Hastings Moore (; January 26, 1862 – December 30, 1932), usually cited as E. H. Moore or E. Hastings Moore, was an American mathematician.
Life
Moore, the son of a Methodist minister and grandson of US Congressman Eliakim H. Moore, di ...
in 1920,
Arne Bjerhammar in 1951, and
Roger Penrose
Sir Roger Penrose (born 8 August 1931) is an English mathematician, mathematical physicist, philosopher of science and Nobel Laureate in Physics. He is Emeritus Rouse Ball Professor of Mathematics in the University of Oxford, an emeritus fello ...
in 1955. Earlier,
Erik Ivar Fredholm
Erik Ivar Fredholm (7 April 1866 – 17 August 1927) was a Swedish mathematician whose work on integral equations and operator theory foreshadowed the theory of Hilbert spaces.
Biography
Fredholm was born in Stockholm in 1866. He obtained his P ...
had introduced the concept of a pseudoinverse of
integral operator
An integral operator is an operator that involves integration. Special instances are:
* The operator of integration itself, denoted by the integral symbol
* Integral linear operators, which are linear operators induced by bilinear forms invol ...
s in 1903. When referring to a matrix, the term
pseudoinverse
In mathematics, and in particular, algebra, a generalized inverse (or, g-inverse) of an element ''x'' is an element ''y'' that has some properties of an inverse element but not necessarily all of them. The purpose of constructing a generalized inv ...
, without further specification, is often used to indicate the Moore–Penrose inverse. The term
generalized inverse
In mathematics, and in particular, algebra, a generalized inverse (or, g-inverse) of an element ''x'' is an element ''y'' that has some properties of an inverse element but not necessarily all of them. The purpose of constructing a generalized in ...
is sometimes used as a synonym for pseudoinverse.
A common use of the pseudoinverse is to compute a "best fit" (
least squares) solution to a
system of linear equations that lacks a solution (see below under
§ Applications).
Another use is to find the minimum (
Euclidean) norm solution to a system of linear equations with multiple solutions. The pseudoinverse facilitates the statement and proof of results in linear algebra.
The pseudoinverse is defined and unique for all matrices whose entries are
real
Real may refer to:
Currencies
* Brazilian real (R$)
* Central American Republic real
* Mexican real
* Portuguese real
* Spanish real
* Spanish colonial real
Music Albums
* ''Real'' (L'Arc-en-Ciel album) (2000)
* ''Real'' (Bright album) (2010) ...
or
complex
Complex commonly refers to:
* Complexity, the behaviour of a system whose components interact in multiple ways so possible interactions are difficult to describe
** Complex system, a system composed of many components which may interact with each ...
numbers. It can be computed using the
singular value decomposition
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any \ m \times n\ matrix. It is re ...
. In the special case where is a
normal matrix In mathematics, a complex square matrix is normal if it commutes with its conjugate transpose :
The concept of normal matrices can be extended to normal operators on infinite dimensional normed spaces and to normal elements in C*-algebras. As ...
(for example, a Hermitian matrix), the pseudoinverse annihilates the
kernel
Kernel may refer to:
Computing
* Kernel (operating system), the central component of most operating systems
* Kernel (image processing), a matrix used for image convolution
* Compute kernel, in GPGPU programming
* Kernel method, in machine learn ...
of and acts as a traditional inverse of on the subspace orthogonal to the kernel.
Notation
In the following discussion, the following conventions are adopted.
* will denote one of the
fields
Fields may refer to:
Music
* Fields (band), an indie rock band formed in 2006
* Fields (progressive rock band), a progressive rock band formed in 1971
* ''Fields'' (album), an LP by Swedish-based indie rock band Junip (2010)
* "Fields", a song b ...
of real or complex numbers, denoted , , respectively. The vector space of matrices over is denoted by .
* For , and denote the transpose and Hermitian transpose (also called
conjugate transpose
In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex c ...
) respectively. If
, then
.
* For , (standing for "
range
Range may refer to:
Geography
* Range (geographic), a chain of hills or mountains; a somewhat linear, complex mountainous or hilly area (cordillera, sierra)
** Mountain range, a group of mountains bordered by lowlands
* Range, a term used to i ...
") denotes the
column space
In linear algebra, the column space (also called the range or image) of a matrix ''A'' is the span (set of all possible linear combinations) of its column vectors. The column space of a matrix is the image or range of the corresponding mat ...
(image) of (the space spanned by the column vectors of ) and denotes the
kernel
Kernel may refer to:
Computing
* Kernel (operating system), the central component of most operating systems
* Kernel (image processing), a matrix used for image convolution
* Compute kernel, in GPGPU programming
* Kernel method, in machine learn ...
(null space) of .
* Finally, for any positive integer , denotes the
identity matrix.
Definition
For , a pseudoinverse of is defined as a matrix satisfying all of the following four criteria, known as the Moore–Penrose conditions:
# need not be the general identity matrix, but it maps all column vectors of to themselves:
# acts like a
weak inverse
In mathematics, the term weak inverse is used with several meanings.
Theory of semigroups
In the theory of semigroups, a weak inverse of an element ''x'' in a semigroup is an element ''y'' such that . If every element has a weak inverse, the s ...
:
# is
Hermitian {{Short description, none
Numerous things are named after the French mathematician Charles Hermite (1822–1901):
Hermite
* Cubic Hermite spline, a type of third-degree spline
* Gauss–Hermite quadrature, an extension of Gaussian quadrature m ...
:
# is also Hermitian:
exists for any matrix , but, when the latter has full
rank
Rank is the relative position, value, worth, complexity, power, importance, authority, level, etc. of a person or object within a ranking, such as:
Level or position in a hierarchical organization
* Academic rank
* Diplomatic rank
* Hierarchy
* ...
(that is, the rank of is ), then can be expressed as a simple algebraic formula.
In particular, when has linearly independent columns (and thus matrix is invertible), can be computed as
This particular pseudoinverse constitutes a ''left inverse'', since, in this case,
.
When has linearly independent rows (matrix is invertible), can be computed as
This is a ''right inverse'', as
.
Properties
Existence and uniqueness
The pseudoinverse exists and is unique: for any matrix , there is precisely one matrix , that satisfies the four properties of the definition.
A matrix satisfying the first condition of the definition is known as a generalized inverse. If the matrix also satisfies the second definition, it is called a
generalized ''reflexive'' inverse. Generalized inverses always exist but are not in general unique. Uniqueness is a consequence of the last two conditions.
Basic properties
Proofs for the properties below can be found at
b:Topics in Abstract Algebra/Linear algebra.
* If has real entries, then so does .
* If is
invertible
In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers.
Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
, its pseudoinverse is its inverse. That is,
.
[.]
* For any
and any
where
(We use
for "reciprocal".)
* More generally, for any
and any
the
rectangular diagonal matrix
In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal m ...
where
* It follows that the pseudoinverse of a
zero matrix In mathematics, particularly linear algebra, a zero matrix or null matrix is a matrix all of whose entries are zero. It also serves as the additive identity of the additive group of m \times n matrices, and is denoted by the symbol O or 0 followed ...
is its transpose.
* The pseudoinverse of the pseudoinverse is the original matrix:
.
* Pseudoinversion commutes with transposition, complex conjugation, and taking the conjugate transpose:
* The pseudoinverse of a scalar multiple of is the reciprocal multiple of :`
for .
Identities
The following identity formula can be used to cancel or expand certain subexpressions involving pseudoinverses:
Equivalently, substituting
for
gives
while substituting
for
gives
Reduction to Hermitian case
The computation of the pseudoinverse is reducible to its construction in the Hermitian case. This is possible through the equivalences:
as and are Hermitian.
Products
Suppose . Then the following are equivalent:
#
#
#
#
#
The following are sufficient conditions for :
# has orthonormal columns (then
), or
# has orthonormal rows (then
), or
# has linearly independent columns (then
) and has linearly independent rows (then
), or
#
, or
#
.
The following is a necessary condition for :
#
The last sufficient condition yields the equalities
NB: The equality does not hold in general.
See the counterexample:
Projectors
and
are
orthogonal projection operators, that is, they are Hermitian (
,
) and idempotent (
and
). The following hold:
*
and
* is the
orthogonal projector onto the
range
Range may refer to:
Geography
* Range (geographic), a chain of hills or mountains; a somewhat linear, complex mountainous or hilly area (cordillera, sierra)
** Mountain range, a group of mountains bordered by lowlands
* Range, a term used to i ...
of (which equals the
orthogonal complement In the mathematical fields of linear algebra and functional analysis, the orthogonal complement of a subspace ''W'' of a vector space ''V'' equipped with a bilinear form ''B'' is the set ''W''⊥ of all vectors in ''V'' that are orthogonal to every ...
of the kernel of ).
* is the orthogonal projector onto the range of (which equals the orthogonal complement of the kernel of ).
*
is the orthogonal projector onto the kernel of .
*
is the orthogonal projector onto the kernel of .
The last two properties imply the following identities:
*
*
Another property is the following: if is Hermitian and idempotent (true if and only if it represents an orthogonal projection), then, for any matrix the following equation holds:
This can be proven by defining matrices
,
, and checking that is indeed a pseudoinverse for by verifying that the defining properties of the pseudoinverse hold, when is Hermitian and idempotent.
From the last property it follows that, if is Hermitian and idempotent, for any matrix
Finally, if is an orthogonal projection matrix, then its pseudoinverse trivially coincides with the matrix itself, that is,
.
Geometric construction
If we view the matrix as a linear map over the field then can be decomposed as follows. We write for the
direct sum, for the
orthogonal complement In the mathematical fields of linear algebra and functional analysis, the orthogonal complement of a subspace ''W'' of a vector space ''V'' equipped with a bilinear form ''B'' is the set ''W''⊥ of all vectors in ''V'' that are orthogonal to every ...
, for the
kernel
Kernel may refer to:
Computing
* Kernel (operating system), the central component of most operating systems
* Kernel (image processing), a matrix used for image convolution
* Compute kernel, in GPGPU programming
* Kernel method, in machine learn ...
of a map, and for the
image
An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensiona ...
of a map. Notice that
and
. The restriction
is then an isomorphism. This implies that on is the inverse of this isomorphism, and is zero on
In other words: To find for given in , first project orthogonally onto the range of , finding a point in the range. Then form , that is, find those vectors in that sends to . This will be an affine subspace of parallel to the kernel of . The element of this subspace that has the smallest length (that is, is closest to the origin) is the answer we are looking for. It can be found by taking an arbitrary member of and projecting it orthogonally onto the orthogonal complement of the kernel of .
This description is closely related to the
Minimum norm solution to a linear system.
Subspaces
Limit relations
The pseudoinverse are limits:
(see
Tikhonov regularization
Ridge regression is a method of estimating the coefficients of multiple-regression models in scenarios where the independent variables are highly correlated. It has been used in many fields including econometrics, chemistry, and engineering. Also ...
). These limits exist even if or do not exist.
Continuity
In contrast to ordinary matrix inversion, the process of taking pseudoinverses is not
continuous
Continuity or continuous may refer to:
Mathematics
* Continuity (mathematics), the opposing concept to discreteness; common examples include
** Continuous probability distribution or random variable in probability and statistics
** Continuous ...
: if the sequence converges to the matrix (in the
maximum norm or Frobenius norm, say), then need not converge to . However, if all the matrices have the same rank as , will converge to .
Derivative
The derivative of a real valued pseudoinverse matrix which has constant rank at a point may be calculated in terms of the derivative of the original matrix:
Examples
Since for invertible matrices the pseudoinverse equals the usual inverse, only examples of non-invertible matrices are considered below.
Special cases
Scalars
It is also possible to define a pseudoinverse for scalars and vectors. This amounts to treating these as matrices. The pseudoinverse of a scalar is zero if is zero and the reciprocal of otherwise:
Vectors
The pseudoinverse of the null (all zero) vector is the transposed null vector. The pseudoinverse of a non-null vector is the conjugate transposed vector divided by its squared magnitude:
Linearly independent columns
If the rank of is identical to its column rank, , (for ,) there are
linearly independent
In the theory of vector spaces, a set of vectors is said to be if there is a nontrivial linear combination of the vectors that equals the zero vector. If no such linear combination exists, then the vectors are said to be . These concepts are ...
columns, and is invertible. In this case, an explicit formula is:
It follows that is then a left inverse of :
.
Linearly independent rows
If the rank of is identical to its row rank, , (for ,) there are
linearly independent
In the theory of vector spaces, a set of vectors is said to be if there is a nontrivial linear combination of the vectors that equals the zero vector. If no such linear combination exists, then the vectors are said to be . These concepts are ...
rows, and is invertible. In this case, an explicit formula is:
It follows that is a right inverse of :
.
Orthonormal columns or rows
This is a special case of either full column rank or full row rank (treated above). If has orthonormal columns (
) or orthonormal rows (
), then:
Normal matrices
If is
normal Normal(s) or The Normal(s) may refer to:
Film and television
* ''Normal'' (2003 film), starring Jessica Lange and Tom Wilkinson
* ''Normal'' (2007 film), starring Carrie-Anne Moss, Kevin Zegers, Callum Keith Rennie, and Andrew Airlie
* ''Norma ...
; that is, it commutes with its conjugate transpose, then its pseudoinverse can be computed by diagonalizing it, mapping all nonzero eigenvalues to their inverses, and mapping zero eigenvalues to zero. A corollary is that commuting with its transpose implies that it commutes with its pseudoinverse.
Orthogonal projection matrices
This is a special case of a Normal matrix with eigenvalues 0 and 1. If is an orthogonal projection matrix, that is,
and
, then the pseudoinverse trivially coincides with the matrix itself:
Circulant matrices
For a
circulant matrix
In linear algebra, a circulant matrix is a square matrix in which all row vectors are composed of the same elements and each row vector is rotated one element to the right relative to the preceding row vector. It is a particular kind of Toeplit ...
, the singular value decomposition is given by the
Fourier transform
A Fourier transform (FT) is a mathematical transform that decomposes functions into frequency components, which are represented by the output of the transform as a function of frequency. Most commonly functions of time or space are transformed, ...
, that is, the singular values are the Fourier coefficients. Let be the
Discrete Fourier Transform (DFT) matrix; then
Construction
Rank decomposition
Let denote the
rank
Rank is the relative position, value, worth, complexity, power, importance, authority, level, etc. of a person or object within a ranking, such as:
Level or position in a hierarchical organization
* Academic rank
* Diplomatic rank
* Hierarchy
* ...
of . Then can be
(rank) decomposed as
where and are of rank . Then
.
The QR method
For
computing the product or and their inverses explicitly is often a source of numerical rounding errors and computational cost in practice. An alternative approach using the
QR decomposition
In linear algebra, a QR decomposition, also known as a QR factorization or QU factorization, is a decomposition of a matrix ''A'' into a product ''A'' = ''QR'' of an orthogonal matrix ''Q'' and an upper triangular matrix ''R''. QR decomp ...
of may be used instead.
Consider the case when is of full column rank, so that
. Then the
Cholesky decomposition
In linear algebra, the Cholesky decomposition or Cholesky factorization (pronounced ) is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose, which is useful for effici ...
, where is an
upper triangular matrix
In mathematics, a triangular matrix is a special kind of square matrix. A square matrix is called if all the entries ''above'' the main diagonal are zero. Similarly, a square matrix is called if all the entries ''below'' the main diagonal are ...
, may be used. Multiplication by the inverse is then done easily by solving a system with multiple right-hand sides,
which may be solved by
forward substitution
In mathematics, a triangular matrix is a special kind of square matrix. A square matrix is called if all the entries ''above'' the main diagonal are zero. Similarly, a square matrix is called if all the entries ''below'' the main diagonal are ...
followed by
back substitution
In mathematics, a triangular matrix is a special kind of square matrix. A square matrix is called if all the entries ''above'' the main diagonal are zero. Similarly, a square matrix is called if all the entries ''below'' the main diagonal are ...
.
The Cholesky decomposition may be computed without forming explicitly, by alternatively using the
QR decomposition
In linear algebra, a QR decomposition, also known as a QR factorization or QU factorization, is a decomposition of a matrix ''A'' into a product ''A'' = ''QR'' of an orthogonal matrix ''Q'' and an upper triangular matrix ''R''. QR decomp ...
of
, where
has orthonormal columns,
, and is upper triangular. Then
so is the Cholesky factor of .
The case of full row rank is treated similarly by using the formula
and using a similar argument, swapping the roles of and .
Singular value decomposition (SVD)
A computationally simple and accurate way to compute the pseudoinverse is by using the
singular value decomposition
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any \ m \times n\ matrix. It is re ...
.
[Linear Systems & Pseudo-Inverse](_blank)
/ref> If is the singular value decomposition of , then . For a rectangular diagonal matrix
In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal m ...
such as , we get the pseudoinverse by taking the reciprocal of each non-zero element on the diagonal, leaving the zeros in place, and then transposing the matrix. In numerical computation, only elements larger than some small tolerance are taken to be nonzero, and the others are replaced by zeros. For example, in the MATLAB
MATLAB (an abbreviation of "MATrix LABoratory") is a proprietary multi-paradigm programming language and numeric computing environment developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementation ...
or GNU Octave
GNU Octave is a high-level programming language primarily intended for scientific computing and numerical computation. Octave helps in solving linear and nonlinear problems numerically, and for performing other numerical experiments using a langu ...
function , the tolerance is taken to be , where ε is the machine epsilon.
The computational cost of this method is dominated by the cost of computing the SVD, which is several times higher than matrix–matrix multiplication, even if a state-of-the art implementation (such as that of LAPACK
LAPACK ("Linear Algebra Package") is a standard software library for numerical linear algebra. It provides routines for solving systems of linear equations and linear least squares, eigenvalue problems, and singular value decomposition. It als ...
) is used.
The above procedure shows why taking the pseudoinverse is not a continuous operation: if the original matrix has a singular value 0 (a diagonal entry of the matrix above), then modifying slightly may turn this zero into a tiny positive number, thereby affecting the pseudoinverse dramatically as we now have to take the reciprocal of a tiny number.
Block matrices
Optimized approaches exist for calculating the pseudoinverse of block structured matrices.
The iterative method of Ben-Israel and Cohen
Another method for computing the pseudoinverse (cf. Drazin inverse In mathematics, the Drazin inverse, named after Michael P. Drazin, is a kind of generalized inverse of a matrix.
Let ''A'' be a square matrix. The index of ''A'' is the least nonnegative integer ''k'' such that rank(''A'k''+1) = rank(''A'k'') ...
) uses the recursion
which is sometimes referred to as hyper-power sequence. This recursion produces a sequence converging quadratically to the pseudoinverse of if it is started with an appropriate satisfying . The choice (where , with denoting the largest singular value of ) has been argued not to be competitive to the method using the SVD mentioned above, because even for moderately ill-conditioned matrices it takes a long time before enters the region of quadratic convergence. However, if started with already close to the Moore–Penrose inverse and , for example , convergence is fast (quadratic).
Updating the pseudoinverse
For the cases where has full row or column rank, and the inverse of the correlation matrix ( for with full row rank or for full column rank) is already known, the pseudoinverse for matrices related to can be computed by applying the Sherman–Morrison–Woodbury formula to update the inverse of the correlation matrix, which may need less work. In particular, if the related matrix differs from the original one by only a changed, added or deleted row or column, incremental algorithms exist that exploit the relationship.
Similarly, it is possible to update the Cholesky factor when a row or column is added, without creating the inverse of the correlation matrix explicitly. However, updating the pseudoinverse in the general rank-deficient case is much more complicated.
Software libraries
High-quality implementations of SVD, QR, and back substitution are available in standard libraries, such as LAPACK
LAPACK ("Linear Algebra Package") is a standard software library for numerical linear algebra. It provides routines for solving systems of linear equations and linear least squares, eigenvalue problems, and singular value decomposition. It als ...
. Writing one's own implementation of SVD is a major programming project that requires a significant numerical expertise. In special circumstances, such as parallel computing
Parallel computing is a type of computation in which many calculations or processes are carried out simultaneously. Large problems can often be divided into smaller ones, which can then be solved at the same time. There are several different fo ...
or embedded computing, however, alternative implementations by QR or even the use of an explicit inverse might be preferable, and custom implementations may be unavoidable.
The Python package NumPy provides a pseudoinverse calculation through its functions matrix.I
and linalg.pinv
; its pinv
uses the SVD-based algorithm. SciPy
SciPy (pronounced "sigh pie") is a free and open-source Python library used for scientific computing and technical computing.
SciPy contains modules for optimization, linear algebra, integration, interpolation, special functions, FFT, ...
adds a function scipy.linalg.pinv
that uses a least-squares solver.
The MASS package for R provides a calculation of the Moore–Penrose inverse through the ginv
function. The ginv
function calculates a pseudoinverse using the singular value decomposition provided by the svd
function in the base R package. An alternative is to employ the pinv
function available in the pracma package.
The Octave programming language provides a pseudoinverse through the standard package function pinv
and the pseudo_inverse()
method.
In Julia (programming language)
Julia is a high-level, dynamic programming language. Its features are well suited for numerical analysis and computational science.
Distinctive aspects of Julia's design include a type system with parametric polymorphism in a dynamic programmi ...
, the LinearAlgebra package of the standard library provides an implementation of the Moore–Penrose inverse pinv()
implemented via singular-value decomposition.
Applications
Linear least-squares
The pseudoinverse provides a least squares solution to a system of linear equations.
For , given a system of linear equations
in general, a vector that solves the system may not exist, or if one does exist, it may not be unique. The pseudoinverse solves the "least-squares" problem as follows:
* , we have where and denotes the Euclidean norm
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean s ...
. This weak inequality holds with equality if and only if for any vector ; this provides an infinitude of minimizing solutions unless has full column rank, in which case is a zero matrix. The solution with minimum Euclidean norm is [
This result is easily extended to systems with multiple right-hand sides, when the Euclidean norm is replaced by the Frobenius norm. Let .
* , we have where and denotes the ]Frobenius norm
In mathematics, a matrix norm is a vector norm in a vector space whose elements (vectors) are matrices (of given dimensions).
Preliminaries
Given a field K of either real or complex numbers, let K^ be the -vector space of matrices with m ro ...
.
Obtaining all solutions of a linear system
If the linear system
has any solutions, they are all given by
for arbitrary vector . Solution(s) exist if and only if .[ If the latter holds, then the solution is unique if and only if has full column rank, in which case is a zero matrix. If solutions exist but does not have full column rank, then we have an ]indeterminate system In mathematics, particularly in algebra, an indeterminate system is a system of simultaneous equations (e.g., linear equations) which has more than one solution (sometimes infinitely many solutions). In the case of a linear system, the system may b ...
, all of whose infinitude of solutions are given by this last equation.
Minimum norm solution to a linear system
For linear systems with non-unique solutions (such as under-determined systems), the pseudoinverse may be used to construct the solution of minimum Euclidean norm
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean s ...
among all solutions.
* If is satisfiable, the vector is a solution, and satisfies for all solutions.
This result is easily extended to systems with multiple right-hand sides, when the Euclidean norm is replaced by the Frobenius norm. Let .
* If is satisfiable, the matrix is a solution, and satisfies for all solutions.
Condition number
Using the pseudoinverse and a matrix norm
In mathematics, a matrix norm is a vector norm in a vector space whose elements (vectors) are matrices (of given dimensions).
Preliminaries
Given a field K of either real or complex numbers, let K^ be the -vector space of matrices with m ro ...
, one can define a condition number for any matrix:
A large condition number implies that the problem of finding least-squares solutions to the corresponding system of linear equations is ill-conditioned in the sense that small errors in the entries of can lead to huge errors in the entries of the solution.[
]
Generalizations
Besides for matrices over real and complex numbers, the conditions hold for matrices over biquaternion
In abstract algebra, the biquaternions are the numbers , where , and are complex numbers, or variants thereof, and the elements of multiply as in the quaternion group and commute with their coefficients. There are three types of biquaternions co ...
s, also called "complex quaternions".
In order to solve more general least-squares problems, one can define Moore–Penrose inverses for all continuous linear operators between two Hilbert space
In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natural ...
s and , using the same four conditions as in our definition above. It turns out that not every continuous linear operator has a continuous linear pseudoinverse in this sense. Those that do are precisely the ones whose range is closed in .
A notion of pseudoinverse exists for matrices over an arbitrary field
Field may refer to:
Expanses of open ground
* Field (agriculture), an area of land used for agricultural purposes
* Airfield, an aerodrome that lacks the infrastructure of an airport
* Battlefield
* Lawn, an area of mowed grass
* Meadow, a grass ...
equipped with an arbitrary involutive automorphism. In this more general setting, a given matrix doesn't always have a pseudoinverse. The necessary and sufficient condition for a pseudoinverse to exist is that , where denotes the result of applying the involution operation to the transpose of . When it does exist, it is unique. Example: Consider the field of complex numbers equipped with the identity involution (as opposed to the involution considered elsewhere in the article); do there exist matrices that fail to have pseudoinverses in this sense? Consider the matrix . Observe that while . So this matrix doesn't have a pseudoinverse in this sense.
In abstract algebra
In mathematics, more specifically algebra, abstract algebra or modern algebra is the study of algebraic structures. Algebraic structures include groups, rings, fields, modules, vector spaces, lattices, and algebras over a field. The term ''a ...
, a Moore–Penrose inverse may be defined on a *-regular semigroup. This abstract definition coincides with the one in linear algebra.
See also
* Drazin inverse In mathematics, the Drazin inverse, named after Michael P. Drazin, is a kind of generalized inverse of a matrix.
Let ''A'' be a square matrix. The index of ''A'' is the least nonnegative integer ''k'' such that rank(''A'k''+1) = rank(''A'k'') ...
* Hat matrix
In statistics, the projection matrix (\mathbf), sometimes also called the influence matrix or hat matrix (\mathbf), maps the vector of response values (dependent variable values) to the vector of fitted values (or predicted values). It describes t ...
* Inverse element
In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers.
Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that is ...
* Linear least squares (mathematics)
Linear least squares (LLS) is the least squares approximation of linear functions to data.
It is a set of formulations for solving statistical problems involved in linear regression, including variants for ordinary (unweighted), weighted, and ...
* Pseudo-determinant In linear algebra and statistics, the pseudo-determinant is the product of all non-zero eigenvalues of a square matrix. It coincides with the regular determinant when the matrix is non-singular.
Definition
The pseudo-determinant of a square '' ...
* Von Neumann regular ring
In mathematics, a von Neumann regular ring is a ring ''R'' (associative, with 1, not necessarily commutative) such that for every element ''a'' in ''R'' there exists an ''x'' in ''R'' with . One may think of ''x'' as a "weak inverse" of the elemen ...
Notes
References
*
*
*
*
External links
*
Interactive program & tutorial of Moore–Penrose Pseudoinverse
*
*
*
The Moore–Penrose Pseudoinverse. A Tutorial Review of the Theory
{{DEFAULTSORT:Moore-Penrose inverse
Matrix theory
Singular value decomposition
Numerical linear algebra