HOME

TheInfoList



OR:

In
linear algebra Linear algebra is the branch of mathematics concerning linear equations such as: :a_1x_1+\cdots +a_nx_n=b, linear maps such as: :(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n, and their representations in vector spaces and through matrice ...
and
functional analysis Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure (e.g. inner product, norm, topology, etc.) and the linear functions defi ...
, a projection is a
linear transformation In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that pre ...
P from a
vector space In mathematics and physics, a vector space (also called a linear space) is a set whose elements, often called '' vectors'', may be added together and multiplied ("scaled") by numbers called ''scalars''. Scalars are often real numbers, but can ...
to itself (an
endomorphism In mathematics, an endomorphism is a morphism from a mathematical object to itself. An endomorphism that is also an isomorphism is an automorphism. For example, an endomorphism of a vector space is a linear map , and an endomorphism of a gr ...
) such that P\circ P=P. That is, whenever P is applied twice to any vector, it gives the same result as if it were applied once (i.e. P is
idempotent Idempotence (, ) is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. The concept of idempotence arises in a number of pl ...
). It leaves its
image An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensio ...
unchanged. This definition of "projection" formalizes and generalizes the idea of graphical projection. One can also consider the effect of a projection on a geometrical object by examining the effect of the projection on points in the object.


Definitions

A projection on a vector space V is a linear operator P : V \to V such that P^2 = P. When V has an
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
and is
complete Complete may refer to: Logic * Completeness (logic) * Completeness of a theory, the property of a theory that every formula in the theory's language or its negation is provable Mathematics * The completeness of the real numbers, which implies t ...
(i.e. when V is a
Hilbert space In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natural ...
) the concept of
orthogonality In mathematics, orthogonality is the generalization of the geometric notion of '' perpendicularity''. By extension, orthogonality is also used to refer to the separation of specific features of a system. The term also has specialized meanings in ...
can be used. A projection P on a Hilbert space V is called an orthogonal projection if it satisfies \langle P \mathbf x, \mathbf y \rangle = \langle \mathbf x, P \mathbf y \rangle for all \mathbf x, \mathbf y \in V. A projection on a Hilbert space that is not orthogonal is called an oblique projection.


Projection matrix

* In the
finite-dimensional In mathematics, the dimension of a vector space ''V'' is the cardinality (i.e., the number of vectors) of a basis of ''V'' over its base field. p. 44, §2.36 It is sometimes called Hamel dimension (after Georg Hamel) or algebraic dimension to d ...
case, a
square matrix In mathematics, a square matrix is a matrix with the same number of rows and columns. An ''n''-by-''n'' matrix is known as a square matrix of order Any two square matrices of the same order can be added and multiplied. Square matrices are often ...
P is called a projection matrix if it is equal to its square, i.e. if P^2 = P. * A square matrix P is called an orthogonal projection matrix if P^2 = P = P^ for a real
matrix Matrix most commonly refers to: * ''The Matrix'' (franchise), an American media franchise ** '' The Matrix'', a 1999 science-fiction action film ** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchi ...
, and respectively P^2 = P = P^ for a complex matrix, where P^ denotes the
transpose In linear algebra, the transpose of a matrix is an operator which flips a matrix over its diagonal; that is, it switches the row and column indices of the matrix by producing another matrix, often denoted by (among other notations). The tr ...
of P and P^ denotes the adjoint or Hermitian transpose of P. * A projection matrix that is not an orthogonal projection matrix is called an oblique projection matrix. The
eigenvalue In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denote ...
s of a projection matrix must be 0 or 1.


Examples


Orthogonal projection

For example, the function which maps the point (x,y,z) in three-dimensional space \mathbb^3 to the point (x,y,0) is an orthogonal projection onto the ''xy''-plane. This function is represented by the matrix P = \begin 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end. The action of this matrix on an arbitrary vector is P \begin x \\ y \\ z \end = \begin x \\ y \\ 0 \end. To see that P is indeed a projection, i.e., P = P^2, we compute P^2 \begin x \\ y \\ z \end = P \begin x \\ y \\ 0 \end = \begin x \\ y \\ 0 \end = P\begin x \\ y \\ z \end. Observing that P^ = P shows that the projection is an orthogonal projection.


Oblique projection

A simple example of a non-orthogonal (oblique) projection is P = \begin 0 & 0 \\ \alpha & 1 \end. Via matrix multiplication, one sees that P^2 = \begin 0 & 0 \\ \alpha & 1 \end \begin 0 & 0 \\ \alpha & 1 \end = \begin 0 & 0 \\ \alpha & 1 \end = P. showing that P is indeed a projection. The projection P is orthogonal
if and only if In logic and related fields such as mathematics and philosophy, "if and only if" (shortened as "iff") is a biconditional logical connective between statements, where either both statements are true or both are false. The connective is bic ...
\alpha = 0 because only then P^ = P.


Properties and classification


Idempotence

By definition, a projection P is
idempotent Idempotence (, ) is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. The concept of idempotence arises in a number of pl ...
(i.e. P^2 = P).


Open map

Every projection is an
open map In mathematics, more specifically in topology, an open map is a function between two topological spaces that maps open sets to open sets. That is, a function f : X \to Y is open if for any open set U in X, the image f(U) is open in Y. Likewise, ...
, meaning that it maps each
open set In mathematics, open sets are a generalization of open intervals in the real line. In a metric space (a set along with a distance defined between any two points), open sets are the sets that, with every point , contain all points that are su ...
in the
domain Domain may refer to: Mathematics *Domain of a function, the set of input values for which the (total) function is defined ** Domain of definition of a partial function ** Natural domain of a partial function **Domain of holomorphy of a function * ...
to an open set in the subspace topology of the
image An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensio ...
. That is, for any vector \mathbf and any ball B_\mathbf (with positive radius) centered on \mathbf, there exists a ball B_ (with positive radius) centered on P\mathbf that is wholly contained in the image P(B_\mathbf).


Complementarity of image and kernel

Let W be a finite-dimensional vector space and P be a projection on W. Suppose the subspaces U and V are the
image An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensio ...
and kernel of P respectively. Then P has the following properties: # P is the identity operator I on U: \forall \mathbf x \in U: P \mathbf x = \mathbf x. # We have a
direct sum The direct sum is an operation between structures in abstract algebra, a branch of mathematics. It is defined differently, but analogously, for different kinds of structures. To see how the direct sum is used in abstract algebra, consider a mor ...
W = U \oplus V. Every vector \mathbf x \in W may be decomposed uniquely as \mathbf x = \mathbf u + \mathbf v with \mathbf u = P \mathbf x and \mathbf v = \mathbf x - P \mathbf x = \left(I-P\right) \mathbf x, and where \mathbf u \in U, \mathbf v \in V. The image and kernel of a projection are ''complementary'', as are P and Q = I - P. The operator Q is also a projection as the image and kernel of P become the kernel and image of Q and vice versa. We say P is a projection along V onto U (kernel/image) and Q is a projection along U onto V.


Spectrum

In infinite-dimensional vector spaces, the
spectrum A spectrum (plural ''spectra'' or ''spectrums'') is a condition that is not limited to a specific set of values but can vary, without gaps, across a continuum. The word was first used scientifically in optics to describe the rainbow of colors ...
of a projection is contained in \ as (\lambda I - P)^ = \frac 1 \lambda I + \frac 1 P. Only 0 or 1 can be an
eigenvalue In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denote ...
of a projection. This implies that an orthogonal projection P is always a positive semi-definite matrix. In general, the corresponding
eigenspace In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denote ...
s are (respectively) the kernel and range of the projection. Decomposition of a vector space into direct sums is not unique. Therefore, given a subspace V, there may be many projections whose range (or kernel) is V. If a projection is nontrivial it has minimal polynomial x^2 - x = x (x-1), which factors into distinct linear factors, and thus P is diagonalizable.


Product of projections

The product of projections is not in general a projection, even if they are orthogonal. If two projections
commute Commute, commutation or commutative may refer to: * Commuting, the process of travelling between a place of residence and a place of work Mathematics * Commutative property, a property of a mathematical operation whose result is insensitive to th ...
then their product is a projection, but the converse is false: the product of two non-commuting projections may be a projection. If two orthogonal projections commute then their product is an orthogonal projection. If the product of two orthogonal projections is an orthogonal projection, then the two orthogonal projections commute (more generally: two self-adjoint
endomorphism In mathematics, an endomorphism is a morphism from a mathematical object to itself. An endomorphism that is also an isomorphism is an automorphism. For example, an endomorphism of a vector space is a linear map , and an endomorphism of a gr ...
s commute if and only if their product is self-adjoint).


Orthogonal projections

When the vector space W has an
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
and is complete (is a
Hilbert space In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natural ...
) the concept of
orthogonality In mathematics, orthogonality is the generalization of the geometric notion of '' perpendicularity''. By extension, orthogonality is also used to refer to the separation of specific features of a system. The term also has specialized meanings in ...
can be used. An orthogonal projection is a projection for which the range U and the null space V are orthogonal subspaces. Thus, for every \mathbf x and \mathbf y in W, \langle P \mathbf x, (\mathbf y - P \mathbf y) \rangle = \langle (\mathbf x - P \mathbf x) , P \mathbf y \rangle = 0. Equivalently: \langle \mathbf x, P \mathbf y \rangle = \langle P \mathbf x, P \mathbf y \rangle = \langle P \mathbf x, \mathbf y \rangle. A projection is orthogonal if and only if it is
self-adjoint In mathematics, and more specifically in abstract algebra, an element ''x'' of a *-algebra is self-adjoint if x^*=x. A self-adjoint element is also Hermitian, though the reverse doesn't necessarily hold. A collection ''C'' of elements of a st ...
. Using the self-adjoint and idempotent properties of P, for any \mathbf x and \mathbf y in W we have P\mathbf \in U, \mathbf - P\mathbf \in V, and \langle P \mathbf x, \mathbf y - P \mathbf y \rangle = \langle P^2 \mathbf x, \mathbf y - P \mathbf y \rangle = \langle P \mathbf x, P \left(I-P\right) \mathbf y \rangle = \langle P \mathbf x, \left(P-P^2\right) \mathbf y \rangle = 0 where \langle \cdot, \cdot \rangle is the inner product associated with W. Therefore, P and I - P are orthogonal projections. The other direction, namely that if P is orthogonal then it is self-adjoint, follows from \langle \mathbf x, P \mathbf y \rangle = \langle P \mathbf x, \mathbf y \rangle = \langle \mathbf x, P^* \mathbf y \rangle for every x and y in W; thus P=P^*.


Properties and special cases

An orthogonal projection is a bounded operator. This is because for every \mathbf v in the vector space we have, by the
Cauchy–Schwarz inequality The Cauchy–Schwarz inequality (also called Cauchy–Bunyakovsky–Schwarz inequality) is considered one of the most important and widely used inequalities in mathematics. The inequality for sums was published by . The corresponding inequality f ...
: \left \, P \mathbf v\right\, ^2 = \langle P \mathbf v, P \mathbf v \rangle = \langle P \mathbf v, \mathbf v \rangle \leq \left\, P \mathbf v\right\, \cdot \left\, \mathbf v\right\, Thus \left\, P \mathbf v\right\, \leq \left\, \mathbf v\right\, . For finite-dimensional complex or real vector spaces, the
standard inner product In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar (mathematics), scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidea ...
can be substituted for \langle \cdot, \cdot \rangle.


=Formulas

= A simple case occurs when the orthogonal projection is onto a line. If \mathbf u is a
unit vector In mathematics, a unit vector in a normed vector space is a vector (often a spatial vector) of length 1. A unit vector is often denoted by a lowercase letter with a circumflex, or "hat", as in \hat (pronounced "v-hat"). The term ''direction v ...
on the line, then the projection is given by the
outer product In linear algebra, the outer product of two coordinate vectors is a matrix. If the two vectors have dimensions ''n'' and ''m'', then their outer product is an ''n'' × ''m'' matrix. More generally, given two tensors (multidimensional arrays of nu ...
P_\mathbf = \mathbf u \mathbf u^\mathsf. (If \mathbf u is complex-valued, the transpose in the above equation is replaced by a Hermitian transpose). This operator leaves u invariant, and it annihilates all vectors orthogonal to \mathbf u, proving that it is indeed the orthogonal projection onto the line containing u. A simple way to see this is to consider an arbitrary vector \mathbf x as the sum of a component on the line (i.e. the projected vector we seek) and another perpendicular to it, \mathbf x = \mathbf x_\parallel + \mathbf x_\perp. Applying projection, we get P_ \mathbf x = \mathbf u \mathbf u^\mathsf \mathbf x_\parallel + \mathbf u \mathbf u^\mathsf \mathbf x_\perp = \mathbf u \left( \sgn\left(\mathbf u^\mathsf \mathbf x_\parallel\right) \left \, \mathbf x_\parallel \right \, \right) + \mathbf u \cdot \mathbf 0 = \mathbf x_\parallel by the properties of the
dot product In mathematics, the dot product or scalar productThe term ''scalar product'' means literally "product with a scalar as a result". It is also used sometimes for other symmetric bilinear forms, for example in a pseudo-Euclidean space. is an alg ...
of parallel and perpendicular vectors. This formula can be generalized to orthogonal projections on a subspace of arbitrary
dimension In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coord ...
. Let \mathbf u_1, \ldots, \mathbf u_k be an
orthonormal basis In mathematics, particularly linear algebra, an orthonormal basis for an inner product space ''V'' with finite dimension is a basis for V whose vectors are orthonormal, that is, they are all unit vectors and orthogonal to each other. For examp ...
of the subspace U, with the assumption that the integer k \geq 1, and let A denote the n \times k matrix whose columns are \mathbf u_1, \ldots, \mathbf u_k, i.e., A = \begin \mathbf u_1 & \cdots & \mathbf u_k \end. Then the projection is given by: P_A = A A^\mathsf which can be rewritten as P_A = \sum_i \langle \mathbf u_i, \cdot \rangle \mathbf u_i. The matrix A^\mathsf is the
partial isometry In functional analysis a partial isometry is a linear map between Hilbert spaces such that it is an isometry on the orthogonal complement of its kernel. The orthogonal complement of its kernel is called the initial subspace and its range is call ...
that vanishes on the orthogonal complement of U and A is the isometry that embeds U into the underlying vector space. The range of P_A is therefore the ''final space'' of A. It is also clear that A A^ is the identity operator on U. The orthonormality condition can also be dropped. If \mathbf u_1, \ldots, \mathbf u_k is a (not necessarily orthonormal) basis with k \geq 1, and A is the matrix with these vectors as columns, then the projection is: P_A = A \left(A^\mathsf A\right)^ A^\mathsf. The matrix A still embeds U into the underlying vector space but is no longer an isometry in general. The matrix \left(A^\mathsfA\right)^ is a "normalizing factor" that recovers the norm. For example, the
rank Rank is the relative position, value, worth, complexity, power, importance, authority, level, etc. of a person or object within a ranking, such as: Level or position in a hierarchical organization * Academic rank * Diplomatic rank * Hierarchy * ...
-1 operator \mathbf u \mathbf u^\mathsf is not a projection if \left\, \mathbf u \right\, \neq 1. After dividing by \mathbf u^\mathsf \mathbf u = \left\, \mathbf u \right\, ^2, we obtain the projection \mathbf u \left(\mathbf u^\mathsf \mathbf u \right)^ \mathbf u^\mathsf onto the subspace spanned by u. In the general case, we can have an arbitrary
positive definite In mathematics, positive definiteness is a property of any object to which a bilinear form or a sesquilinear form may be naturally associated, which is positive-definite. See, in particular: * Positive-definite bilinear form * Positive-definite fu ...
matrix D defining an inner product \langle x, y \rangle_D = y^\dagger Dx, and the projection P_A is given by P_A x = \operatorname_ \left\, x - y\right\, ^2_D. Then P_A = A \left(A^\mathsf D A\right)^ A^\mathsf D. When the range space of the projection is generated by a frame (i.e. the number of generators is greater than its dimension), the formula for the projection takes the form: P_A = A A^+. Here A^+ stands for the Moore–Penrose pseudoinverse. This is just one of many ways to construct the projection operator. If \begin A & B \end is a non-singular matrix and A^\mathsfB = 0 (i.e., B is the
null space In mathematics, the kernel of a linear map, also known as the null space or nullspace, is the linear subspace of the domain of the map which is mapped to the zero vector. That is, given a linear map between two vector spaces and , the kern ...
matrix of A), the following holds: \begin I &= \begin A & B \end \begin A & B \end^\begin A^\mathsf \\ B^\mathsf \end^ \begin A^\mathsf \\ B^\mathsf \end \\ &= \begin A & B \end \left( \begin A^\mathsf \\ B^\mathsf \end \begin A & B \end \right )^ \begin A^\mathsf \\B^\mathsf \end \\ &= \begin A & B \end \beginA^\mathsfA&O\\O&B^\mathsfB\end^ \begin A^\mathsf \\ B^\mathsf \end\\ pt &= A \left(A^\mathsfA\right)^ A^\mathsf + B \left(B^\mathsfB\right)^ B^\mathsf \end If the orthogonal condition is enhanced to A^\mathsfW B = A^\mathsfW^\mathsfB = 0 with W non-singular, the following holds: I = \beginA & B\end \begin\left(A^\mathsf W A\right)^ A^\mathsf \\ \left(B^\mathsf W B\right)^ B^\mathsf \end W. All these formulas also hold for complex inner product spaces, provided that the
conjugate transpose In mathematics, the conjugate transpose, also known as the Hermitian transpose, of an m \times n complex matrix \boldsymbol is an n \times m matrix obtained by transposing \boldsymbol and applying complex conjugate on each entry (the complex c ...
is used instead of the transpose. Further details on sums of projectors can be found in Banerjee and Roy (2014). Also see Banerjee (2004) for application of sums of projectors in basic
spherical trigonometry Spherical trigonometry is the branch of spherical geometry that deals with the metrical relationships between the sides and angles of spherical triangles, traditionally expressed using trigonometric functions. On the sphere, geodesics are grea ...
.


Oblique projections

The term ''oblique projections'' is sometimes used to refer to non-orthogonal projections. These projections are also used to represent spatial figures in two-dimensional drawings (see oblique projection), though not as frequently as orthogonal projections. Whereas calculating the fitted value of an
ordinary least squares In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the ...
regression requires an orthogonal projection, calculating the fitted value of an instrumental variables regression requires an oblique projection. Projections are defined by their null space and the basis vectors used to characterize their range (which is the complement of the null space). When these basis vectors are orthogonal to the null space, then the projection is an orthogonal projection. When these basis vectors are not orthogonal to the null space, the projection is an oblique projection, or just a general projection.


A matrix representation formula for a nonzero projection operator

Let P be a linear operator P : V \to V such that P^2 = P and assume that P : V \to V is not the zero operator. Let the vectors \mathbf u_1, \ldots, \mathbf u_k form a basis for the range of the projection, and assemble these vectors in the n \times k matrix A. Therefore the integer k \geq 1, otherwise k = 0 and P is the zero operator. The range and the null space are complementary spaces, so the null space has dimension n - k. It follows that the orthogonal complement of the null space has dimension k. Let \mathbf v_1, \ldots, \mathbf v_k form a basis for the orthogonal complement of the null space of the projection, and assemble these vectors in the matrix B. Then the projection P (with the condition k \geq 1) is given by P = A \left(B^\mathsf A\right)^ B^\mathsf. This expression generalizes the formula for orthogonal projections given above. A standard proof of this expression is the following. For any vector \mathbf x in the vector space V, we can decompose \mathbf = \mathbf_1 + \mathbf_2, where vector \mathbf_1 = P(\mathbf) is in the image of P, and vector \mathbf_2 = \mathbf - P(\mathbf). So P(\mathbf_2) = P(\mathbf) - P^2(\mathbf)= \mathbf, and then \mathbf_2 is in the null space of P. In other words, the vector \mathbf_1 is in the column space of A, so \mathbf_1 = A \mathbf for some k dimension vector \mathbf and the vector \mathbf_2 satisfies B^\mathsf \mathbf_2=\mathbf by the construction of B. Put these conditions together, and we find a vector \mathbf so that B^\mathsf (\mathbf-A\mathbf)=\mathbf. Since matrices A and B are of full rank k by their construction, the k\times k-matrix B^\mathsf A is invertible. So the equation B^\mathsf (\mathbf-A\mathbf)=\mathbf gives the vector \mathbf= (B^A)^ B^ \mathbf. In this way, P\mathbf = \mathbf_1 = A\mathbf= A(B^A)^ B^ \mathbf for any vector \mathbf \in V and hence P = A(B^A)^ B^. In the case that P is an orthogonal projection, we can take A = B, and it follows that P=A \left(A^\mathsf A\right)^ A^\mathsf. By using this formula, one can easily check that P=P^\mathsf. In general, if the vector space is over complex number field, one then uses the Hermitian transpose A^* and has the formula P=A \left(A^* A\right)^ A^*. Recall that one can define the
Moore–Penrose inverse In mathematics, and in particular linear algebra, the Moore–Penrose inverse of a matrix is the most widely known generalization of the inverse matrix. It was independently described by E. H. Moore in 1920, Arne Bjerhammar in 1951, and Ro ...
of the matrix A by A^= (A^*A)^A^* since A has full column rank, so P=A A^.


Singular Values

Note that I-P is also an oblique projection. The singular values of P and I-P can be computed by an
orthonormal basis In mathematics, particularly linear algebra, an orthonormal basis for an inner product space ''V'' with finite dimension is a basis for V whose vectors are orthonormal, that is, they are all unit vectors and orthogonal to each other. For examp ...
of A. Let Q_A be an orthonormal basis of A and let Q_A^ be the orthogonal complement of Q_A. Denote the singular values of the matrix Q_A^T A (B^T A)^ B^T Q_A^ by the positive values \gamma_1 \ge \gamma_2 \ge \ldots \ge \gamma_k . With this, the singular values for P are: \sigma_i = \begin \sqrt & 1 \le i \le k \\ 0 & \text \end and the singular values for I-P are \sigma_i = \begin \sqrt & 1 \le i \le k \\ 1 & k+1 \le i \le n-k \\ 0 & \text \end This implies that the largest singular values of P and I-P are equal, and thus that the matrix norm of the oblique projections are the same. However, the
condition number In numerical analysis, the condition number of a function measures how much the output value of the function can change for a small change in the input argument. This is used to measure how sensitive a function is to changes or errors in the inpu ...
satisfies the relation \kappa(I-P) = \frac \ge \frac = \kappa(P), and is therefore not necessarily equal.


Finding projection with an inner product

Let V be a vector space (in this case a plane) spanned by orthogonal vectors \mathbf u_1, \mathbf u_2, \dots, \mathbf u_p. Let y be a vector. One can define a projection of \mathbf y onto V as \operatorname_V \mathbf y = \frac \mathbf u^i where repeated indices are summed over ( Einstein sum notation). The vector \mathbf y can be written as an orthogonal sum such that \mathbf y = \operatorname_V \mathbf y + \mathbf z. \operatorname_V \mathbf y is sometimes denoted as \hat. There is a theorem in linear algebra that states that this \mathbf z is the smallest distance (the '' orthogonal distance'') from \mathbf y to V and is commonly used in areas such as
machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine ...
.


Canonical forms

Any projection P=P^2 on a vector space of dimension d over a field is a
diagonalizable matrix In linear algebra, a square matrix A is called diagonalizable or non-defective if it is similar to a diagonal matrix, i.e., if there exists an invertible matrix P and a diagonal matrix D such that or equivalently (Such D are not unique. ...
, since its minimal polynomial divides x^2-x, which splits into distinct linear factors. Thus there exists a basis in which P has the form :P = I_r\oplus 0_ where r is the
rank Rank is the relative position, value, worth, complexity, power, importance, authority, level, etc. of a person or object within a ranking, such as: Level or position in a hierarchical organization * Academic rank * Diplomatic rank * Hierarchy * ...
of P. Here I_r is the
identity matrix In linear algebra, the identity matrix of size n is the n\times n square matrix with ones on the main diagonal and zeros elsewhere. Terminology and notation The identity matrix is often denoted by I_n, or simply by I if the size is immaterial or ...
of size r, and 0_ is the zero matrix of size d-r. If the vector space is complex and equipped with an
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
, then there is an ''orthonormal'' basis in which the matrix of ''P'' is :P = \begin1&\sigma_1 \\ 0&0\end \oplus \cdots \oplus \begin1&\sigma_k \\ 0&0\end \oplus I_m \oplus 0_s. where \sigma_1 \geq \sigma_2\geq \dots \geq \sigma_k > 0. The
integer An integer is the number zero (), a positive natural number (, , , etc.) or a negative integer with a minus sign ( −1, −2, −3, etc.). The negative numbers are the additive inverses of the corresponding positive numbers. In the languag ...
s k,s,m and the real numbers \sigma_i are uniquely determined. Note that 2k+s+m=d. The factor I_m \oplus 0_s corresponds to the maximal invariant subspace on which P acts as an ''orthogonal'' projection (so that ''P'' itself is orthogonal if and only if k=0) and the \sigma_i-blocks correspond to the ''oblique'' components.


Projections on normed vector spaces

When the underlying vector space X is a (not necessarily finite-dimensional) normed vector space, analytic questions, irrelevant in the finite-dimensional case, need to be considered. Assume now X is a
Banach space In mathematics, more specifically in functional analysis, a Banach space (pronounced ) is a complete normed vector space. Thus, a Banach space is a vector space with a metric that allows the computation of vector length and distance between vector ...
. Many of the algebraic results discussed above survive the passage to this context. A given direct sum decomposition of X into complementary subspaces still specifies a projection, and vice versa. If X is the direct sum X = U \oplus V, then the operator defined by P(u+v) = u is still a projection with range U and kernel V. It is also clear that P^2 = P. Conversely, if P is projection on X, i.e. P^2 = P, then it is easily verified that (1-P)^2 = (1-P). In other words, 1 - P is also a projection. The relation P^2 = P implies 1 = P + (1-P) and X is the direct sum \operatorname(P) \oplus \operatorname(1 - P). However, in contrast to the finite-dimensional case, projections need not be
continuous Continuity or continuous may refer to: Mathematics * Continuity (mathematics), the opposing concept to discreteness; common examples include ** Continuous probability distribution or random variable in probability and statistics ** Continuous g ...
in general. If a subspace U of X is not closed in the norm topology, then the projection onto U is not continuous. In other words, the range of a continuous projection P must be a closed subspace. Furthermore, the kernel of a continuous projection (in fact, a continuous linear operator in general) is closed. Thus a ''continuous'' projection P gives a decomposition of X into two complementary ''closed'' subspaces: X = \operatorname(P) \oplus \ker(P) = \ker(1-P) \oplus \ker(P). The converse holds also, with an additional assumption. Suppose U is a closed subspace of X. If there exists a closed subspace V such that , then the projection P with range U and kernel V is continuous. This follows from the closed graph theorem. Suppose and . One needs to show that Px=y. Since U is closed and , ''y'' lies in U, i.e. . Also, . Because V is closed and , we have x-y \in V, i.e. P(x-y)=Px-Py=Px-y=0, which proves the claim. The above argument makes use of the assumption that both U and V are closed. In general, given a closed subspace U, there need not exist a complementary closed subspace V, although for
Hilbert space In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natural ...
s this can always be done by taking the orthogonal complement. For Banach spaces, a one-dimensional subspace always has a closed complementary subspace. This is an immediate consequence of Hahn–Banach theorem. Let U be the linear span of u. By Hahn–Banach, there exists a bounded
linear functional In mathematics, a linear form (also known as a linear functional, a one-form, or a covector) is a linear map from a vector space to its field of scalars (often, the real numbers or the complex numbers). If is a vector space over a field , the ...
\varphi such that . The operator P(x)=\varphi(x)u satisfies P^2=P, i.e. it is a projection. Boundedness of \varphi implies continuity of P and therefore \ker(P) = \operatorname(I-P) is a closed complementary subspace of U.


Applications and further considerations

Projections (orthogonal and otherwise) play a major role in
algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
s for certain linear algebra problems: *
QR decomposition In linear algebra, a QR decomposition, also known as a QR factorization or QU factorization, is a decomposition of a matrix ''A'' into a product ''A'' = ''QR'' of an orthogonal matrix ''Q'' and an upper triangular matrix ''R''. QR decomp ...
(see Householder transformation and Gram–Schmidt decomposition); *
Singular value decomposition In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any \ m \times n\ matrix. It is re ...
* Reduction to Hessenberg form (the first step in many eigenvalue algorithms) *
Linear regression In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is cal ...
* Projective elements of matrix algebras are used in the construction of certain K-groups in
Operator K-theory In mathematics, operator K-theory is a noncommutative analogue of topological K-theory for Banach algebras with most applications used for C*-algebras. Overview Operator K-theory resembles topological K-theory more than algebraic K-theory. In pa ...
As stated above, projections are a special case of idempotents. Analytically, orthogonal projections are non-commutative generalizations of characteristic functions. Idempotents are used in classifying, for instance, semisimple algebras, while
measure theory In mathematics, the concept of a measure is a generalization and formalization of geometrical measures (length, area, volume) and other common notions, such as mass and probability of events. These seemingly distinct concepts have many simila ...
begins with considering characteristic functions of measurable sets. Therefore, as one can imagine, projections are very often encountered in the context of operator algebras. In particular, a von Neumann algebra is generated by its complete lattice of projections.


Generalizations

More generally, given a map between normed vector spaces T\colon V \to W, one can analogously ask for this map to be an isometry on the orthogonal complement of the kernel: that (\ker T)^\perp \to W be an isometry (compare
Partial isometry In functional analysis a partial isometry is a linear map between Hilbert spaces such that it is an isometry on the orthogonal complement of its kernel. The orthogonal complement of its kernel is called the initial subspace and its range is call ...
); in particular it must be onto. The case of an orthogonal projection is when ''W'' is a subspace of ''V.'' In Riemannian geometry, this is used in the definition of a Riemannian submersion.


See also

* Centering matrix, which is an example of a projection matrix. *
Dykstra's projection algorithm Dykstra's algorithm is a method that computes a point in the intersection of convex sets, and is a variant of the alternating projection method (also called the projections onto convex sets method). In its simplest form, the method finds a poin ...
to compute the projection onto an intersection of sets *
Invariant subspace In mathematics, an invariant subspace of a linear mapping ''T'' : ''V'' → ''V '' i.e. from some vector space ''V'' to itself, is a subspace ''W'' of ''V'' that is preserved by ''T''; that is, ''T''(''W'') ⊆ ''W''. General desc ...
* Least-squares spectral analysis *
Orthogonalization In linear algebra, orthogonalization is the process of finding a set of orthogonal vectors that span a particular subspace. Formally, starting with a linearly independent set of vectors in an inner product space (most commonly the Euclidean s ...
* Properties of trace


Notes


References

* * *


External links

* , from MIT OpenCourseWare * , by
Pavel Grinfeld Pavel Grinfeld (also known as Greenfield) is an Americans, American mathematician and associate professor of Applied Mathematics at Drexel University working on problems in The Calculus of Moving Surfaces, moving surfaces in applied mathematics ...
.
Planar Geometric Projections Tutorial
– a simple-to-follow tutorial explaining the different types of planar geometric projections. {{linear algebra Functional analysis Linear algebra Linear operators