The Info List - Orthogonal Projection

--- Advertisement ---

In linear algebra and functional analysis , a PROJECTION is a linear transformation P from a vector space to itself such that P 2 = P. That is, whenever P is applied twice to any value, it gives the same result as if it were applied once (idempotent ). It leaves its image unchanged. Though abstract, this definition of "projection" formalizes and generalizes the idea of graphical projection . One can also consider the effect of a projection on a geometrical object by examining the effect of the projection on points in the object.


* 1 Simple example

* 1.1 Orthogonal projection * 1.2 Oblique projection

* 2 Properties and classification

* 2.1 Orthogonal projections

* 2.1.1 Properties and special cases

* Formulas

* 2.2 Oblique projections

* 3 Canonical forms * 4 Projections on normed vector spaces * 5 Applications and further considerations * 6 Generalizations * 7 See also * 8 Notes * 9 References * 10 External links



For example, the function which maps the point (x, y, z) in three-dimensional space R3 to the point (x, y, 0) is an orthogonal projection onto the x–y plane. This function is represented by the matrix P = . {displaystyle P={begin{bmatrix}1&0&0\0&1&0\0&0 width:17.559ex; height:9.176ex;" alt="P={begin{bmatrix}1&0&0\0&1&0\0&0"> P ( x y z ) = ( x y 0 ) . {displaystyle P{begin{pmatrix}x\y\zend{pmatrix}}={begin{pmatrix}x\y\0end{pmatrix}}.}

To see that P is indeed a projection, i.e., P = P2, we compute P 2 ( x y z ) = P ( x y 0 ) = ( x y 0 ) = P ( x y z ) . {displaystyle P^{2}{begin{pmatrix}x\y\zend{pmatrix}}=P{begin{pmatrix}x\y\0end{pmatrix}}={begin{pmatrix}x\y\0end{pmatrix}}=P{begin{pmatrix}x\y\zend{pmatrix}}.}


A simple example of a non-orthogonal (oblique) projection (for definition see below) is P = . {displaystyle P={begin{bmatrix}0&0\alpha width:13.743ex; height:6.176ex;" alt="P={begin{bmatrix}0&0\alpha "> P 2 = = = P . {displaystyle P^{2}={begin{bmatrix}0&0\alpha &1end{bmatrix}}{begin{bmatrix}0&0\alpha &1end{bmatrix}}={begin{bmatrix}0&0\alpha width:39.296ex; height:6.176ex;" alt="P^{2}={begin{bmatrix}0&0\alpha &1end{bmatrix}}{begin{bmatrix}0&0\alpha &1end{bmatrix}}={begin{bmatrix}0&0\alpha "> The transformation T is the projection along k onto m. The range of T is m and the null space is k.

Let W be a finite dimensional vector space and P be a projection on W. Suppose the subspaces U and V are the range and kernel of P respectively. Then P has the following properties:

* By definition, P {displaystyle P} is idempotent (i.e. P 2 = P {displaystyle P^{2}=P} ). * P {displaystyle P} is the identity operator I {displaystyle I} on U {displaystyle U} x U : P x = x {displaystyle forall xin U:Px=x} . * We have a direct sum W = U V {displaystyle W=Uoplus V} . Every vector x W {displaystyle xin W} may be decomposed uniquely as x = u + v {displaystyle x=u+v} with u = P x {displaystyle u=Px} and v = x P x = ( I P ) x {displaystyle v=x-Px=(I-P)x} , and where u U , v V {displaystyle uin U,vin V} .

The range and kernel of a projection are complementary, as are P {displaystyle P} and Q = I P {displaystyle Q=I-P} . The operator Q {displaystyle Q} is also a projection and the range and kernel of P {displaystyle P} become the kernel and range of Q {displaystyle Q} and vice versa. We say P {displaystyle P} is a projection along V onto U (kernel/range) and Q {displaystyle Q} is a projection along U onto V.

In infinite dimensional vector spaces, the spectrum of a projection is contained in {0, 1} as ( I P ) 1 = 1 I + 1 ( 1 ) P {displaystyle (lambda I-P)^{-1}={frac {1}{lambda }}I+{frac {1}{lambda (lambda -1)}}P} .

Only 0 or 1 can be an eigenvalue of a projection. The corresponding eigenspaces are (respectively) the kernel and range of the projection. Decomposition of a vector space into direct sums is not unique in general. Therefore, given a subspace V {displaystyle V} , there may be many projections whose range (or kernel) is V {displaystyle V} .

If a projection is nontrivial it has minimal polynomial x 2 x = x ( x 1 ) {displaystyle x^{2}-x=x(x-1)} , which factors into distinct roots, and thus P {displaystyle P} is diagonalizable .

The product of projections is not, in general, a projection, even if they are orthogonal. If projections commute, then their product is a projection.


When the vector space W has an inner product and is complete (is a Hilbert space
Hilbert space
) the concept of orthogonality can be used. An ORTHOGONAL PROJECTION is a projection for which the range U and the null space V are orthogonal subspaces . Thus, for every x and y in W, P x , ( y P y ) = ( x P x ) , P y = 0 {displaystyle langle Px,(y-Py)rangle =langle (x-Px),Pyrangle =0} . Equivalently: x , P y = P x , P y = P x , y {displaystyle langle x,Pyrangle =langle Px,Pyrangle =langle Px,yrangle } .

A projection is orthogonal if and only if it is self-adjoint . Using the self-adjoint and idempotent properties of P, for any x and y in W we have Px ∈ U, y − Py ∈ V, and P x , y P y = P 2 x , y P y = P x , P ( I P ) y = P x , ( P P 2 ) y = 0 {displaystyle langle Px,y-Pyrangle =langle P^{2}x,y-Pyrangle =langle Px,P(I-P)yrangle =langle Px,(P-P^{2})yrangle =0,}

where , {displaystyle langle cdot ,cdot rangle } is the inner product associated with W. Therefore, Px and y − Py are orthogonal. The other direction, namely that if P is orthogonal then it is self-adjoint, follows from x , P y = P x , y = x , P y {displaystyle langle x,Pyrangle =langle Px,yrangle =langle x,P^{*}yrangle }

for every x and y in W; thus P = P*.


Let H be a complete metric space with an inner product , and let U be a closed linear subspace of H (and hence complete as well).

For every x the following set of non-negative norms { x u u U } {displaystyle {x-uuin U}} has an infimum , and due to the completeness of U it is a minimum . We define Px as the point in U were this minimum is obtained.

Obviously Px is in U. It remains to show that Px satisfies = 0 and that it is linear.

Let us define a = x P x {displaystyle a=x-Px} . For every non-zero v in U, the following holds: a a , v v 2 v 2 = a 2 a , v 2 v 2 {displaystyle a-{frac {langle a,vrangle }{v^{2}}}v^{2}=a^{2}-{frac {{langle a,vrangle }^{2}}{v^{2}}}}

By defining w = P x + a , v v 2 v {displaystyle w=Px+{frac {langle a,vrangle }{v^{2}}}v} we see that x w P x {displaystyle x-w {displaystyle langle a,vrangle } vanishes. Since Px was chosen as the minimum of the abovementioned set, it follows that a , v {displaystyle langle a,vrangle } indeed vanishes. In particular, (for v = Px): x P x , P x = 0 {displaystyle langle x-Px,Pxrangle =0} .

Linearity follows from the vanishing of x P x , v {displaystyle langle x-Px,vrangle } for every v in U: ( x + y ) P ( x + y ) , v = 0 {displaystyle langle left(x+yright)-Pleft(x+yright),vrangle =0} ( x P x ) + ( y P y ) , v = 0 {displaystyle langle left(x-Pxright)+left(y-Pyright),vrangle =0}

By taking the difference between the equations we have P x + P y P ( x + y ) , v = 0 {displaystyle langle Px+Py-Pleft(x+yright),vrangle =0}

But since we may choose v = Px + Py − P(x + y) (as it is itself in U) it follows that Px + Py = P(x + y). Similarly we have λPx = P(λx) for every scalar λ.

Properties And Special

An orthogonal projection is a bounded operator . This is because for every v in the vector space we have, by Cauchy–Schwarz inequality : P v 2 = P v , P v = P v , v P v v {displaystyle Pv^{2}=langle Pv,Pvrangle =langle Pv,vrangle leqslant Pvcdot v}

Thus P v v {displaystyle Pvleqslant v} .

For finite dimensional complex or real vector spaces, the standard inner product can be substituted for , {displaystyle langle cdot ,cdot rangle } .


A simple case occurs when the orthogonal projection is onto a line. If u is a unit vector on the line, then the projection is given by the outer product P u = u u T . {displaystyle P_{u}=uu^{mathrm {T} }.}

(If u is complex-valued, the transpose in the above equation is replaced by a Hermitian transpose). This operator leaves u invariant, and it annihilates all vectors orthogonal to u, proving that it is indeed the orthogonal projection onto the line containing u. A simple way to see this is to consider an arbitrary vector x {displaystyle x} as the sum of a component on the line (i.e. the projected vector we seek) and another perpendicular to it, x = x + x {displaystyle x=x_{parallel }+x_{perp }} . Applying projection, we get P u x = u u T x + u u T x = u ( s i g n ( u T x ) x ) + u 0 = x {displaystyle P_{u}x=uu^{mathrm {T} }x_{parallel }+uu^{mathrm {T} }x_{perp }=uleft(mathrm {sign} (u^{mathrm {T} }x_{parallel })x_{parallel }right)+ucdot 0=x_{parallel }}

by the properties of the dot product of parallel and perpendicular vectors.

This formula can be generalized to orthogonal projections on a subspace of arbitrary dimension. Let u1, ..., uk be an orthonormal basis of the subspace U, and let A denote the n-by-k matrix whose columns are u1, ..., uk. Then the projection is given by: P A = A A T {displaystyle P_{A}=AA^{mathrm {T} }}

which can be rewritten as P A = i u i , u i . {displaystyle P_{A}=sum _{i}langle u_{i},cdot rangle u_{i}.}

The matrix AT is the partial isometry that vanishes on the orthogonal complement of U and A is the isometry that embeds U into the underlying vector space. The range of PA is therefore the final space of A. It is also clear that A·AT is the identity operator on U.

The orthonormality condition can also be dropped. If u1, ..., uk is a (not necessarily orthonormal) basis, and A is the matrix with these vectors as columns, then the projection is: P A = A ( A T A ) 1 A T . {displaystyle P_{A}=A(A^{mathrm {T} }A)^{-1}A^{mathrm {T} }.}

The matrix A still embeds U into the underlying vector space but is no longer an isometry in general. The matrix (ATA)−1 is a "normalizing factor" that recovers the norm. For example, the rank-1 operator uuT is not a projection if u 1. {displaystyle uneq 1.} After dividing by u T u = u 2 , {displaystyle u^{mathrm {T} }u=u^{2},} we obtain the projection u(uTu)−1uT onto the subspace spanned by u.

When the range space of the projection is generated by a frame (i.e. the number of generators is greater than its dimension), the formula for the projection takes the form: P A = A A + {displaystyle P_{A}=AA^{+}} . Here A + {displaystyle A^{+}} stands for the Moore–Penrose pseudoinverse . This is just one of many ways to construct the projection operator.

If {displaystyle {begin{bmatrix}A"> A T B = 0 {displaystyle A^{mathrm {T} }B=0} (i.e., B is the null space matrix of A), the following holds: I = 1 1 = ( ) 1 = 1 = A ( A T A ) 1 A T + B ( B T B ) 1 B T {displaystyle {begin{aligned}I&={begin{bmatrix}A&Bend{bmatrix}}{begin{bmatrix}A&Bend{bmatrix}}^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}\&={begin{bmatrix}A&Bend{bmatrix}}left({begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}{begin{bmatrix}A&Bend{bmatrix}}right)^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}\&={begin{bmatrix}A&Bend{bmatrix}}{begin{bmatrix}A^{mathrm {T} }A&O\O&B^{mathrm {T} }Bend{bmatrix}}^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}\ width:39.522ex; height:25.176ex;" alt="{displaystyle {begin{aligned}I&={begin{bmatrix}A&Bend{bmatrix}}{begin{bmatrix}A&Bend{bmatrix}}^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}\&={begin{bmatrix}A&Bend{bmatrix}}left({begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}{begin{bmatrix}A&Bend{bmatrix}}right)^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}\&={begin{bmatrix}A&Bend{bmatrix}}{begin{bmatrix}A^{mathrm {T} }A&O\O&B^{mathrm {T} }Bend{bmatrix}}^{-1}{begin{bmatrix}A^{mathrm {T} }\B^{mathrm {T} }end{bmatrix}}\"> I = W . {displaystyle I={begin{bmatrix}A width:33.337ex; height:6.509ex;" alt="{displaystyle I={begin{bmatrix}A"> P = A ( B T A ) 1 B T . {displaystyle P=A(B^{mathrm {T} }A)^{-1}B^{mathrm {T} }.}

This expression generalizes the formula for orthogonal projections given above.


Any projection P = P2 on a vector space of dimension d over a field is a diagonalizable matrix , since its minimal polynomial is x2 − x, which splits into distinct linear factors. Thus there exists a basis in which P has the form P = I r 0 d r {displaystyle P=I_{r}oplus 0_{d-r}}

where r is the rank of P. Here Ir is the identity matrix of size r, and 0d−r is the zero matrix of size d − r. If the vector space is complex and equipped with an inner product , then there is an orthonormal basis in which the matrix of P is P = I m 0 s {displaystyle P={begin{bmatrix}1&sigma _{1}\0&0end{bmatrix}}oplus cdots oplus {begin{bmatrix}1&sigma _{k}\0 width:42.181ex; height:6.176ex;" alt="P={begin{bmatrix}1&sigma _{1}\0&0end{bmatrix}}oplus cdots oplus {begin{bmatrix}1&sigma _{k}\0"> i {displaystyle sigma _{i}} are uniquely determined. Note that 2k + s + m = d. The factor Im ⊕ 0s corresponds to the maximal invariant subspace on which P acts as an orthogonal projection (so that P itself is orthogonal if and only if k = 0) and the σi-blocks correspond to the oblique components.


When the underlying vector space X {displaystyle X} is a (not necessarily finite-dimensional) normed vector space , analytic questions, irrelevant in the finite-dimensional case, need to be considered. Assume now X {displaystyle X} is a Banach space
Banach space

Many of the algebraic notions discussed above survive the passage to this context. A given direct sum decomposition of X {displaystyle X} into complementary subspaces still specifies a projection, and vice versa. If X {displaystyle X} is the direct sum X = U V {displaystyle X=Uoplus V} , then the operator defined by P ( u + v ) = u {displaystyle P(u+v)=u} is still a projection with range U {displaystyle U} and kernel V {displaystyle V} . It is also clear that P 2 = P {displaystyle P^{2}=P} . Conversely, if P {displaystyle P} is projection on X {displaystyle X} , i.e. P 2 = P {displaystyle P^{2}=P} , then it is easily verified that ( 1 P ) 2 = ( 1 P ) {displaystyle (1-P)^{2}=(1-P)} . In other words, 1 P {displaystyle 1-P} is also a projection. The relation P 2 = P {displaystyle P^{2}=P} implies 1 = P + ( 1 P ) {displaystyle 1=P+(1-P)} and X {displaystyle X} is the direct sum r a n ( P ) r a n ( 1 P ) {displaystyle mathrm {ran} (P)oplus mathrm {ran} (1-P)} .

However, in contrast to the finite-dimensional case, projections need not be continuous in general. If a subspace U {displaystyle U} of X {displaystyle X} is not closed in the norm topology, then projection onto U {displaystyle U} is not continuous. In other words, the range of a continuous projection P {displaystyle P} must be a closed subspace. Furthermore, the kernel of a continuous projection (in fact, a continuous linear operator in general) is