Orthogonal projection

In linear algebra and functional analysis, a projection is a linear transformation $P$ from a vector space to itself (an endomorphism) such that $P\circ P=P$. That is, whenever $P$ is applied twice to any vector, it gives the same result as if it were applied once (i.e. $P$ is idempotent). It leaves its image unchanged. This definition of "projection" formalizes and generalizes the idea of graphical projection. One can also consider the effect of a projection on a geometrical object by examining the effect of the projection on points in the object.

Definitions

A projection on a vector space $V$ is a linear operator $P : V \to V$ such that $P^2 = P$.

When $V$ has an inner product and is complete (i.e. when $V$ is a Hilbert space) the concept of orthogonality can be used. A projection $P$ on a Hilbert space $V$ is called an orthogonal projection if it satisfies $\langle P \mathbf x, \mathbf y \rangle = \langle \mathbf x, P \mathbf y \rangle$ for all $\mathbf x, \mathbf y \in V$. A projection on a Hilbert space that is not orthogonal is called an oblique projection.

Projection matrix

* In the finite-dimensional case, a square matrix $P$ is called a projection matrix if it is equal to its square, i.e. if $P^2 = P$.
* A square matrix $P$ is called an orthogonal projection matrix if $P^2 = P = P^$ for a real matrix, and respectively $P^2 = P = P^$ for a complex matrix, where $P^$ denotes the transpose of $P$ and $P^$ denotes the adjoint or Hermitian transpose of $P$.
* A projection matrix that is not an orthogonal projection matrix is called an oblique projection matrix.
The eigenvalues of a projection matrix must be 0 or 1.

Examples

Orthogonal projection

For example, the function which maps the point $(x,y,z)$ in three-dimensional space $\mathbb^3$ to the point $(x,y,0)$ is an orthogonal projection onto the ''xy''-plane. This function is represented by the matrix
$P = \begin 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end.$

The action of this matrix on an arbitrary vector is
$P \begin x \\ y \\ z \end = \begin x \\ y \\ 0 \end.$

To see that $P$ is indeed a projection, i.e., $P = P^2$, we compute
$P^2 \begin x \\ y \\ z \end = P \begin x \\ y \\ 0 \end = \begin x \\ y \\ 0 \end = P\begin x \\ y \\ z \end.$

Observing that $P^ = P$ shows that the projection is an orthogonal projection.

Oblique projection

A simple example of a non-orthogonal (oblique) projection is
$P = \begin 0 & 0 \\ \alpha & 1 \end.$

Via matrix multiplication, one sees that
$P^2 = \begin 0 & 0 \\ \alpha & 1 \end \begin 0 & 0 \\ \alpha & 1 \end = \begin 0 & 0 \\ \alpha & 1 \end = P.$
showing that $P$ is indeed a projection.

The projection $P$ is orthogonal if and only if $\alpha = 0$ because only then $P^ = P.$

Properties and classification

Idempotence

By definition, a projection $P$ is idempotent (i.e. $P^2 = P$).

Open map

Every projection is an open map, meaning that it maps each open set in the domain to an open set in the subspace topology of the image. That is, for any vector $\mathbf$ and any ball $B_\mathbf$ (with positive radius) centered on $\mathbf$, there exists a ball $B_$ (with positive radius) centered on $P\mathbf$ that is wholly contained in the image $P(B_\mathbf)$.

Complementarity of image and kernel

Let $W$ be a finite-dimensional vector space and $P$ be a projection on $W$. Suppose the subspaces $U$ and $V$ are the image and kernel of $P$ respectively. Then $P$ has the following properties:

# $P$ is the identity operator $I$ on $U$: $\forall \mathbf x \in U: P \mathbf x = \mathbf x.$
# We have a direct sum $W = U \oplus V$. Every vector $\mathbf x \in W$ may be decomposed uniquely as $\mathbf x = \mathbf u + \mathbf v$ with $\mathbf u = P \mathbf x$ and $\mathbf v = \mathbf x - P \mathbf x = \left(I-P\right) \mathbf x$, and where $\mathbf u \in U, \mathbf v \in V.$

The image and kernel of a projection are ''complementary'', as are $P$ and $Q = I - P$. The operator $Q$ is also a projection as the image and kernel of $P$ become the kernel and image of $Q$ and vice versa. We say $P$ is a projection along $V$ onto $U$ (kernel/image) and $Q$ is a projection along $U$ onto $V$.

Spectrum

In infinite-dimensional vector spaces, the spectrum of a projection is contained in $\$ as
$(\lambda I - P)^ = \frac 1 \lambda I + \frac 1 P.$
Only 0 or 1 can be an eigenvalue of a projection. This implies that an orthogonal projection $P$ is always a positive semi-definite matrix. In general, the corresponding eigenspaces are (respectively) the kernel and range of the projection. Decomposition of a vector space into direct sums is not unique. Therefore, given a subspace $V$, there may be many projections whose range (or kernel) is $V$.

If a projection is nontrivial it has minimal polynomial $x^2 - x = x (x-1)$, which factors into distinct linear factors, and thus $P$ is diagonalizable.

Product of projections

The product of projections is not in general a projection, even if they are orthogonal. If two projections commute then their product is a projection, but the converse is false: the product of two non-commuting projections may be a projection.

If two orthogonal projections commute then their product is an orthogonal projection. If the product of two orthogonal projections is an orthogonal projection, then the two orthogonal projections commute (more generally: two self-adjoint endomorphisms commute if and only if their product is self-adjoint).

Orthogonal projections

When the vector space $W$ has an inner product and is complete (is a Hilbert space) the concept of orthogonality can be used. An orthogonal projection is a projection for which the range $U$ and the null space $V$ are orthogonal subspaces. Thus, for every $\mathbf x$ and $\mathbf y$ in $W$, $\langle P \mathbf x, (\mathbf y - P \mathbf y) \rangle = \langle (\mathbf x - P \mathbf x) , P \mathbf y \rangle = 0$. Equivalently:
$\langle \mathbf x, P \mathbf y \rangle = \langle P \mathbf x, P \mathbf y \rangle = \langle P \mathbf x, \mathbf y \rangle.$

A projection is orthogonal if and only if it is self-adjoint. Using the self-adjoint and idempotent properties of $P$, for any $\mathbf x$ and $\mathbf y$ in $W$ we have $P\mathbf \in U$, $\mathbf - P\mathbf \in V$, and
$\langle P \mathbf x, \mathbf y - P \mathbf y \rangle = \langle P^2 \mathbf x, \mathbf y - P \mathbf y \rangle = \langle P \mathbf x, P \left(I-P\right) \mathbf y \rangle = \langle P \mathbf x, \left(P-P^2\right) \mathbf y \rangle = 0$
where $\langle \cdot, \cdot \rangle$ is the inner product associated with $W$. Therefore, $P$ and $I - P$ are orthogonal projections. The other direction, namely that if $P$ is orthogonal then it is self-adjoint, follows from
$\langle \mathbf x, P \mathbf y \rangle = \langle P \mathbf x, \mathbf y \rangle = \langle \mathbf x, P^* \mathbf y \rangle$
for every $x$ and $y$ in $W$; thus $P=P^*$.

Properties and special cases

An orthogonal projection is a bounded operator. This is because for every $\mathbf v$ in the vector space we have, by the Cauchy–Schwarz inequality:
$\left \, P \mathbf v\right\, ^2 = \langle P \mathbf v, P \mathbf v \rangle = \langle P \mathbf v, \mathbf v \rangle \leq \left\, P \mathbf v\right\, \cdot \left\, \mathbf v\right\,$
Thus $\left\, P \mathbf v\right\, \leq \left\, \mathbf v\right\,$.

For finite-dimensional complex or real vector spaces, the standard inner product can be substituted for $\langle \cdot, \cdot \rangle$.

=Formulas

=
A simple case occurs when the orthogonal projection is onto a line. If $\mathbf u$ is a unit vector on the line, then the projection is given by the outer product
$P_\mathbf = \mathbf u \mathbf u^\mathsf.$
(If $\mathbf u$ is complex-valued, the transpose in the above equation is replaced by a Hermitian transpose). This operator leaves u invariant, and it annihilates all vectors orthogonal to $\mathbf u$, proving that it is indeed the orthogonal projection onto the line containing u. A simple way to see this is to consider an arbitrary vector $\mathbf x$ as the sum of a component on the line (i.e. the projected vector we seek) and another perpendicular to it, $\mathbf x = \mathbf x_\parallel + \mathbf x_\perp$. Applying projection, we get
$P_ \mathbf x = \mathbf u \mathbf u^\mathsf \mathbf x_\parallel + \mathbf u \mathbf u^\mathsf \mathbf x_\perp = \mathbf u \left( \sgn\left(\mathbf u^\mathsf \mathbf x_\parallel\right) \left \, \mathbf x_\parallel \right \, \right) + \mathbf u \cdot \mathbf 0 = \mathbf x_\parallel$
by the properties of the dot product of parallel and perpendicular vectors.

This formul