In

_{''i''}: in matrix form,
: Minimize $(y\; -\; X\backslash beta)^\backslash textsf(y\; -\; X\backslash beta)$
where $y$ is a vector of

linear algebra
Linear algebra is the branch of mathematics concerning linear equations such as:
:a_1x_1+\cdots +a_nx_n=b,
linear maps such as:
:(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n,
and their representations in vector spaces and through matrix (mat ...

, an idempotent matrix is a matrix
Matrix or MATRIX may refer to:
Science and mathematics
* Matrix (mathematics), a rectangular array of numbers, symbols, or expressions
* Matrix (logic), part of a formula in prenex normal form
* Matrix (biology), the material in between a eukaryoti ...

which, when multiplied by itself, yields itself. That is, the matrix $A$ is idempotent if and only if $A^2\; =\; A$. For this product $A^2$ to be defined
A definition is a statement of the meaning of a term (a word, phrase, or other set of symbols). Definitions can be classified into two large categories, intensional definitions (which try to give the sense of a term) and extensional definitions ...

, $A$ must necessarily be a square matrix
In mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...

. Viewed this way, idempotent matrices are idempotent element
Idempotence (, ) is the property of certain operation (mathematics), operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. The concept of idempotence a ...

s of matrix ring
In abstract algebra
In algebra, which is a broad division of mathematics, abstract algebra (occasionally called modern algebra) is the study of algebraic structures. Algebraic structures include group (mathematics), groups, ring (mathematics), ...

s.
Example

Examples of $2\; \backslash times\; 2$ idempotent matrices are: $$\backslash begin\; 1\; \&\; 0\; \backslash \backslash \; 0\; \&\; 1\; \backslash end\; \backslash qquad\; \backslash begin\; 3\; \&\; -6\; \backslash \backslash \; 1\; \&\; -2\; \backslash end$$ Examples of $3\; \backslash times\; 3$ idempotent matrices are: $$\backslash begin\; 1\; \&\; 0\; \&\; 0\; \backslash \backslash \; 0\; \&\; 1\; \&\; 0\; \backslash \backslash \; 0\; \&\; 0\; \&\; 1\; \backslash end\; \backslash qquad\; \backslash begin\; 2\; \&\; -2\; \&\; -4\; \backslash \backslash \; -1\; \&\; 3\; \&\; 4\; \backslash \backslash \; 1\; \&\; -2\; \&\; -3\; \backslash end$$Real 2 × 2 case

If a matrix $\backslash begina\; \&\; b\; \backslash \backslash \; c\; \&\; d\; \backslash end$ is idempotent, then * $a\; =\; a^2\; +\; bc,$ * $b\; =\; ab\; +\; bd,$ implying $b(1\; -\; a\; -\; d)\; =\; 0$ so $b\; =\; 0$ or $d\; =\; 1\; -\; a,$ * $c\; =\; ca\; +\; cd,$ implying $c(1\; -\; a\; -\; d)\; =\; 0$ so $c\; =\; 0$ or $d\; =\; 1\; -\; a,$ * $d\; =\; bc\; +\; d^2.$ Thus a necessary condition for a 2 × 2 matrix to be idempotent is that either it isdiagonal
In geometry
Geometry (from the grc, γεωμετρία; ' "earth", ' "measurement") is, with , one of the oldest branches of . It is concerned with properties of space that are related with distance, shape, size, and relative position o ...

or its trace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band)
Trace was a Netherlands, Dutch progressive rock trio founded by Rick van der Linden in 1974 after leavin ...

equals 1.
For idempotent diagonal matrices, $a$ and $d$ must be either 1 or 0.
If $b=c$, the matrix $\backslash begina\; \&\; b\; \backslash \backslash \; b\; \&\; 1\; -\; a\; \backslash end$ will be idempotent provided $a^2\; +\; b^2\; =\; a\; ,$ so ''a'' satisfies the quadratic equation
In algebra
Algebra (from ar, الجبر, lit=reunion of broken parts, bonesetting, translit=al-jabr) is one of the areas of mathematics, broad areas of mathematics, together with number theory, geometry and mathematical analysis, analysis. ...

:$a^2\; -\; a\; +\; b^2\; =\; 0\; ,$ or $\backslash left(a\; -\; \backslash frac\backslash right)^2\; +\; b^2\; =\; \backslash frac$
which is a circle
A circle is a shape
A shape or figure is the form of an object or its external boundary, outline, or external surface
File:Water droplet lying on a damask.jpg, Water droplet lying on a damask. Surface tension is high enough to preven ...

with center (1/2, 0) and radius 1/2. In terms of an angle θ,
:$A\; =\; \backslash frac\backslash begin1\; -\; \backslash cos\backslash theta\; \&\; \backslash sin\backslash theta\; \backslash \backslash \; \backslash sin\backslash theta\; \&\; 1\; +\; \backslash cos\backslash theta\; \backslash end$ is idempotent.
However, $b=c$ is not a necessary condition: any matrix
:$\backslash begina\; \&\; b\; \backslash \backslash \; c\; \&\; 1\; -\; a\backslash end$ with $a^2\; +\; bc\; =\; a$ is idempotent.
Properties

Singularity and regularity

The only non-singular
Singular may refer to:
* Singular, the grammatical number
In linguistics, grammatical number is a grammatical category of nouns, pronouns, adjectives, and verb agreement (linguistics), agreement that expresses count distinctions (such as "one", ...

idempotent matrix is the identity matrix
In linear algebra, the identity matrix of size ''n'' is the ''n'' × ''n'' square matrix
In mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structu ...

; that is, if a non-identity matrix is idempotent, its number of independent rows (and columns) is less than its number of rows (and columns).
This can be seen from writing $A^2\; =\; A$, assuming that has full rank (is non-singular), and pre-multiplying by $A^$ to obtain $A\; =\; IA\; =\; A^A^2\; =\; A^A\; =\; I$.
When an idempotent matrix is subtracted from the identity matrix, the result is also idempotent. This holds since
:$(I-A)(I-A)\; =\; I-A-A+A^2\; =\; I-A-A+A\; =\; I-A.$
If a matrix is idempotent then for all positive integers n, $A^n\; =\; A$. This can be shown using proof by induction. Clearly we have the result for $n\; =\; 1$, as $A^1\; =\; A$. Suppose that $A^\; =\; A$. Then, $A^k\; =\; A^A\; =\; AA\; =\; A$, since is idempotent. Hence by the principle of induction, the result follows.
Eigenvalues

An idempotent matrix is alwaysdiagonalizable
In linear algebra, a square matrix A is called diagonalizable or non-defective if it is similar to a diagonal matrix, i.e., if there exists an invertible matrixIn linear algebra, an ''n''-by-''n'' square matrix is called invertible (also ...

and its eigenvalue
In linear algebra, an eigenvector () or characteristic vector of a Linear map, linear transformation is a nonzero Vector space, vector that changes at most by a Scalar (mathematics), scalar factor when that linear transformation is applied to i ...

s are either 0 or 1. This can be shown through letting $\backslash mathbf$ be a non-zero eigenvector of some idempotent matrix $A$. Then we have:$A\; \backslash mathbf\; =\backslash lambda\; x\; \backslash implies\; A^2\backslash mathbf=A\backslash lambda\; x\; =\; \backslash lambda\; A\; x\; =\; \backslash lambda^2\; x\backslash implies$$Ax\; =\backslash lambda^2\; x\; =\; \backslash implies\; \backslash lambda\; x\; =\; \backslash lambda^2\; x\; \backslash implies\; \backslash lambda\; =\; 0\; \backslash text\; 1$
Trace

Thetrace
Trace may refer to:
Arts and entertainment Music
* ''Trace'' (Son Volt album), 1995
* ''Trace'' (Died Pretty album), 1993
* Trace (band)
Trace was a Netherlands, Dutch progressive rock trio founded by Rick van der Linden in 1974 after leavin ...

of an idempotent matrix — the sum of the elements on its main diagonal — equals the rank
Rank is the relative position, value, worth, complexity, power, importance, authority, level, etc. of a person or object within a ranking
A ranking is a relationship between a set of items such that, for any two items, the first is either "rank ...

of the matrix and thus is always an integer. This provides an easy way of computing the rank, or alternatively an easy way of determining the trace of a matrix whose elements are not specifically known (which is helpful in statistics
Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data
Data (; ) are individual facts, statistics, or items of information, often numeric. In a more technical sens ...

, for example, in establishing the degree of bias
Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is closed-minded
Open-mindedness is receptiveness to new ideas. Open-mindedness relates to the way in which people approach the views and kn ...

in using a sample variance
In probability theory
Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expr ...

as an estimate of a ).
Relationships between idempotent matrices

In regression analysis, the matrix $M\; =\; I\; -\; X(X\text{'}X)^\; X\text{'}$ is known to produce the residuals $e$ from the regression of the vector of dependent variables $y$ on the matrix of covariates $X$. (See the section on Applications.) Now, let $X\_1$ be a matrix formed from a subset of the columns of $X$, and let $M\_1\; =\; I\; -\; X\_1\; (X\_1\text{'}X\_1)^X\_1\text{'}$. It is easy to show that both $M$ and $M\_1$ are idempotent, but a somewhat surprising fact is that $M\; M\_1\; =\; M$. This is because $M\; X\_1\; =\; 0$, or in other words, the residuals from the regression of the columns of $X\_1$ on $X$ are 0 since $X\_1$ can be perfectly interpolated as it is a subset of $X$ (by direct substitution it is also straightforward to show that $M\; X\; =\; 0$). This leads to two other important results: one is that $(M\_1\; -\; M)$ is symmetric and idempotent, and the other is that $(M\_1\; -\; M)\; M\; =\; 0$, i.e., $(M\_1\; -\; M)$ is orthogonal to $M$. These results play a key role, for example, in the derivation of the F test.Applications

Idempotent matrices arise frequently inregression analysis
In ing, regression analysis is a set of statistical processes for the relationships between a (often called the 'outcome' or 'response' variable) and one or more s (often called 'predictors', 'covariates', 'explanatory variables' or 'features' ...

and econometrics
Econometrics is the application of Statistics, statistical methods to economic data in order to give Empirical evidence, empirical content to economic relationships.M. Hashem Pesaran (1987). "Econometrics," ''The New Palgrave: A Dictionary of Econ ...

. For example, in ordinary least squares
In statistics, ordinary least squares (OLS) is a type of linear least squares method for estimating the unknown statistical parameter, parameters in a linear regression model. OLS chooses the parameters of a linear function of a set of explanatory ...

, the regression problem is to choose a vector of coefficient estimates so as to minimize the sum of squared residuals (mispredictions) ''e''dependent variable
Dependent and Independent variables are variables in mathematical modeling, statistical modeling and experimental sciences. Dependent variables receive this name because, in an experiment, their values are studied under the supposition or demand ...

observations, and $X$ is a matrix each of whose columns is a column of observations on one of the independent variables
Dependent and Independent variables are variables in mathematical modeling
A mathematical model is a description of a system
A system is a group of Interaction, interacting or interrelated elements that act according to a set of rules to f ...

. The resulting estimator is
:$\backslash hat\backslash beta\; =\; \backslash left(X^\backslash textsfX\backslash right)^X^\backslash textsfy$
where superscript ''T'' indicates a transpose
In linear algebra
Linear algebra is the branch of mathematics concerning linear equations such as:
:a_1x_1+\cdots +a_nx_n=b,
linear maps such as:
:(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n,
and their representations in vector spaces a ...

, and the vector of residuals is
:$\backslash hat\; =\; y\; -\; X\; \backslash hat\backslash beta\; =\; y\; -\; X\backslash left(X^\backslash textsfX\backslash right)^X^\backslash textsfy\; =\; \backslash left;\; href="/html/ALL/s/\_-\_X\backslash left(X^\backslash textsfX\backslash right)^X^\backslash textsf\backslash right.html"\; ;"title="\; -\; X\backslash left(X^\backslash textsfX\backslash right)^X^\backslash textsf\backslash right">\; -\; X\backslash left(X^\backslash textsfX\backslash right)^X^\backslash textsf\backslash right$
Here both $M$ and $X\backslash left(X^\backslash textsfX\backslash right)^X^\backslash textsf$(the latter being known as the hat matrix
In statistics
Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industrial, or social problem, it is conventional to begin with ...

) are idempotent and symmetric matrices, a fact which allows simplification when the sum of squared residuals is computed:
:$\backslash hat^\backslash textsf\backslash hat\; =\; (My)^\backslash textsf(My)\; =\; y^\backslash textsfM^\backslash textsfMy\; =\; y^\backslash textsfMMy\; =\; y^\backslash textsfMy.$
The idempotency of $M$ plays a role in other calculations as well, such as in determining the variance of the estimator $\backslash hat$.
An idempotent linear operator $P$ is a projection operator on the range space along its null space
In mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...

. $P$ is an orthogonal projection
In linear algebra and functional analysis
Image:Drum vibration mode12.gif, 200px, One of the possible modes of vibration of an idealized circular drum head. These modes are eigenfunctions of a linear operator on a function space, a common const ...

operator if and only if it is idempotent and symmetric
Symmetry (from Greek συμμετρία ''symmetria'' "agreement in dimensions, due proportion, arrangement") in everyday language refers to a sense of harmonious and beautiful proportion and balance. In mathematics, "symmetry" has a more pre ...

.
See also

*Idempotence
Idempotence (, ) is the property of certain operations
Operation or Operations may refer to:
Science and technology
* Surgical operation
Surgery ''cheirourgikē'' (composed of χείρ, "hand", and ἔργον, "work"), via la, chirurgiae, m ...

* Nilpotent
In mathematics
Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...

* Projection (linear algebra)
In linear algebra
Linear algebra is the branch of mathematics concerning linear equations such as:
:a_1x_1+\cdots +a_nx_n=b,
linear maps such as:
:(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n,
and their representations in vector spaces a ...

* Hat matrix
In statistics
Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industrial, or social problem, it is conventional to begin with ...

References

{{Matrix classes Algebra Regression analysis Matrices