HOME

TheInfoList




In
linear algebra Linear algebra is the branch of mathematics concerning linear equations such as: :a_1x_1+\cdots +a_nx_n=b, linear maps such as: :(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n, and their representations in vector spaces and through matrix (mat ...
, an idempotent matrix is a
matrix Matrix or MATRIX may refer to: Science and mathematics * Matrix (mathematics), a rectangular array of numbers, symbols, or expressions * Matrix (logic), part of a formula in prenex normal form * Matrix (biology), the material in between a eukaryoti ...
which, when multiplied by itself, yields itself. That is, the matrix A is idempotent if and only if A^2 = A. For this product A^2 to be
defined A definition is a statement of the meaning of a term (a word, phrase, or other set of symbols). Definitions can be classified into two large categories, intensional definitions (which try to give the sense of a term) and extensional definitions ...

defined
, A must necessarily be a
square matrix In mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...
. Viewed this way, idempotent matrices are
idempotent element Idempotence (, ) is the property of certain operation (mathematics), operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. The concept of idempotence a ...
s of
matrix ring In abstract algebra In algebra, which is a broad division of mathematics, abstract algebra (occasionally called modern algebra) is the study of algebraic structures. Algebraic structures include group (mathematics), groups, ring (mathematics), ...
s.


Example

Examples of 2 \times 2 idempotent matrices are: \begin 1 & 0 \\ 0 & 1 \end \qquad \begin 3 & -6 \\ 1 & -2 \end Examples of 3 \times 3 idempotent matrices are: \begin 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end \qquad \begin 2 & -2 & -4 \\ -1 & 3 & 4 \\ 1 & -2 & -3 \end


Real 2 × 2 case

If a matrix \begina & b \\ c & d \end is idempotent, then * a = a^2 + bc, * b = ab + bd, implying b(1 - a - d) = 0 so b = 0 or d = 1 - a, * c = ca + cd, implying c(1 - a - d) = 0 so c = 0 or d = 1 - a, * d = bc + d^2. Thus a necessary condition for a 2 × 2 matrix to be idempotent is that either it is
diagonal In geometry Geometry (from the grc, γεωμετρία; ' "earth", ' "measurement") is, with , one of the oldest branches of . It is concerned with properties of space that are related with distance, shape, size, and relative position o ...
or its
trace Trace may refer to: Arts and entertainment Music * ''Trace'' (Son Volt album), 1995 * ''Trace'' (Died Pretty album), 1993 * Trace (band) Trace was a Netherlands, Dutch progressive rock trio founded by Rick van der Linden in 1974 after leavin ...
equals 1. For idempotent diagonal matrices, a and d must be either 1 or 0. If b=c, the matrix \begina & b \\ b & 1 - a \end will be idempotent provided a^2 + b^2 = a , so ''a'' satisfies the
quadratic equation In algebra Algebra (from ar, الجبر, lit=reunion of broken parts, bonesetting, translit=al-jabr) is one of the areas of mathematics, broad areas of mathematics, together with number theory, geometry and mathematical analysis, analysis. ...

quadratic equation
:a^2 - a + b^2 = 0 , or \left(a - \frac\right)^2 + b^2 = \frac which is a
circle A circle is a shape A shape or figure is the form of an object or its external boundary, outline, or external surface File:Water droplet lying on a damask.jpg, Water droplet lying on a damask. Surface tension is high enough to preven ...

circle
with center (1/2, 0) and radius 1/2. In terms of an angle θ, :A = \frac\begin1 - \cos\theta & \sin\theta \\ \sin\theta & 1 + \cos\theta \end is idempotent. However, b=c is not a necessary condition: any matrix :\begina & b \\ c & 1 - a\end with a^2 + bc = a is idempotent.


Properties


Singularity and regularity

The only non-
singular Singular may refer to: * Singular, the grammatical number In linguistics, grammatical number is a grammatical category of nouns, pronouns, adjectives, and verb agreement (linguistics), agreement that expresses count distinctions (such as "one", ...
idempotent matrix is the
identity matrix In linear algebra, the identity matrix of size ''n'' is the ''n'' × ''n'' square matrix In mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structu ...

identity matrix
; that is, if a non-identity matrix is idempotent, its number of independent rows (and columns) is less than its number of rows (and columns). This can be seen from writing A^2 = A, assuming that has full rank (is non-singular), and pre-multiplying by A^ to obtain A = IA = A^A^2 = A^A = I. When an idempotent matrix is subtracted from the identity matrix, the result is also idempotent. This holds since :(I-A)(I-A) = I-A-A+A^2 = I-A-A+A = I-A. If a matrix is idempotent then for all positive integers n, A^n = A. This can be shown using proof by induction. Clearly we have the result for n = 1, as A^1 = A. Suppose that A^ = A. Then, A^k = A^A = AA = A, since is idempotent. Hence by the principle of induction, the result follows.


Eigenvalues

An idempotent matrix is always
diagonalizable In linear algebra, a square matrix A is called diagonalizable or non-defective if it is similar to a diagonal matrix, i.e., if there exists an invertible matrixIn linear algebra, an ''n''-by-''n'' square matrix is called invertible (also ...
and its
eigenvalue In linear algebra, an eigenvector () or characteristic vector of a Linear map, linear transformation is a nonzero Vector space, vector that changes at most by a Scalar (mathematics), scalar factor when that linear transformation is applied to i ...

eigenvalue
s are either 0 or 1. This can be shown through letting \mathbf be a non-zero eigenvector of some idempotent matrix A. Then we have:A \mathbf =\lambda x \implies A^2\mathbf=A\lambda x = \lambda A x = \lambda^2 x\implies Ax =\lambda^2 x = \implies \lambda x = \lambda^2 x \implies \lambda = 0 \text 1


Trace

The
trace Trace may refer to: Arts and entertainment Music * ''Trace'' (Son Volt album), 1995 * ''Trace'' (Died Pretty album), 1993 * Trace (band) Trace was a Netherlands, Dutch progressive rock trio founded by Rick van der Linden in 1974 after leavin ...
of an idempotent matrix — the sum of the elements on its main diagonal — equals the
rank Rank is the relative position, value, worth, complexity, power, importance, authority, level, etc. of a person or object within a ranking A ranking is a relationship between a set of items such that, for any two items, the first is either "rank ...
of the matrix and thus is always an integer. This provides an easy way of computing the rank, or alternatively an easy way of determining the trace of a matrix whose elements are not specifically known (which is helpful in
statistics Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data Data (; ) are individual facts, statistics, or items of information, often numeric. In a more technical sens ...

statistics
, for example, in establishing the degree of
bias Bias is a disproportionate weight ''in favor of'' or ''against'' an idea or thing, usually in a way that is closed-minded Open-mindedness is receptiveness to new ideas. Open-mindedness relates to the way in which people approach the views and kn ...
in using a
sample variance In probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expr ...

sample variance
as an estimate of a
population variance
population variance
).


Relationships between idempotent matrices

In regression analysis, the matrix M = I - X(X'X)^ X' is known to produce the residuals e from the regression of the vector of dependent variables y on the matrix of covariates X. (See the section on Applications.) Now, let X_1 be a matrix formed from a subset of the columns of X, and let M_1 = I - X_1 (X_1'X_1)^X_1'. It is easy to show that both M and M_1 are idempotent, but a somewhat surprising fact is that M M_1 = M. This is because M X_1 = 0, or in other words, the residuals from the regression of the columns of X_1 on X are 0 since X_1 can be perfectly interpolated as it is a subset of X (by direct substitution it is also straightforward to show that M X = 0). This leads to two other important results: one is that (M_1 - M) is symmetric and idempotent, and the other is that (M_1 - M) M = 0, i.e., (M_1 - M) is orthogonal to M. These results play a key role, for example, in the derivation of the F test.


Applications

Idempotent matrices arise frequently in
regression analysis In ing, regression analysis is a set of statistical processes for the relationships between a (often called the 'outcome' or 'response' variable) and one or more s (often called 'predictors', 'covariates', 'explanatory variables' or 'features' ...
and
econometrics Econometrics is the application of Statistics, statistical methods to economic data in order to give Empirical evidence, empirical content to economic relationships.M. Hashem Pesaran (1987). "Econometrics," ''The New Palgrave: A Dictionary of Econ ...

econometrics
. For example, in
ordinary least squares In statistics, ordinary least squares (OLS) is a type of linear least squares method for estimating the unknown statistical parameter, parameters in a linear regression model. OLS chooses the parameters of a linear function of a set of explanatory ...
, the regression problem is to choose a vector of coefficient estimates so as to minimize the sum of squared residuals (mispredictions) ''e''''i'': in matrix form, : Minimize (y - X\beta)^\textsf(y - X\beta) where y is a vector of
dependent variable Dependent and Independent variables are variables in mathematical modeling, statistical modeling and experimental sciences. Dependent variables receive this name because, in an experiment, their values are studied under the supposition or demand ...

dependent variable
observations, and X is a matrix each of whose columns is a column of observations on one of the
independent variables Dependent and Independent variables are variables in mathematical modeling A mathematical model is a description of a system A system is a group of Interaction, interacting or interrelated elements that act according to a set of rules to f ...

independent variables
. The resulting estimator is :\hat\beta = \left(X^\textsfX\right)^X^\textsfy where superscript ''T'' indicates a
transpose In linear algebra Linear algebra is the branch of mathematics concerning linear equations such as: :a_1x_1+\cdots +a_nx_n=b, linear maps such as: :(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n, and their representations in vector spaces a ...

transpose
, and the vector of residuals is : \hat = y - X \hat\beta = y - X\left(X^\textsfX\right)^X^\textsfy = \left - X\left(X^\textsfX\right)^X^\textsf\right = My. Here both M and X\left(X^\textsfX\right)^X^\textsf(the latter being known as the
hat matrix In statistics Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industrial, or social problem, it is conventional to begin with ...
) are idempotent and symmetric matrices, a fact which allows simplification when the sum of squared residuals is computed: :\hat^\textsf\hat = (My)^\textsf(My) = y^\textsfM^\textsfMy = y^\textsfMMy = y^\textsfMy. The idempotency of M plays a role in other calculations as well, such as in determining the variance of the estimator \hat. An idempotent linear operator P is a projection operator on the range space along its
null space In mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...
. P is an
orthogonal projection In linear algebra and functional analysis Image:Drum vibration mode12.gif, 200px, One of the possible modes of vibration of an idealized circular drum head. These modes are eigenfunctions of a linear operator on a function space, a common const ...

orthogonal projection
operator if and only if it is idempotent and
symmetric Symmetry (from Greek συμμετρία ''symmetria'' "agreement in dimensions, due proportion, arrangement") in everyday language refers to a sense of harmonious and beautiful proportion and balance. In mathematics, "symmetry" has a more pre ...
.


See also

*
Idempotence Idempotence (, ) is the property of certain operations Operation or Operations may refer to: Science and technology * Surgical operation Surgery ''cheirourgikē'' (composed of χείρ, "hand", and ἔργον, "work"), via la, chirurgiae, m ...
*
Nilpotent In mathematics Mathematics (from Ancient Greek, Greek: ) includes the study of such topics as quantity (number theory), mathematical structure, structure (algebra), space (geometry), and calculus, change (mathematical analysis, analysis). It ...
*
Projection (linear algebra) In linear algebra Linear algebra is the branch of mathematics concerning linear equations such as: :a_1x_1+\cdots +a_nx_n=b, linear maps such as: :(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n, and their representations in vector spaces a ...
*
Hat matrix In statistics Statistics is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industrial, or social problem, it is conventional to begin with ...


References

{{Matrix classes Algebra Regression analysis Matrices