In
linear algebra
Linear algebra is the branch of mathematics concerning linear equations such as:
:a_1x_1+\cdots +a_nx_n=b,
linear maps such as:
:(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n,
and their representations in vector spaces and through matrices ...
, a generalized eigenvector of an
matrix
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** ''The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
is a
vector
Vector most often refers to:
*Euclidean vector, a quantity with a magnitude and a direction
*Vector (epidemiology), an agent that carries and transmits an infectious pathogen into another living organism
Vector may also refer to:
Mathematic ...
which satisfies certain criteria which are more relaxed than those for an (ordinary)
eigenvector
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted ...
.
Let
be an
-dimensional
vector space
In mathematics and physics, a vector space (also called a linear space) is a set whose elements, often called '' vectors'', may be added together and multiplied ("scaled") by numbers called ''scalars''. Scalars are often real numbers, but can ...
; let
be a
linear map
In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that pr ...
in , the set of all linear maps from
into itself; and let
be the
matrix representation
Matrix representation is a method used by a computer language to store matrix (mathematics), matrices of more than one dimension in computer storage, memory.
Fortran and C (programming language), C use different schemes for their native arrays. Fo ...
of
with respect to some ordered
basis
Basis may refer to:
Finance and accounting
* Adjusted basis, the net cost of an asset after adjusting for various tax-related items
*Basis point, 0.01%, often used in the context of interest rates
* Basis trading, a trading strategy consisting ...
.
There may not always exist a full set of
linearly independent
In the theory of vector spaces, a set of vectors is said to be if there is a nontrivial linear combination of the vectors that equals the zero vector. If no such linear combination exists, then the vectors are said to be . These concepts are ...
eigenvectors of
that form a complete basis for
. That is, the matrix
may not be
diagonalizable
In linear algebra, a square matrix AÂ is called diagonalizable or non-defective if it is similar to a diagonal matrix, i.e., if there exists an invertible matrix PÂ and a diagonal matrix D such that or equivalently (Such D are not unique.) F ...
. This happens when the
algebraic multiplicity
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted ...
of at least one
eigenvalue
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted ...
is greater than its
geometric multiplicity
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted ...
(the
nullity of the matrix
, or the
dimension
In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coor ...
of its
nullspace
In mathematics, the kernel of a linear map, also known as the null space or nullspace, is the linear subspace of the domain of the map which is mapped to the zero vector. That is, given a linear map between two vector spaces and , the kernel of ...
). In this case,
is called a
defective eigenvalue and
is called a
defective matrix
In linear algebra, a defective matrix is a square matrix that does not have a complete basis of eigenvectors, and is therefore not diagonalizable. In particular, an ''n'' × ''n'' matrix is defective if and only if it does not hav ...
.
A generalized eigenvector
corresponding to
, together with the matrix
generate a Jordan chain of linearly independent generalized eigenvectors which form a basis for an
invariant subspace In mathematics, an invariant subspace of a linear mapping ''T'' : ''V'' → ''V '' i.e. from some vector space ''V'' to itself, is a subspace ''W'' of ''V'' that is preserved by ''T''; that is, ''T''(''W'') ⊆ ''W''.
General descri ...
of
.
Using generalized eigenvectors, a set of linearly independent eigenvectors of
can be extended, if necessary, to a complete basis for
. This basis can be used to determine an "almost diagonal matrix"
in
Jordan normal form
In linear algebra, a Jordan normal form, also known as a Jordan canonical form (JCF),
is an upper triangular matrix of a particular form called a Jordan matrix representing a linear operator on a finite-dimensional vector space with respect to so ...
,
similar to
, which is useful in computing certain
matrix function
In mathematics, every analytic function can be used for defining a matrix function that maps square matrices with complex entries to square matrices of the same size.
This is used for defining the exponential of a matrix, which is involved in th ...
s of
. The matrix
is also useful in solving the
system of linear differential equations
In mathematics, a linear differential equation is a differential equation that is defined by a linear polynomial in the unknown function and its derivatives, that is an equation of the form
:a_0(x)y + a_1(x)y' + a_2(x)y'' \cdots + a_n(x)y^ = ...
where
need not be diagonalizable.
The dimension of the generalized eigenspace corresponding to a given eigenvalue
is the algebraic multiplicity of
.
Overview and definition
There are several equivalent ways to define an ordinary eigenvector. For our purposes, an eigenvector
associated with an eigenvalue
of an
×
matrix
is a nonzero vector for which
, where
is the
×
identity matrix and
is the
zero vector
In mathematics, a zero element is one of several generalizations of 0, the number zero to other algebraic structures. These alternate meanings may or may not reduce to the same thing, depending on the context.
Additive identities
An additive iden ...
of length
. That is,
is in the
kernel
Kernel may refer to:
Computing
* Kernel (operating system), the central component of most operating systems
* Kernel (image processing), a matrix used for image convolution
* Compute kernel, in GPGPU programming
* Kernel method, in machine learn ...
of the
transformation
Transformation may refer to:
Science and mathematics
In biology and medicine
* Metamorphosis, the biological process of changing physical form after birth or hatching
* Malignant transformation, the process of cells becoming cancerous
* Tran ...
. If
has
linearly independent eigenvectors, then
is similar to a diagonal matrix
. That is, there exists an
invertible matrix
In linear algebra, an -by- square matrix is called invertible (also nonsingular or nondegenerate), if there exists an -by- square matrix such that
:\mathbf = \mathbf = \mathbf_n \
where denotes the -by- identity matrix and the multiplicati ...
such that
is diagonalizable through the similarity transformation
. The matrix
is called a
spectral matrix In linear algebra, the modal matrix is used in the diagonalization process involving eigenvalues and eigenvectors.
Specifically the modal matrix M for the matrix A is the ''n'' × ''n'' matrix formed with the eigenvectors of A as columns in M. It ...
for
. The matrix
is called a
modal matrix In linear algebra, the modal matrix is used in the diagonalization process involving eigenvalues and eigenvectors.
Specifically the modal matrix M for the matrix A is the ''n'' × ''n'' matrix formed with the eigenvectors of A as columns in M. It ...
for
. Diagonalizable matrices are of particular interest since matrix functions of them can be computed easily.
On the other hand, if
does not have
linearly independent eigenvectors associated with it, then
is not diagonalizable.
Definition: A vector
is a generalized eigenvector of rank ''m'' of the matrix
and corresponding to the eigenvalue
if
:
but
:
Clearly, a generalized eigenvector of rank 1 is an ordinary eigenvector. Every
×
matrix
has
linearly independent generalized eigenvectors associated with it and can be shown to be similar to an "almost diagonal" matrix
in Jordan normal form. That is, there exists an invertible matrix
such that
. The matrix
in this case is called a
generalized modal matrix In linear algebra, the modal matrix is used in the diagonalization process involving eigenvalues and eigenvectors.
Specifically the modal matrix M for the matrix A is the ''n'' × ''n'' matrix formed with the eigenvectors of A as columns in M. It ...
for
. If
is an eigenvalue of algebraic multiplicity
, then
will have
linearly independent generalized eigenvectors corresponding to
. These results, in turn, provide a straightforward method for computing certain matrix functions of
.
Note: For an
matrix
over a
field
Field may refer to:
Expanses of open ground
* Field (agriculture), an area of land used for agricultural purposes
* Airfield, an aerodrome that lacks the infrastructure of an airport
* Battlefield
* Lawn, an area of mowed grass
* Meadow, a grass ...
to be expressed in Jordan normal form, all eigenvalues of
must be in
. That is, the
characteristic polynomial must factor completely into linear factors. For example, if
has
real-valued
In mathematics, value may refer to several, strongly related notions.
In general, a mathematical value may be any definite mathematical object. In elementary mathematics, this is most often a number – for example, a real number such as or an i ...
elements, then it may be necessary for the eigenvalues and the components of the eigenvectors to have
complex values.
The set
spanned by all generalized eigenvectors for a given
, forms the generalized eigenspace for
.
Examples
Here are some examples to illustrate the concept of generalized eigenvectors. Some of the details will be described later.
Example 1
This example is simple but clearly illustrates the point. This type of matrix is used frequently in textbooks.
Suppose
:
Then there is only one eigenvalue,
, and its algebraic multiplicity is ''m'' = 2.
Notice that this matrix is in Jordan normal form but is not
diagonal
In geometry, a diagonal is a line segment joining two vertices of a polygon or polyhedron, when those vertices are not on the same edge. Informally, any sloping line is called diagonal. The word ''diagonal'' derives from the ancient Greek δΠ...
. Hence, this matrix is not diagonalizable. Since there is one
superdiagonal entry, there will be one generalized eigenvector of rank greater than 1 (or one could note that the vector space
is of dimension 2, so there can be at most one generalized eigenvector of rank greater than 1). Alternatively, one could compute the dimension of the
nullspace
In mathematics, the kernel of a linear map, also known as the null space or nullspace, is the linear subspace of the domain of the map which is mapped to the zero vector. That is, given a linear map between two vector spaces and , the kernel of ...
of
to be ''p'' = 1, and thus there are ''m'' – ''p'' = 1 generalized eigenvectors of rank greater than 1.
The ordinary eigenvector
is computed as usual (see the
eigenvector
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted ...
page for examples). Using this eigenvector, we compute the generalized eigenvector
by solving
:
Writing out the values:
:
This simplifies to
:
The element
has no restrictions. The generalized eigenvector of rank 2 is then
, where ''a'' can have any scalar value. The choice of ''a'' = 0 is usually the simplest.
Note that
:
so that
is a generalized eigenvector,
:
so that
is an ordinary eigenvector, and that
and
are linearly independent and hence constitute a basis for the vector space
.
Example 2
This example is more complex than
Example 1. Unfortunately, it is a little difficult to construct an interesting example of low order.
The matrix
:
has ''eigenvalues''
and
with ''algebraic multiplicities''
and
, but ''geometric multiplicities''
and
.
The ''generalized eigenspaces'' of
are calculated below.
is the ordinary eigenvector associated with
.
is a generalized eigenvector associated with
.
is the ordinary eigenvector associated with
.
and
are generalized eigenvectors associated with
.
:
:
:
:
:
This results in a basis for each of the ''generalized eigenspaces'' of
.
Together the two ''chains'' of generalized eigenvectors span the space of all 5-dimensional column vectors.
:
An "almost diagonal" matrix
in ''Jordan normal form'', similar to
is obtained as follows:
:
:
where
is a
generalized modal matrix In linear algebra, the modal matrix is used in the diagonalization process involving eigenvalues and eigenvectors.
Specifically the modal matrix M for the matrix A is the ''n'' × ''n'' matrix formed with the eigenvectors of A as columns in M. It ...
for
, the columns of
are a
canonical basis
In mathematics, a canonical basis is a basis of an algebraic structure that is canonical in a sense that depends on the precise context:
* In a coordinate space, and more generally in a free module, it refers to the standard basis defined by the ...
for
, and
.
Jordan chains
Definition: Let
be a generalized eigenvector of rank ''m'' corresponding to the matrix
and the eigenvalue
. The chain generated by
is a set of vectors
given by
Thus, in general,
The vector
, given by (), is a generalized eigenvector of rank ''j'' corresponding to the eigenvalue
. A chain is a linearly independent set of vectors.
Canonical basis
Definition: A set of ''n'' linearly independent generalized eigenvectors is a canonical basis if it is composed entirely of Jordan chains.
Thus, once we have determined that a generalized eigenvector of rank ''m'' is in a canonical basis, it follows that the ''m'' − 1 vectors
that are in the Jordan chain generated by
are also in the canonical basis.
Let
be an eigenvalue of
of algebraic multiplicity
. First, find the
ranks (matrix ranks) of the matrices
. The integer
is determined to be the ''first integer'' for which
has rank
(''n'' being the number of rows or columns of
, that is,
is ''n'' × ''n'').
Now define
:
The variable
designates the number of linearly independent generalized eigenvectors of rank ''k'' corresponding to the eigenvalue
that will appear in a canonical basis for
. Note that
:
.
Computation of generalized eigenvectors
In the preceding sections we have seen techniques for obtaining the
linearly independent generalized eigenvectors of a canonical basis for the vector space
associated with an
matrix
. These techniques can be combined into a procedure:
:Solve the
characteristic equation of
for eigenvalues
and their algebraic multiplicities
;
:For each
::Determine
;
::Determine
;
::Determine
for
;
::Determine each Jordan chain for
;
Example 3
The matrix
:
has an eigenvalue
of algebraic multiplicity
and an eigenvalue
of algebraic multiplicity
. We also have
. For
we have
.
:
:
:
The first integer
for which
has rank
is
.
We now define
:
:
:
Consequently, there will be three linearly independent generalized eigenvectors; one each of ranks 3, 2 and 1. Since
corresponds to a single chain of three linearly independent generalized eigenvectors, we know that there is a generalized eigenvector
of rank 3 corresponding to
such that
but
Equations () and () represent
linear systems that can be solved for
. Let
:
Then
:
and
:
Thus, in order to satisfy the conditions () and (), we must have
and
. No restrictions are placed on
and
. By choosing
, we obtain
:
as a generalized eigenvector of rank 3 corresponding to
. Note that it is possible to obtain infinitely many other generalized eigenvectors of rank 3 by choosing different values of
,
and
, with
. Our first choice, however, is the simplest.
Now using equations (), we obtain
and
as generalized eigenvectors of rank 2 and 1, respectively, where
:
and
:
The
simple eigenvalue can be dealt with using
standard techniques and has an ordinary eigenvector
:
A canonical basis for
is
:
and
are generalized eigenvectors associated with
, while
is the ordinary eigenvector associated with
.
This is a fairly simple example. In general, the numbers
of linearly independent generalized eigenvectors of rank
will not always be equal. That is, there may be several chains of different lengths corresponding to a particular eigenvalue.
Generalized modal matrix
Let
be an ''n'' × ''n'' matrix. A generalized modal matrix
for
is an ''n'' × ''n'' matrix whose columns, considered as vectors, form a canonical basis for
and appear in
according to the following rules:
* All Jordan chains consisting of one vector (that is, one vector in length) appear in the first columns of
.
* All vectors of one chain appear together in adjacent columns of
.
* Each chain appears in
in order of increasing rank (that is, the generalized eigenvector of rank 1 appears before the generalized eigenvector of rank 2 of the same chain, which appears before the generalized eigenvector of rank 3 of the same chain, etc.).
Jordan normal form
Let
be an ''n''-dimensional vector space; let
be a linear map in , the set of all linear maps from
into itself; and let
be the matrix representation of
with respect to some ordered basis. It can be shown that if the
characteristic polynomial of
factors into linear factors, so that
has the form
:
where
are the distinct eigenvalues of
, then each
is the algebraic multiplicity of its corresponding eigenvalue
and
is similar to a matrix
in Jordan normal form, where each
appears
consecutive times on the diagonal, and the entry directly above each
(that is, on the
superdiagonal) is either 0 or 1. All other entries (that is, off the diagonal and superdiagonal) are 0. More precisely,
is a
Jordan matrix
In the mathematical discipline of matrix theory, a Jordan matrix, named after Camille Jordan, is a block diagonal matrix over a ring (whose identities are the zero 0 and one 1), where each block along the diagonal, called a Jordan block, has the ...
whose Jordan blocks corresponding to the same eigenvalue are grouped together (but no ordering is imposed among the eigenvalues, nor among the blocks for a given eigenvalue). The matrix
is as close as one can come to a diagonalization of
. If
is diagonalizable, then all entries above the diagonal are zero. Note that some textbooks have the ones on the
subdiagonal, that is, immediately below the main diagonal instead of on the superdiagonal. The eigenvalues are still on the main diagonal.
Every ''n'' × ''n'' matrix
is similar to a matrix
in Jordan normal form, obtained through the similarity transformation
, where
is a generalized modal matrix for
. (See
Note
Note, notes, or NOTE may refer to:
Music and entertainment
* Musical note, a pitched sound (or a symbol for a sound) in music
* ''Notes'' (album), a 1987 album by Paul Bley and Paul Motian
* ''Notes'', a common (yet unofficial) shortened version ...
above.)
Example 4
Find a matrix in Jordan normal form that is similar to
:
Solution: The characteristic equation of
is
, hence,
is an eigenvalue of algebraic multiplicity three. Following the procedures of the previous sections, we find that
:
and
:
Thus,
and
, which implies that a canonical basis for
will contain one linearly independent generalized eigenvector of rank 2 and two linearly independent generalized eigenvectors of rank 1, or equivalently, one chain of two vectors
and one chain of one vector
. Designating
, we find that
:
and
:
where
is a generalized modal matrix for
, the columns of
are a canonical basis for
, and
. Note that since generalized eigenvectors themselves are not unique, and since some of the columns of both
and
may be interchanged, it follows that both
and
are not unique.
Example 5
In
Example 3, we found a canonical basis of linearly independent generalized eigenvectors for a matrix
. A generalized modal matrix for
is
:
A matrix in Jordan normal form, similar to
is
:
so that
.
Applications
Matrix functions
Three of the most fundamental operations which can be performed on
square matrices
In mathematics, a square matrix is a matrix with the same number of rows and columns. An ''n''-by-''n'' matrix is known as a square matrix of order Any two square matrices of the same order can be added and multiplied.
Square matrices are often ...
are matrix addition, multiplication by a scalar, and matrix multiplication. These are exactly those operations necessary for defining a
polynomial
In mathematics, a polynomial is an expression consisting of indeterminates (also called variables) and coefficients, that involves only the operations of addition, subtraction, multiplication, and positive-integer powers of variables. An example ...
function of an ''n'' × ''n'' matrix
. If we recall from basic
calculus
Calculus, originally called infinitesimal calculus or "the calculus of infinitesimals", is the mathematical study of continuous change, in the same way that geometry is the study of shape, and algebra is the study of generalizations of arithm ...
that many functions can be written as a
Maclaurin series
Maclaurin or MacLaurin is a surname. Notable people with the surname include:
* Colin Maclaurin (1698–1746), Scottish mathematician
* Normand MacLaurin (1835–1914), Australian politician and university administrator
* Henry Normand MacLaurin ...
, then we can define more general functions of matrices quite easily. If
is diagonalizable, that is
:
with
:
then
:
and the evaluation of the Maclaurin series for functions of
is greatly simplified. For example, to obtain any power ''k'' of
, we need only compute
, premultiply
by
, and postmultiply the result by
.
Using generalized eigenvectors, we can obtain the Jordan normal form for
and these results can be generalized to a straightforward method for computing functions of nondiagonalizable matrices. (See
Matrix function#Jordan decomposition.)
Differential equations
Consider the problem of solving the system of linear ordinary differential equations
where
:
and
If the matrix
is a diagonal matrix so that
for
, then the system () reduces to a system of ''n'' equations which take the form
In this case, the general solution is given by
:
:
::
:
In the general case, we try to diagonalize
and reduce the system () to a system like () as follows. If
is diagonalizable, we have
, where
is a modal matrix for
. Substituting
, equation () takes the form
, or
where
The solution of () is
:
:
::
:
The solution
of () is then obtained using the relation ().
On the other hand, if
is not diagonalizable, we choose
to be a generalized modal matrix for
, such that
is the Jordan normal form of
. The system
has the form
where the
are the eigenvalues from the main diagonal of
and the
are the ones and zeros from the superdiagonal of
. The system () is often more easily solved than (). We may solve the last equation in () for
, obtaining
. We then substitute this solution for
into the next to last equation in () and solve for
. Continuing this procedure, we work through () from the last equation to the first, solving the entire system for
. The solution
is then obtained using the relation ().
Lemma: Given the following chain of generalized eigenvector of length r as:
:
:
:
:
:
These functions solve the system of equations:
:
Proof: let define the following sum:
:
Then:
:
on the other hand we have:
:
:
:
:
:
as required.
Notes
References
*
*
*
*
*
*
*
*
*
*
*
*
{{Areas of mathematics, collapsed
Linear algebra
Matrix theory