The Rayleigh–Ritz method is a direct numerical method of approximating
eigenvalues
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
, originated in the context of solving physical
boundary value problems
Boundary or Boundaries may refer to:
* Border, in political geography
Entertainment
* ''Boundaries'' (2016 film), a 2016 Canadian film
* ''Boundaries'' (2018 film), a 2018 American-Canadian road trip film
*Boundary (cricket), the edge of the pla ...
and named after
Lord Rayleigh
John William Strutt, 3rd Baron Rayleigh, (; 12 November 1842 – 30 June 1919) was an English mathematician and physicist who made extensive contributions to science. He spent all of his academic career at the University of Cambridge. Amo ...
and
Walther Ritz
Walther Heinrich Wilhelm Ritz (22 February 1878 – 7 July 1909) was a Swiss theoretical physicist. He is most famous for his work with Johannes Rydberg on the Rydberg–Ritz combination principle. Ritz is also known for the variational method n ...
.
The name Rayleigh–Ritz is being debated
vs. the
Ritz method
The Ritz method is a direct method to find an approximate solution for boundary value problems. The method is named after Walther Ritz, and is also commonly called the Rayleigh–Ritz method and the Ritz-Galerkin method.
In quantum mechanics, ...
after
Walther Ritz
Walther Heinrich Wilhelm Ritz (22 February 1878 – 7 July 1909) was a Swiss theoretical physicist. He is most famous for his work with Johannes Rydberg on the Rydberg–Ritz combination principle. Ritz is also known for the variational method n ...
, since the numerical procedure has been published by
Walther Ritz
Walther Heinrich Wilhelm Ritz (22 February 1878 – 7 July 1909) was a Swiss theoretical physicist. He is most famous for his work with Johannes Rydberg on the Rydberg–Ritz combination principle. Ritz is also known for the variational method n ...
in 1908-1909. According to,
Lord Rayleigh
John William Strutt, 3rd Baron Rayleigh, (; 12 November 1842 – 30 June 1919) was an English mathematician and physicist who made extensive contributions to science. He spent all of his academic career at the University of Cambridge. Amo ...
wrote a paper congratulating Ritz on his work in 1911, but stating that he himself had used Ritz's method in many places in his book and in another publication. This statement, although later disputed, and the fact that the method in the trivial case of a single vector results in the
Rayleigh quotient
In mathematics, the Rayleigh quotient () for a given complex Hermitian matrix ''M'' and nonzero vector ''x'' is defined as:
R(M,x) = .
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the co ...
make the arguable misnomer persist. According to,
citing
Richard Courant
Richard Courant (January 8, 1888 – January 27, 1972) was a German American mathematician. He is best known by the general public for the book '' What is Mathematics?'', co-written with Herbert Robbins. His research focused on the areas of real ...
, both
Lord Rayleigh
John William Strutt, 3rd Baron Rayleigh, (; 12 November 1842 – 30 June 1919) was an English mathematician and physicist who made extensive contributions to science. He spent all of his academic career at the University of Cambridge. Amo ...
and
Walther Ritz
Walther Heinrich Wilhelm Ritz (22 February 1878 – 7 July 1909) was a Swiss theoretical physicist. He is most famous for his work with Johannes Rydberg on the Rydberg–Ritz combination principle. Ritz is also known for the variational method n ...
independently conceived the idea of utilizing the equivalence between
boundary value problems
Boundary or Boundaries may refer to:
* Border, in political geography
Entertainment
* ''Boundaries'' (2016 film), a 2016 Canadian film
* ''Boundaries'' (2018 film), a 2018 American-Canadian road trip film
*Boundary (cricket), the edge of the pla ...
of
partial differential equations
In mathematics, a partial differential equation (PDE) is an equation which imposes relations between the various partial derivatives of a multivariable function.
The function is often thought of as an "unknown" to be solved for, similarly to ...
on the one hand and problems of the
calculus of variations
The calculus of variations (or Variational Calculus) is a field of mathematical analysis that uses variations, which are small changes in functions
and functionals, to find maxima and minima of functionals: mappings from a set of functions t ...
on the other hand for numerical calculation of the solutions, by substituting for the variational problems simpler approximating extremum problems in which a finite number of parameters need to be determined; see the article
Ritz method
The Ritz method is a direct method to find an approximate solution for boundary value problems. The method is named after Walther Ritz, and is also commonly called the Rayleigh–Ritz method and the Ritz-Galerkin method.
In quantum mechanics, ...
for details. Ironically for the debate, the modern justification of the algorithm drops the
calculus of variations
The calculus of variations (or Variational Calculus) is a field of mathematical analysis that uses variations, which are small changes in functions
and functionals, to find maxima and minima of functionals: mappings from a set of functions t ...
in favor of the simpler and more general approach of
orthogonal projection
In linear algebra and functional analysis, a projection is a linear transformation P from a vector space to itself (an endomorphism) such that P\circ P=P. That is, whenever P is applied twice to any vector, it gives the same result as if it wer ...
as in
Galerkin method
In mathematics, in the area of numerical analysis, Galerkin methods, named after the Russian mathematician Boris Galerkin, convert a continuous operator problem, such as a differential equation, commonly in a weak formulation, to a discrete proble ...
named after
Boris Galerkin
Boris Grigoryevich Galerkin (russian: Бори́с Григо́рьевич Галёркин, surname more accurately romanized as Galyorkin; –12 July 1945) was a Soviet mathematician and an engineer.
Biography
Early days
Galerkin was born on ...
, thus leading also to the
Ritz-Galerkin method naming.
It is used in all applications that involve approximating
eigenvalues and eigenvectors
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
, often under different names. In
quantum mechanics
Quantum mechanics is a fundamental theory in physics that provides a description of the physical properties of nature at the scale of atoms and subatomic particles. It is the foundation of all quantum physics including quantum chemistry, ...
, where a system of particles is described using a
Hamiltonian
Hamiltonian may refer to:
* Hamiltonian mechanics, a function that represents the total energy of a system
* Hamiltonian (quantum mechanics), an operator corresponding to the total energy of that system
** Dyall Hamiltonian, a modified Hamiltonian ...
, the
Ritz method
The Ritz method is a direct method to find an approximate solution for boundary value problems. The method is named after Walther Ritz, and is also commonly called the Rayleigh–Ritz method and the Ritz-Galerkin method.
In quantum mechanics, ...
uses
trial wave functions to approximate the ground state eigenfunction with the lowest energy. In the
finite element method
The finite element method (FEM) is a popular method for numerically solving differential equations arising in engineering and mathematical modeling. Typical problem areas of interest include the traditional fields of structural analysis, heat ...
context, mathematically the same algorithm is commonly called the
Ritz-Galerkin method. The Rayleigh–Ritz method or
Ritz method
The Ritz method is a direct method to find an approximate solution for boundary value problems. The method is named after Walther Ritz, and is also commonly called the Rayleigh–Ritz method and the Ritz-Galerkin method.
In quantum mechanics, ...
terminology is typical in mechanical and structural engineering to approximate the
eigenmodes
A normal mode of a dynamical system is a pattern of motion in which all parts of the system move sinusoidally with the same frequency and with a fixed phase relation. The free motion described by the normal modes takes place at fixed frequencies. ...
and
resonant frequencies
Resonance describes the phenomenon of increased amplitude that occurs when the frequency of an applied periodic force (or a Fourier component of it) is equal or close to a natural frequency of the system on which it acts. When an oscillati ...
of a structure.
For matrix eigenvalue problems
In numerical linear algebra, the Rayleigh–Ritz method is commonly
applied to approximate an eigenvalue problem
for the matrix
of size
using a projected matrix of a smaller size
, generated from a given matrix
with
orthonormal
In linear algebra, two vectors in an inner product space are orthonormal if they are orthogonal (or perpendicular along a line) unit vectors. A set of vectors form an orthonormal set if all vectors in the set are mutually orthogonal and all of un ...
columns. The matrix version of the algorithm is the most simple:
# Compute the
matrix
, where
denotes the complex-conjugate transpose of
# Solve the eigenvalue problem
# Compute the Ritz vectors
and the Ritz value
# Output approximations
, called the Ritz pairs, to
eigenvalues and eigenvectors
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted b ...
of the original matrix
.
If the subspace with the orthonormal basis given by the columns of the matrix
contains
vectors that are close to eigenvectors of the matrix
, the Rayleigh–Ritz method above finds
Ritz vectors that well approximate these eigenvectors. The easily computable quantity
determines the accuracy of such an approximation for every Ritz pair.
In the easiest case
, the
matrix
turns into a unit column-vector
, the
matrix
is a scalar that is equal to the
Rayleigh quotient
In mathematics, the Rayleigh quotient () for a given complex Hermitian matrix ''M'' and nonzero vector ''x'' is defined as:
R(M,x) = .
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the co ...
, the only
solution to the eigenvalue problem is
and
, and the only one Ritz vector is
itself. Thus, the Rayleigh–Ritz method turns into computing of the
Rayleigh quotient
In mathematics, the Rayleigh quotient () for a given complex Hermitian matrix ''M'' and nonzero vector ''x'' is defined as:
R(M,x) = .
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the co ...
if
.
Another useful connection to the
Rayleigh quotient
In mathematics, the Rayleigh quotient () for a given complex Hermitian matrix ''M'' and nonzero vector ''x'' is defined as:
R(M,x) = .
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the co ...
is that
for every Ritz pair
, allowing to derive some properties of Ritz values
from the corresponding theory for the
Rayleigh quotient
In mathematics, the Rayleigh quotient () for a given complex Hermitian matrix ''M'' and nonzero vector ''x'' is defined as:
R(M,x) = .
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the co ...
. For example, if
is a
Hermitian matrix
In mathematics, a Hermitian matrix (or self-adjoint matrix) is a complex square matrix that is equal to its own conjugate transpose—that is, the element in the -th row and -th column is equal to the complex conjugate of the element in the -th ...
, its
Rayleigh quotient
In mathematics, the Rayleigh quotient () for a given complex Hermitian matrix ''M'' and nonzero vector ''x'' is defined as:
R(M,x) = .
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the co ...
(and thus its every Ritz value) is real and takes values within the closed interval of the smallest and largest eigenvalues of
.
Example
The matrix
has eigenvalues
and the corresponding eigenvectors
Let us take
then
with eigenvalues
and the corresponding eigenvectors
so that the Ritz values are
and the Ritz vectors are
We observe that each one of the Ritz vectors is exactly one of the eigenvectors of
for the given
as well as the Ritz values give exactly two of the three eigenvalues of
. A mathematical explanation for the exact approximation is based on the fact that the
column space
In linear algebra, the column space (also called the range or image) of a matrix ''A'' is the span (set of all possible linear combinations) of its column vectors. The column space of a matrix is the image or range of the corresponding matrix ...
of the matrix
happens to be exactly the same as the subspace spanned by the two eigenvectors
and
in this example.
For matrix singular value problems
Truncated singular value decomposition (SVD) in numerical linear algebra can also use the Rayleigh–Ritz method to find approximations to left and right singular vectors of the matrix
of size
in given subspaces by turning the singular value problem into an eigenvalue problem.
Using the normal matrix
The definition of the singular value
and the corresponding left and right singular vectors is
and
. Having found one set (left of right) of approximate singular vectors and singular values by applying naively the Rayleigh–Ritz method to the
Hermitian {{Short description, none
Numerous things are named after the French mathematician Charles Hermite (1822–1901):
Hermite
* Cubic Hermite spline, a type of third-degree spline
* Gauss–Hermite quadrature, an extension of Gaussian quadrature m ...
normal matrix
or
, whichever one is smaller size, one could determine the other set of left of right singular vectors simply by dividing by the singular values, i.e.,
and
. However, the division is unstable or fails for small or zero singular values.
An alternative approach, e.g., defining the normal matrix as
of size
, takes advantage of the fact that for a given
matrix
with
orthonormal
In linear algebra, two vectors in an inner product space are orthonormal if they are orthogonal (or perpendicular along a line) unit vectors. A set of vectors form an orthonormal set if all vectors in the set are mutually orthogonal and all of un ...
columns the eigenvalue problem of the Rayleigh–Ritz method for the
matrix
can be interpreted as a singular value problem for the
matrix
. This interpretation allows simple simultaneous calculation of both left and right approximate singular vectors as follows.
# Compute the
matrix
.
# Compute the
thin, or economy-sized, SVD with
matrix
,
diagonal matrix
, and
matrix
.
# Compute the matrices of the Ritz left
and right
singular vectors.
# Output approximations
, called the Ritz singular triplets, to selected singular values and the corresponding left and right singular vectors of the original matrix
representing an approximate
Truncated singular value decomposition (SVD) with left singular vectors restricted to the column-space of the matrix
.
The algorithm can be used as a post-processing step where the matrix
is an output of an eigenvalue solver, e.g., such as
LOBPCG Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) is a matrix-free method for finding the largest (or smallest) eigenvalues and the corresponding eigenvectors of a symmetric generalized eigenvalue problem
:A x= \lambda B x,
for a ...
, approximating numerically selected eigenvectors of the normal matrix
.
Example
The matrix
has its normal matrix
singular values
and the corresponding
thin SVD
where the columns of the first multiplier from the complete set of the left singular vectors of the matrix
, the diagonal entries of the middle term are the singular values, and the columns of the last multiplier transposed (although the transposition does not change it)
are the corresponding right singular vectors.
Let us take
with the column-space that is spanned by the two exact right singular vectors
corresponding to the singular values 1 and 2.
Following the algorithm step 1, we compute
and on step 2 its
thin SVD with
Thus we already obtain the singular values 2 and 1 from
and from
the corresponding two left singular vectors
as
and
, which span the column-space of the matrix
, explaining why the approximations are exact for the given
.
Finally, step 3 computes the matrix
recovering from its rows the two right singular vectors
as
and
.
We validate the first vector:
and
Thus, for the given matrix
with its column-space that is spanned by two exact right singular vectors, we determine these right singular vectors, as well as the corresponding left singular vectors and the singular values, all exactly. For an arbitrary matrix
, we obtain approximate singular triplets which are optimal given
in the sense of optimality of the Rayleigh–Ritz method.
See also
*
Ritz method
The Ritz method is a direct method to find an approximate solution for boundary value problems. The method is named after Walther Ritz, and is also commonly called the Rayleigh–Ritz method and the Ritz-Galerkin method.
In quantum mechanics, ...
*
Rayleigh quotient
In mathematics, the Rayleigh quotient () for a given complex Hermitian matrix ''M'' and nonzero vector ''x'' is defined as:
R(M,x) = .
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the co ...
*
Arnoldi iteration In numerical linear algebra, the Arnoldi iteration is an eigenvalue algorithm and an important example of an iterative method. Arnoldi finds an approximation to the eigenvalues and eigenvectors of general (possibly non-Hermitian) matrices by const ...
Notes and references
External links
Course on Calculus of Variations, has a section on Rayleigh–Ritz method
{{DEFAULTSORT:Rayleigh-Ritz Method
Numerical differential equations
Dynamical systems