The concept of
angle
In Euclidean geometry, an angle is the figure formed by two rays, called the '' sides'' of the angle, sharing a common endpoint, called the '' vertex'' of the angle.
Angles formed by two rays lie in the plane that contains the rays. Angles a ...
s between
lines in the
plane
Plane(s) most often refers to:
* Aero- or airplane, a powered, fixed-wing aircraft
* Plane (geometry), a flat, 2-dimensional surface
Plane or planes may also refer to:
Biology
* Plane (tree) or ''Platanus'', wetland native plant
* ''Planes' ...
and between pairs of two lines, two planes or a line and a plane in
space
Space is the boundless three-dimensional extent in which objects and events have relative position and direction. In classical physics, physical space is often conceived in three linear dimensions, although modern physicists usually cons ...
can be generalized to arbitrary
dimension
In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coor ...
. This generalization was first discussed by
Jordan
Jordan ( ar, الأردن; tr. ' ), officially the Hashemite Kingdom of Jordan,; tr. ' is a country in Western Asia. It is situated at the crossroads of Asia, Africa, and Europe, within the Levant region, on the East Bank of the Jordan Rive ...
.
For any pair of
flats
Flat or flats may refer to:
Architecture
* Flat (housing), an apartment in the United Kingdom, Ireland, Australia and other Commonwealth countries
Arts and entertainment
* Flat (music), a symbol () which denotes a lower pitch
* Flat (soldier), ...
in a
Euclidean space
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean ...
of arbitrary dimension one can define a set of mutual angles which are
invariant under
isometric transformation of the Euclidean space. If the flats do not intersect, their shortest
distance
Distance is a numerical or occasionally qualitative measurement of how far apart objects or points are. In physics or everyday usage, distance may refer to a physical length or an estimation based on other criteria (e.g. "two counties over"). ...
is one more invariant.
These angles are called canonical
or principal.
The concept of angles can be generalized to pairs of flats in a finite-dimensional
inner product space
In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
over the
complex number
In mathematics, a complex number is an element of a number system that extends the real numbers with a specific element denoted , called the imaginary unit and satisfying the equation i^= -1; every complex number can be expressed in the fo ...
s.
Jordan's definition
Let
and
be flats of dimensions
and
in the
-dimensional Euclidean space
. By definition, a
translation
Translation is the communication of the meaning of a source-language text by means of an equivalent target-language text. The English language draws a terminological distinction (which does not exist in every language) between ''transla ...
of
or
does not alter their mutual angles. If
and
do not intersect, they will do so upon any translation of
which maps some point in
to some point in
. It can therefore be assumed without loss of generality that
and
intersect.
Jordan shows that
Cartesian coordinates in
can then be defined such that
and
are described, respectively, by the sets of equations
:
:
:
and
:
:
:
with
. Jordan calls these coordinates canonical. By definition, the angles
are the angles between
and
.
The non-negative integers
are constrained by
:
:
:
For these equations to determine the five non-negative integers completely, besides the dimensions
and
and the number
of angles
, the non-negative integer
must be given. This is the number of coordinates
, whose corresponding axes are those lying entirely within both
and
. The integer
is thus the dimension of
. The set of angles
may be supplemented with
angles
to indicate that
has that dimension.
Jordan's proof applies essentially unaltered when
is replaced with the
-dimensional inner product space
over the complex numbers. (For
angles between subspaces, the generalization to
is discussed by Galántai and Hegedũs in terms of the below
variational characterization.
)
Angles between subspaces
Now let
and
be
subspaces of the
-dimensional inner product space over the
real
Real may refer to:
Currencies
* Brazilian real (R$)
* Central American Republic real
* Mexican real
* Portuguese real
* Spanish real
* Spanish colonial real
Music Albums
* ''Real'' (L'Arc-en-Ciel album) (2000)
* ''Real'' (Bright album) (2010) ...
or complex numbers. Geometrically,
and
are flats, so Jordan's definition of mutual angles applies. When for any canonical coordinate
the symbol
denotes the
unit vector
In mathematics, a unit vector in a normed vector space is a vector (often a spatial vector) of length 1. A unit vector is often denoted by a lowercase letter with a circumflex, or "hat", as in \hat (pronounced "v-hat").
The term ''direction v ...
of the
axis, the vectors
form an
orthonormal
In linear algebra, two vectors in an inner product space are orthonormal if they are orthogonal (or perpendicular along a line) unit vectors. A set of vectors form an orthonormal set if all vectors in the set are mutually orthogonal and all of un ...
basis
Basis may refer to:
Finance and accounting
* Adjusted basis, the net cost of an asset after adjusting for various tax-related items
*Basis point, 0.01%, often used in the context of interest rates
* Basis trading, a trading strategy consisting ...
for
and the vectors
form an orthonormal basis for
, where
:
Being related to canonical coordinates, these basic vectors may be called canonical.
When
denote the canonical basic vectors for
and
the canonical basic vectors for
then the inner product
vanishes for any pair of
and
except the following ones.
:
With the above ordering of the basic vectors, the
matrix
Matrix most commonly refers to:
* ''The Matrix'' (franchise), an American media franchise
** ''The Matrix'', a 1999 science-fiction action film
** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
of the inner products
is thus
diagonal
In geometry, a diagonal is a line segment joining two vertices of a polygon or polyhedron, when those vertices are not on the same edge. Informally, any sloping line is called diagonal. The word ''diagonal'' derives from the ancient Greek δΠ...
. In other words, if
and
are arbitrary orthonormal bases in
and
then the
real, orthogonal or
unitary
Unitary may refer to:
Mathematics
* Unitary divisor
* Unitary element
* Unitary group
* Unitary matrix
* Unitary morphism
* Unitary operator
* Unitary transformation
* Unitary representation
* Unitarity (physics)
* ''E''-unitary inverse semigrou ...
transformations from the basis
to the basis
and from the basis
to the basis
realize a
singular value decomposition
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any \ m \times n\ matrix. It is re ...
of the matrix of inner products
. The diagonal matrix elements
are the singular values of the latter matrix. By the uniqueness of the singular value decomposition, the vectors
are then unique up to a real, orthogonal or unitary transformation among them, and the vectors
and
(and hence
) are unique up to equal real, orthogonal or unitary transformations applied simultaneously to the sets of the vectors
associated with a common value of
and to the corresponding sets of vectors
(and hence to the corresponding sets of
).
A singular value
can be interpreted as
corresponding to the angles
introduced above and associated with
and a singular value
can be interpreted as
corresponding to right angles between the
orthogonal spaces
and
, where superscript
denotes the
orthogonal complement In the mathematical fields of linear algebra and functional analysis, the orthogonal complement of a subspace ''W'' of a vector space ''V'' equipped with a bilinear form ''B'' is the set ''W''⊥ of all vectors in ''V'' that are orthogonal to every ...
.
Variational characterization
The
variational characterization of singular values and vectors implies as a special case a variational characterization of the angles between subspaces and their associated canonical vectors. This characterization includes the angles
and
introduced above and orders the angles by increasing value. It can be given the form of the below alternative definition. In this context, it is customary to talk of principal angles and vectors.
Definition
Let
be an inner product space. Given two subspaces
with
, there exists then a sequence of
angles
called the principal angles, the first one defined as
:
where
is the
inner product
In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
and
the induced
norm
Naturally occurring radioactive materials (NORM) and technologically enhanced naturally occurring radioactive materials (TENORM) consist of materials, usually industrial wastes or by-products enriched with radioactive elements found in the envi ...
. The vectors
and
are the corresponding ''principal vectors.''
The other principal angles and vectors are then defined recursively via
:
This means that the principal angles
form a set of minimized angles between the two subspaces, and the principal vectors in each subspace are orthogonal to each other.
Examples
Geometric example
Geometrically, subspaces are
flats
Flat or flats may refer to:
Architecture
* Flat (housing), an apartment in the United Kingdom, Ireland, Australia and other Commonwealth countries
Arts and entertainment
* Flat (music), a symbol () which denotes a lower pitch
* Flat (soldier), ...
(points, lines, planes etc.) that include the origin, thus any two subspaces intersect at least in the origin. Two two-dimensional subspaces
and
generate a set of two angles. In a three-dimensional
Euclidean space
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean ...
, the subspaces
and
are either identical, or their intersection forms a line. In the former case, both
. In the latter case, only
, where vectors
and
are on the line of the intersection
and have the same direction. The angle
will be the angle between the subspaces
and
in the
orthogonal complement In the mathematical fields of linear algebra and functional analysis, the orthogonal complement of a subspace ''W'' of a vector space ''V'' equipped with a bilinear form ''B'' is the set ''W''⊥ of all vectors in ''V'' that are orthogonal to every ...
to
. Imagining the angle between two planes in 3D, one intuitively thinks of the largest angle,
.
Algebraic example
In 4-dimensional real coordinate space R
4, let the two-dimensional subspace
be
spanned by
and
, and let the two-dimensional subspace
be
spanned by
and
with some real
and
such that
. Then
and
are, in fact, the pair of principal vectors corresponding to the angle
with
, and
and
are the principal vectors corresponding to the angle
with
To construct a pair of subspaces with any given set of
angles
in a
(or larger) dimensional
Euclidean space
Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean ...
, take a subspace
with an orthonormal basis
and complete it to an orthonormal basis
of the Euclidean space, where
. Then, an orthonormal basis of the other subspace
is, e.g.,
:
Basic properties
* If the largest angle is zero, one subspace is a subset of the other.
* If the largest angle is
, there is at least one vector in one subspace perpendicular to the other subspace.
* If the smallest angle is zero, the subspaces intersect at least in a line.
* If the smallest angle is
, the subspaces are orthogonal.
* The number of angles equal to zero is the dimension of the space where the two subspaces intersect.
Advanced properties
* Non-trivial (different from
and
) angles between two subspaces are the same as the non-trivial angles between their orthogonal complements.
* Non-trivial angles between the subspaces
and
and the corresponding non-trivial angles between the subspaces
and
sum up to
.
* The angles between subspaces satisfy the
triangle inequality
In mathematics, the triangle inequality states that for any triangle, the sum of the lengths of any two sides must be greater than or equal to the length of the remaining side.
This statement permits the inclusion of degenerate triangles, but ...
in terms of
majorization In mathematics, majorization is a preorder on vectors of real numbers. Let ^_,\ i=1,\,\ldots,\,n denote the i-th largest element of the vector \mathbf\in\mathbb^n. Given \mathbf,\ \mathbf \in \mathbb^n, we say that \mathbf weakly majorizes (or ...
and thus can be used to define a
distance
Distance is a numerical or occasionally qualitative measurement of how far apart objects or points are. In physics or everyday usage, distance may refer to a physical length or an estimation based on other criteria (e.g. "two counties over"). ...
on the set of all subspaces turning the set into a
metric space
In mathematics, a metric space is a set together with a notion of '' distance'' between its elements, usually called points. The distance is measured by a function called a metric or distance function. Metric spaces are the most general set ...
.
* The
sine of the angles between subspaces satisfy the
triangle inequality
In mathematics, the triangle inequality states that for any triangle, the sum of the lengths of any two sides must be greater than or equal to the length of the remaining side.
This statement permits the inclusion of degenerate triangles, but ...
in terms of
majorization In mathematics, majorization is a preorder on vectors of real numbers. Let ^_,\ i=1,\,\ldots,\,n denote the i-th largest element of the vector \mathbf\in\mathbb^n. Given \mathbf,\ \mathbf \in \mathbb^n, we say that \mathbf weakly majorizes (or ...
and thus can be used to define a
distance
Distance is a numerical or occasionally qualitative measurement of how far apart objects or points are. In physics or everyday usage, distance may refer to a physical length or an estimation based on other criteria (e.g. "two counties over"). ...
on the set of all subspaces turning the set into a
metric space
In mathematics, a metric space is a set together with a notion of '' distance'' between its elements, usually called points. The distance is measured by a function called a metric or distance function. Metric spaces are the most general set ...
.
For example, the
sine of the largest angle is known as a
gap between subspaces.
Extensions
The notion of the angles and some of the variational properties can be naturally extended to arbitrary
inner products and subspaces with infinite
dimensions
In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coordin ...
.
Computation
Historically, the principal angles and vectors first appear in the context of
canonical correlation
In statistics, canonical-correlation analysis (CCA), also called canonical variates analysis, is a way of inferring information from cross-covariance matrices. If we have two vectors ''X'' = (''X''1, ..., ''X'n'') and ''Y' ...
and were
originally computed using
SVD
''Svenska Dagbladet'' (, "The Swedish Daily News"), abbreviated SvD, is a daily newspaper published in Stockholm, Sweden.
History and profile
The first issue of ''Svenska Dagbladet'' appeared on 18 December 1884. During the beginning of the ...
of corresponding
covariance
In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the ...
matrices. However, as first noticed in,
the
canonical correlation
In statistics, canonical-correlation analysis (CCA), also called canonical variates analysis, is a way of inferring information from cross-covariance matrices. If we have two vectors ''X'' = (''X''1, ..., ''X'n'') and ''Y' ...
is related to the
cosine of the principal angles, which is
ill-conditioned
In numerical analysis, the condition number of a function measures how much the output value of the function can change for a small change in the input argument. This is used to measure how sensitive a function is to changes or errors in the input ...
for small angles, leading to very inaccurate computation of highly correlated principal vectors in finite
precision
Precision, precise or precisely may refer to:
Science, and technology, and mathematics Mathematics and computing (general)
* Accuracy and precision, measurement deviation from true value and its scatter
* Significant figures, the number of digit ...
computer arithmetic
In computing, an arithmetic logic unit (ALU) is a combinational digital circuit that performs arithmetic and bitwise operations on integer binary numbers. This is in contrast to a floating-point unit (FPU), which operates on floating point numb ...
. The
sine-based algorithm
fixes this issue, but creates a new problem of very inaccurate computation of highly uncorrelated principal vectors, since the
sine function is
ill-conditioned
In numerical analysis, the condition number of a function measures how much the output value of the function can change for a small change in the input argument. This is used to measure how sensitive a function is to changes or errors in the input ...
for angles close to /2. To produce accurate principal vectors in
computer arithmetic
In computing, an arithmetic logic unit (ALU) is a combinational digital circuit that performs arithmetic and bitwise operations on integer binary numbers. This is in contrast to a floating-point unit (FPU), which operates on floating point numb ...
for the full range of the principal angles, the combined technique
first compute all principal angles and vectors using the classical
cosine-based approach, and then recomputes the principal angles smaller than /4 and the corresponding principal vectors using the
sine-based approach.
The combined technique
is implemented in
open-source libraries
Octave and
SciPy
SciPy (pronounced "sigh pie") is a free and open-source Python library used for scientific computing and technical computing.
SciPy contains modules for optimization, linear algebra, integration, interpolation, special functions, FFT, ...
and contributed and
MATLAB FileExchange function subspacea
/ref> to MATLAB
MATLAB (an abbreviation of "MATrix LABoratory") is a proprietary multi-paradigm programming language and numeric computing environment developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementa ...
.
See also
*Singular value decomposition
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any \ m \times n\ matrix. It is re ...
*Canonical correlation
In statistics, canonical-correlation analysis (CCA), also called canonical variates analysis, is a way of inferring information from cross-covariance matrices. If we have two vectors ''X'' = (''X''1, ..., ''X'n'') and ''Y' ...
References
[
]
[{{Citation
, last = Kato
, first =D.T.
, publisher = Springer, New York
, title = Perturbation Theory for Linear Operators
, year = 1996
]
Analytic geometry
Linear algebra