linear algebra Linear algebra is the branch of mathematics concerning linear equations such as :a_1x_1+\cdots +a_nx_n=b, linear maps such as :(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n, and their representations in vector spaces and through matrix (mathemat ...

, orthogonalization is the process of finding a

set Set, The Set, SET or SETS may refer to: Science, technology, and mathematics Mathematics *Set (mathematics), a collection of elements *Category of sets, the category whose objects and morphisms are sets and total functions, respectively Electro ...

of orthogonal vectors that span a particular subspace. Formally, starting with a

linearly independent In the theory of vector spaces, a set of vectors is said to be if there exists no nontrivial linear combination of the vectors that equals the zero vector. If such a linear combination exists, then the vectors are said to be . These concep ...

set of vectors in an

inner product space In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, ofte ...

(most commonly the

Euclidean space Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are ''Euclidean spaces ...

R^''n''), orthogonalization results in a set of

orthogonal In mathematics, orthogonality (mathematics), orthogonality is the generalization of the geometric notion of ''perpendicularity''. Although many authors use the two terms ''perpendicular'' and ''orthogonal'' interchangeably, the term ''perpendic ...

vectors that generate the same subspace as the vectors ''v''₁, ... , ''v''_''k''. Every vector in the new set is orthogonal to every other vector in the new set; and the new set and the old set have the same

linear span In mathematics, the linear span (also called the linear hull or just span) of a set S of elements of a vector space V is the smallest linear subspace of V that contains S. It is the set of all finite linear combinations of the elements of , and ...

. In addition, if we want the resulting vectors to all be

unit vector In mathematics, a unit vector in a normed vector space is a Vector (mathematics and physics), vector (often a vector (geometry), spatial vector) of Norm (mathematics), length 1. A unit vector is often denoted by a lowercase letter with a circumfle ...

s, then we normalize each vector and the procedure is called orthonormalization. Orthogonalization is also possible with respect to any

symmetric bilinear form In mathematics, a symmetric bilinear form on a vector space is a bilinear map from two copies of the vector space to the field of scalars such that the order of the two vectors does not affect the value of the map. In other words, it is a biline ...

(not necessarily an inner product, not necessarily over

real number In mathematics, a real number is a number that can be used to measure a continuous one- dimensional quantity such as a duration or temperature. Here, ''continuous'' means that pairs of values can have arbitrarily small differences. Every re ...

s), but standard algorithms may encounter

division by zero In mathematics, division by zero, division (mathematics), division where the divisor (denominator) is 0, zero, is a unique and problematic special case. Using fraction notation, the general example can be written as \tfrac a0, where a is the di ...

in this more general setting.

Orthogonalization algorithms

Methods for performing orthogonalization include: *

Gram–Schmidt process In mathematics, particularly linear algebra and numerical analysis, the Gram–Schmidt process or Gram-Schmidt algorithm is a way of finding a set of two or more vectors that are perpendicular to each other. By technical definition, it is a metho ...

, which uses

projection Projection or projections may refer to: Physics * Projection (physics), the action/process of light, heat, or sound reflecting from a surface to another in a different direction * The display of images by a projector Optics, graphics, and carto ...

* Householder transformation, which uses reflection * Givens rotation *Symmetric orthogonalization, which uses the

Singular value decomposition In linear algebra, the singular value decomposition (SVD) is a Matrix decomposition, factorization of a real number, real or complex number, complex matrix (mathematics), matrix into a rotation, followed by a rescaling followed by another rota ...

When performing orthogonalization on a computer, the Householder transformation is usually preferred over the Gram–Schmidt process since it is more numerically stable, i.e. rounding errors tend to have less serious effects. On the other hand, the Gram–Schmidt process produces the jth orthogonalized vector after the jth iteration, while orthogonalization using Householder reflections produces all the vectors only at the end. This makes only the Gram–Schmidt process applicable for

iterative method In computational mathematics, an iterative method is a Algorithm, mathematical procedure that uses an initial value to generate a sequence of improving approximate solutions for a class of problems, in which the ''i''-th approximation (called an " ...

s like the Arnoldi iteration. The Givens rotation is more easily parallelized than Householder transformations. Symmetric orthogonalization was formulated by Per-Olov Löwdin.

Local orthogonalization

To compensate for the loss of useful signal in traditional noise attenuation approaches because of incorrect parameter selection or inadequacy of denoising assumptions, a weighting operator can be applied on the initially denoised section for the retrieval of useful signal from the initial noise section. The new denoising process is referred to as the local orthogonalization of signal and noise. It has a wide range of applications in many signals processing and seismic exploration fields.

References

Linear algebra

Orthogonalization algorithms

Local orthogonalization

See also

References