HOME

TheInfoList



OR:

In
mathematics Mathematics is a field of study that discovers and organizes methods, Mathematical theory, theories and theorems that are developed and Mathematical proof, proved for the needs of empirical sciences and mathematics itself. There are many ar ...
, the operator norm measures the "size" of certain
linear operator In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that pr ...
s by assigning each a
real number In mathematics, a real number is a number that can be used to measure a continuous one- dimensional quantity such as a duration or temperature. Here, ''continuous'' means that pairs of values can have arbitrarily small differences. Every re ...
called its . Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces. Informally, the operator norm \, T\, of a linear map T : X \to Y is the maximum factor by which it "lengthens" vectors.


Introduction and definition

Given two normed vector spaces V and W (over the same base field, either the
real number In mathematics, a real number is a number that can be used to measure a continuous one- dimensional quantity such as a duration or temperature. Here, ''continuous'' means that pairs of values can have arbitrarily small differences. Every re ...
s \R or the
complex number In mathematics, a complex number is an element of a number system that extends the real numbers with a specific element denoted , called the imaginary unit and satisfying the equation i^= -1; every complex number can be expressed in the for ...
s \Complex), a
linear map In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that p ...
A : V \to W is continuous
if and only if In logic and related fields such as mathematics and philosophy, "if and only if" (often shortened as "iff") is paraphrased by the biconditional, a logical connective between statements. The biconditional is true in two cases, where either bo ...
there exists a real number c such that \, Av\, \leq c \, v\, \quad \text v\in V. The norm on the left is the one in W and the norm on the right is the one in V. Intuitively, the continuous operator A never increases the length of any vector by more than a factor of c. Thus the
image An image or picture is a visual representation. An image can be Two-dimensional space, two-dimensional, such as a drawing, painting, or photograph, or Three-dimensional space, three-dimensional, such as a carving or sculpture. Images may be di ...
of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as bounded operators. In order to "measure the size" of A, one can take the
infimum In mathematics, the infimum (abbreviated inf; : infima) of a subset S of a partially ordered set P is the greatest element in P that is less than or equal to each element of S, if such an element exists. If the infimum of S exists, it is unique ...
of the numbers c such that the above inequality holds for all v \in V. This number represents the maximum scalar factor by which A "lengthens" vectors. In other words, the "size" of A is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of A as \, A\, _\text = \inf\. The infimum is attained as the set of all such c is closed, nonempty, and bounded from below. It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces V and W.


Examples

Every real m-by-n
matrix Matrix (: matrices or matrixes) or MATRIX may refer to: Science and mathematics * Matrix (mathematics), a rectangular array of numbers, symbols or expressions * Matrix (logic), part of a formula in prenex normal form * Matrix (biology), the m ...
corresponds to a linear map from \R^n to \R^m. Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all m-by-n matrices of real numbers; these induced norms form a subset of
matrix norm In the field of mathematics, norms are defined for elements within a vector space. Specifically, when the vector space comprises matrices, such norms are referred to as matrix norms. Matrix norms differ from vector norms in that they must also ...
s. If we specifically choose the
Euclidean norm Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are ''Euclidean spaces'' ...
on both \R^n and \R^m, then the matrix norm given to a matrix A is the
square root In mathematics, a square root of a number is a number such that y^2 = x; in other words, a number whose ''square'' (the result of multiplying the number by itself, or y \cdot y) is . For example, 4 and −4 are square roots of 16 because 4 ...
of the largest
eigenvalue In linear algebra, an eigenvector ( ) or characteristic vector is a vector that has its direction unchanged (or reversed) by a given linear transformation. More precisely, an eigenvector \mathbf v of a linear transformation T is scaled by a ...
of the matrix A^ A (where A^ denotes the conjugate transpose of A). This is equivalent to assigning the largest singular value of A. Passing to a typical infinite-dimensional example, consider the
sequence space In functional analysis and related areas of mathematics, a sequence space is a vector space whose elements are infinite sequences of real or complex numbers. Equivalently, it is a function space whose elements are functions from the natural num ...
\ell^2, which is an L''p'' space, defined by \ell^2 = \left\. This can be viewed as an infinite-dimensional analogue of the
Euclidean space Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are ''Euclidean spaces ...
\Complex^n. Now consider a bounded sequence s_ = \left(s_n\right)_^\infty. The sequence s_ is an element of the space \ell^\infty, with a norm given by \left\, s_\right\, _\infty = \sup _n \left, s_n\. Define an operator T_s by pointwise multiplication: \left(a_n\right)_^ \;\stackrel\;\ \left(s_n \cdot a_n\right)_^. The operator T_s is bounded with operator norm \left\, T_s\right\, _\text = \left\, s_\right\, _\infty. This discussion extends directly to the case where \ell^2 is replaced by a general L^p space with p > 1 and \ell^\infty replaced by L^\infty.


Equivalent definitions

Let A : V \to W be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition V \neq \ then they are all equivalent: : \begin \, A\, _\text &= \inf &&\ \\ &= \sup &&\ \\ &= \sup &&\ \\ &= \sup &&\ \\ &= \sup &&\ \;\;\;\text V \neq \ \\ &= \sup &&\bigg\ \;\;\;\text V \neq \. \\ \end If V = \ then the sets in the last two rows will be empty, and consequently their
supremum In mathematics, the infimum (abbreviated inf; : infima) of a subset S of a partially ordered set P is the greatest element in P that is less than or equal to each element of S, if such an element exists. If the infimum of S exists, it is unique, ...
s over the set \infty, \infty/math> will equal -\infty instead of the correct value of 0. If the supremum is taken over the set , \infty/math> instead, then the supremum of the empty set is 0 and the formulas hold for any V. Importantly, a linear operator A : V \to W is not, in general, guaranteed to achieve its norm \, A\, _\text = \sup \ on the closed unit ball \, meaning that there might not exist any vector u \in V of norm \, u\, \leq 1 such that \, A\, _\text = \, A u\, (if such a vector does exist and if A \neq 0, then u would necessarily have unit norm \, u\, = 1). R.C. James proved
James's theorem In mathematics, particularly functional analysis Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure (for example, Inner product s ...
in 1964, which states that a
Banach space In mathematics, more specifically in functional analysis, a Banach space (, ) is a complete normed vector space. Thus, a Banach space is a vector space with a metric that allows the computation of vector length and distance between vectors and ...
V is reflexive if and only if every bounded linear functional f \in V^* achieves its norm on the closed unit ball. It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball. If A : V \to W is bounded then \, A\, _\text = \sup \left\ and \, A\, _\text = \left\, ^tA\right\, _\text where ^t A : W^* \to V^* is the
transpose In linear algebra, the transpose of a Matrix (mathematics), matrix is an operator which flips a matrix over its diagonal; that is, it switches the row and column indices of the matrix by producing another matrix, often denoted by (among other ...
of A : V \to W, which is the linear operator defined by w^* \,\mapsto\, w^* \circ A.


Properties

The operator norm is indeed a norm on the space of all bounded operators between V and W. This means \, A\, _\text \geq 0 \mbox \, A\, _\text = 0 \mbox A = 0, \, aA\, _\text = , a, \, A\, _\text \mbox a , \, A + B\, _\text \leq \, A\, _\text + \, B\, _\text. The following inequality is an immediate consequence of the definition: \, Av\, \leq \, A\, _\text \, v\, \ \mbox\ v \in V. The operator norm is also compatible with the composition, or multiplication, of operators: if V, W and X are three normed spaces over the same base field, and A : V \to W and B : W \to X are two bounded operators, then it is a sub-multiplicative norm, that is: \, BA\, _\text \leq \, B\, _\text \, A\, _\text. For bounded operators on V, this implies that operator multiplication is jointly continuous. It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.


Table of common operator norms

By choosing different norms for the codomain, used in computing \, Av\, , and the domain, used in computing \, v\, , we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are
NP-hard In computational complexity theory, a computational problem ''H'' is called NP-hard if, for every problem ''L'' which can be solved in non-deterministic polynomial-time, there is a polynomial-time reduction from ''L'' to ''H''. That is, assumi ...
. Except for the NP-hard norms, all these norms can be calculated in N^2 operations (for an N \times N matrix), with the exception of the \ell_2 - \ell_2 norm (which requires N^3 operations for the exact answer, or fewer if you approximate it with the power method or Lanczos iterations). The norm of the adjoint or transpose can be computed as follows. We have that for any p, q, then \, A\, _ = \, A^*\, _ where p', q' are Hölder conjugate to p, q, that is, 1/p + 1/p' = 1 and 1/q + 1/q' = 1.


Operators on a Hilbert space

Suppose H is a real or complex
Hilbert space In mathematics, a Hilbert space is a real number, real or complex number, complex inner product space that is also a complete metric space with respect to the metric induced by the inner product. It generalizes the notion of Euclidean space. The ...
. If A : H \to H is a bounded linear operator, then we have \, A\, _\text = \left\, A^*\right\, _\text and \left\, A^* A\right\, _\text = \, A\, _\text^2, where A^ denotes the adjoint operator of A (which in
Euclidean space Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are ''Euclidean spaces ...
s with the standard
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, ofte ...
corresponds to the conjugate transpose of the matrix A). In general, the
spectral radius ''Spectral'' is a 2016 Hungarian-American military science fiction action film co-written and directed by Nic Mathieu. Written with Ian Fried (screenwriter), Ian Fried & George Nolfi, the film stars James Badge Dale as DARPA research scientist Ma ...
of A is bounded above by the operator norm of A: \rho(A) \leq \, A\, _\text. To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator A has spectrum \. So \rho(A) = 0 while \, A\, _\text > 0. However, when a matrix N is normal, its Jordan canonical form is diagonal (up to unitary equivalence); this is the
spectral theorem In linear algebra and functional analysis, a spectral theorem is a result about when a linear operator or matrix can be diagonalized (that is, represented as a diagonal matrix in some basis). This is extremely useful because computations involvin ...
. In that case it is easy to see that \rho(N) = \, N\, _\text. This formula can sometimes be used to compute the operator norm of a given bounded operator A: define the Hermitian operator B = A^ A, determine its spectral radius, and take the
square root In mathematics, a square root of a number is a number such that y^2 = x; in other words, a number whose ''square'' (the result of multiplying the number by itself, or y \cdot y) is . For example, 4 and −4 are square roots of 16 because 4 ...
to obtain the operator norm of A. The space of bounded operators on H, with the
topology Topology (from the Greek language, Greek words , and ) is the branch of mathematics concerned with the properties of a Mathematical object, geometric object that are preserved under Continuous function, continuous Deformation theory, deformat ...
induced by operator norm, is not separable. For example, consider the
Lp space In mathematics, the spaces are function spaces defined using a natural generalization of the -norm for finite-dimensional vector spaces. They are sometimes called Lebesgue spaces, named after Henri Lebesgue , although according to the Bourba ...
L^2
, 1 The comma is a punctuation mark that appears in several variants in different languages. Some typefaces render it as a small line, slightly curved or straight, but inclined from the vertical; others give it the appearance of a miniature fille ...
which is a Hilbert space. For 0 < t \leq 1, let \Omega_t be the characteristic function of , t and P_t be the multiplication operator given by \Omega_t, that is, P_t (f) = f \cdot \Omega_t. Then each P_t is a bounded operator with operator norm 1 and \left\, P_t - P_s\right\, _\text = 1 \quad \mbox \quad t \neq s. But \ is an
uncountable set In mathematics, an uncountable set, informally, is an infinite set that contains too many elements to be countable. The uncountability of a set is closely related to its cardinal number: a set is uncountable if its cardinal number is larger t ...
. This implies the space of bounded operators on L^2(
, 1 The comma is a punctuation mark that appears in several variants in different languages. Some typefaces render it as a small line, slightly curved or straight, but inclined from the vertical; others give it the appearance of a miniature fille ...
is not separable, in operator norm. One can compare this with the fact that the sequence space \ell^ is not separable. The
associative algebra In mathematics, an associative algebra ''A'' over a commutative ring (often a field) ''K'' is a ring ''A'' together with a ring homomorphism from ''K'' into the center of ''A''. This is thus an algebraic structure with an addition, a mult ...
of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a
C*-algebra In mathematics, specifically in functional analysis, a C∗-algebra (pronounced "C-star") is a Banach algebra together with an involution satisfying the properties of the adjoint. A particular case is that of a complex algebra ''A'' of contin ...
.


See also

* * * * * * * * * * * *


Notes


References

* . * * * {{Duality and spaces of linear maps Functional analysis Norms (mathematics) Operator theory