In
image analysis, the generalized structure tensor (GST) is an extension of the Cartesian
structure tensor
In mathematics, the structure tensor, also referred to as the second-moment matrix, is a matrix derived from the gradient of a function. It describes the distribution of the gradient in a specified neighborhood around a point and makes the inf ...
to
curvilinear coordinates
In geometry, curvilinear coordinates are a coordinate system for Euclidean space in which the coordinate lines may be curved. These coordinates may be derived from a set of Cartesian coordinates by using a transformation that is locally inve ...
.
It is mainly used to detect and to represent the "direction" parameters of curves, just as the Cartesian structure tensor detects and represents the direction in Cartesian coordinates. Curve families generated by pairs of locally orthogonal functions have been the best studied.
It is a widely known method in applications of image and video processing including computer vision, such as biometric identification by fingerprints,
and studies of human tissue sections.
GST in 2D and locally orthogonal bases
Let the term image represent a function
where
are real variables and
, and
, are real valued functions. GST represents the direction along which the image
can undergo an infinitesimal translation with minimal (
total least squares) error, along the "lines" fulfilling the following conditions:
1. The "lines" are ordinary lines in the curvilinear coordinate basis
:
which are curves in Cartesian coordinates as depicted by the equation above. The error is measured in the
sense and the minimality of the error refers thereby to
L2 norm
In mathematics, a norm is a function from a real or complex vector space to the non-negative real numbers that behaves in certain ways like the distance from the origin: it commutes with scaling, obeys a form of the triangle inequality, and is z ...
.
2. The functions
constitute a harmonic pair, i.e. they fulfill
Cauchy–Riemann equations
In the field of complex analysis in mathematics, the Cauchy–Riemann equations, named after Augustin Cauchy and Bernhard Riemann, consist of a system of two partial differential equations which, together with certain continuity and differenti ...
,
:
Accordingly, such curvilinear coordinates
are locally orthogonal.
Then GST consists in
:
where
are errors of (infinitesimal) translation in the best direction (designated by the angle
) and the worst direction (designated by
). The function
is the window function defining the "outer scale" wherein the detection of
will be carried out, which can be omitted if it is already included in
or if
is the full image (rather than local). The matrix
is the
identity matrix. Using the
chain rule
In calculus, the chain rule is a formula that expresses the derivative of the composition of two differentiable functions and in terms of the derivatives of and . More precisely, if h=f\circ g is the function such that h(x)=f(g(x)) for every , ...
, it can be shown that the integration above can be implemented as
convolutions in Cartesian coordinates applied to the ordinary structure tensor when
pair the real and imaginary parts of an
analytic function ,
:
where
.
Examples of analytic functions include
, as well as monomials
,
, where
is an arbitrary positive or negative integer. The monomials
are also referred to as
harmonic functions in computer vision, and image processing.
Thereby, Cartesian
Structure tensor
In mathematics, the structure tensor, also referred to as the second-moment matrix, is a matrix derived from the gradient of a function. It describes the distribution of the gradient in a specified neighborhood around a point and makes the inf ...
is a special case of GST where
, and
, i.e. the harmonic function is simply
. Thus by choosing a harmonic function
, one can detect all curves that are linear combinations of its real and imaginary parts by convolutions on (rectangular) image grids only, even if
are non-Cartesian. Furthermore, the convolution computations can be done by using complex filters applied to the complex version of the structure tensor. Thus, GST implementations have frequently been done using complex version of the structure tensor, rather than using the (1,1) tensor.
Complex version of GST
As there is a complex version of the ordinary
structure tensor
In mathematics, the structure tensor, also referred to as the second-moment matrix, is a matrix derived from the gradient of a function. It describes the distribution of the gradient in a specified neighborhood around a point and makes the inf ...
, there is also a complex version of the GST
:
which is identical to its cousin with the difference that
is a complex filter. It should be recalled that, the ordinary structure tensor
is a real filter, usually defined by a sampled and scaled Gaussian to delineate the neighborhood, also known as the outer scale. This simplicity is a reason for why GST implementations have predominantly used the complex version above. For curve families
defined by analytic functions
, it can be shown that,
the neighborhood defining function is complex valued,
:
,
a so called symmetry derivative of a Gaussian. Thus, the orientation wise variation of the pattern to be looked for is directly incorporated into the neighborhood defining function, and the detection occurs in the space of the (ordinary) structure tensor.
Basic concept for its use in image processing and computer vision
Efficient detection of
in images is possible by image processing for a pair
,
. Complex convolutions (or the corresponding matrix operations) and point-wise non-linear mappings are the basic computational elements of GST implementations. A total least square error estimation of
is then obtained along with the two errors,
and
. In analogy with the Cartesian
structure tensor
In mathematics, the structure tensor, also referred to as the second-moment matrix, is a matrix derived from the gradient of a function. It describes the distribution of the gradient in a specified neighborhood around a point and makes the inf ...
, the estimated angle is in double angle representation, i.e.
is delivered by computations, and can be used as a shape feature whereas
alone or in combination with
can be used as a quality (confidence, certainty) measure for the angle estimation.
Logarithmic spirals, including circles, can for instance be detected by (complex) convolutions and non-linear mappings.
The spirals can be in
gray (valued) images or in a
binary image, i.e. locations of edge elements of the concerned patterns, such as contours of circles or spirals, must not be known or marked otherwise.
Generalized structure tensor can be used as an alternative to
Hough transform in
image processing and
computer vision to detect patterns whose local orientations can be modelled, for example junction points. The main differences comprise:
*Negative, as well as complex voting are allowed;
*With one template multiple patterns belonging to the same family can be detected;
*Image binarization is not required.
Physical and mathematical interpretation
The curvilinear coordinates of GST can explain physical processes applied to images. A well known pair of processes consist in rotation, and zooming. These are related to the coordinate transformation
and
.
If an image
consists in iso-curves that can be explained by only
i.e. its iso-curves consist in circles
, where
is any real valued differentiable function defined on 1D, the image is invariant to rotations (around the origin).
Zooming (comprising unzooming) operation is modeled similarly. If the image has iso-curves that look like a "star" or bicycle spokes, i.e.
for some differentiable 1D function
then, the image
is invariant to scaling (w.r.t. the origin).
In combination,
:
is invariant to a certain amount of rotation combined with scaling, where the amount is precised by the parameter
.
Analogously, the Cartesian
structure tensor
In mathematics, the structure tensor, also referred to as the second-moment matrix, is a matrix derived from the gradient of a function. It describes the distribution of the gradient in a specified neighborhood around a point and makes the inf ...
is a representation of a
translation too. Here the physical process consists in an ordinary translation of a certain amount along
combined with translation along
,
:
where the amount is specified by the parameter
. Evidently
here represents the direction of the line.
Generally, the estimated
represents the direction (in
coordinates) along which infinitesimal translations leave the image invariant, in practice least variant. With every curvilinear coordinate basis pair, there is thus a pair of infinitesimal translators, a linear combination of which is a
Differential operator. The latter are related to
Lie algebra.
Miscellaneous
"Image" in the context of the GST can mean both an ordinary image and an image neighborhood thereof (local image), depending on context. For example, a photograph is an image as is any neighborhood of it.
See also
*
Structure tensor
In mathematics, the structure tensor, also referred to as the second-moment matrix, is a matrix derived from the gradient of a function. It describes the distribution of the gradient in a specified neighborhood around a point and makes the inf ...
*
Hough transform
*
Tensor
In mathematics, a tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space. Tensors may map between different objects such as vectors, scalars, and even other tensor ...
*
Gaussian
*
Corner detection
*
Edge detection
Edge detection includes a variety of mathematical methods that aim at identifying edges, curves in a digital image at which the image brightness changes sharply or, more formally, has discontinuities. The same problem of finding discontinuitie ...
*
Affine shape adaptation
Affine shape adaptation is a methodology for iteratively adapting the shape of the smoothing kernels in an affine group of smoothing kernels to the local image structure in neighbourhood region of a specific image point. Equivalently, affine shap ...
*
Directional derivative
In mathematics, the directional derivative of a multivariable differentiable (scalar) function along a given vector v at a given point x intuitively represents the instantaneous rate of change of the function, moving through x with a velocity ...
*
Differential operator
*
Lie algebra
References
{{reflist
Tensors
Feature detection (computer vision)