Filtering Problem (stochastic Processes)
   HOME

TheInfoList



OR:

In the theory of
stochastic processes In probability theory and related fields, a stochastic () or random process is a mathematical object usually defined as a family of random variables. Stochastic processes are widely used as mathematical models of systems and phenomena that a ...
, filtering describes the problem of determining the
state State may refer to: Arts, entertainment, and media Literature * ''State Magazine'', a monthly magazine published by the U.S. Department of State * ''The State'' (newspaper), a daily newspaper in Columbia, South Carolina, United States * '' Our ...
of a system from an incomplete and potentially noisy set of observations. While originally motivated by problems in engineering, filtering found applications in many fields from signal processing to finance. The problem of optimal non-linear filtering (even for the non-stationary case) was solved by
Ruslan L. Stratonovich Ruslan Leont'evich Stratonovich (russian: Русла́н Лео́нтьевич Страто́нович) was a Russian physicist, engineer, and probabilist and one of the founders of the theory of stochastic differential equations. Biography R ...
(1959, 1960), see also Harold J. Kushner's work and Moshe Zakai's, who introduced a simplified dynamics for the unnormalized conditional law of the filter known as Zakai equation. The solution, however, is infinite-dimensional in the general case. Certain approximations and special cases are well understood: for example, the linear filters are optimal for Gaussian random variables, and are known as the
Wiener filter In signal processing, the Wiener filter is a filter used to produce an estimate of a desired or target random process by linear time-invariant ( LTI) filtering of an observed noisy process, assuming known stationary signal and noise spectra, and ...
and the
Kalman-Bucy filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimat ...
. More generally, as the solution is infinite dimensional, it requires finite dimensional approximations to be implemented in a computer with finite memory. A finite dimensional approximated nonlinear filter may be more based on heuristics, such as the extended Kalman filter or the assumed density filters, or more methodologically oriented such as for example the Projection Filters, some sub-families of which are shown to coincide with the Assumed Density Filters. Damiano Brigo,
Bernard Hanzon Bernard (''Bernhard'') is a French and West Germanic masculine given name. It is also a surname. The name is attested from at least the 9th century. West Germanic ''Bernhard'' is composed from the two elements ''bern'' "bear" and ''hard'' "brave ...
and François Le Gland, Approximate Nonlinear Filtering by Projection on Exponential Manifolds of Densities, Bernoulli, Vol. 5, N. 3 (1999), pp. 495--534
In general, if the separation principle applies, then filtering also arises as part of the solution of an optimal control problem. For example, the
Kalman filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estima ...
is the estimation part of the optimal control solution to the linear-quadratic-Gaussian control problem.


The mathematical formalism

Consider a
probability space In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a mathematical construct that provides a formal model of a random process or "experiment". For example, one can define a probability space which models t ...
(Ω, Σ, P) and suppose that the (random) state ''Y''''t'' in ''n''-
dimension In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coor ...
al
Euclidean space Euclidean space is the fundamental space of geometry, intended to represent physical space. Originally, that is, in Euclid's ''Elements'', it was the three-dimensional space of Euclidean geometry, but in modern mathematics there are Euclidean sp ...
R''n'' of a system of interest at time ''t'' is a
random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the p ...
''Y''''t'' : Ω → R''n'' given by the solution to an Itō
stochastic differential equation A stochastic differential equation (SDE) is a differential equation in which one or more of the terms is a stochastic process, resulting in a solution which is also a stochastic process. SDEs are used to model various phenomena such as stock ...
of the form :\mathrm Y_ = b(t, Y_) \, \mathrm t + \sigma (t, Y_) \, \mathrm B_, where ''B'' denotes standard ''p''-dimensional
Brownian motion Brownian motion, or pedesis (from grc, πήδησις "leaping"), is the random motion of particles suspended in a medium (a liquid or a gas). This pattern of motion typically consists of random fluctuations in a particle's position insi ...
, ''b'' : [0, +∞) × R''n'' → R''n'' is the drift field, and ''σ'' : [0, +∞) × R''n'' → R''n''×''p'' is the diffusion field. It is assumed that observations ''H''''t'' in R''m'' (note that ''m'' and ''n'' may, in general, be unequal) are taken for each time ''t'' according to :H_ = c(t, Y_) + \gamma (t, Y_) \cdot \mbox. Adopting the Itō interpretation of the stochastic differential and setting : Z_ = \int_^ H_ \, \mathrm s, this gives the following stochastic integral representation for the observations ''Z''''t'': :\mathrm Z_ = c(t, Y_) \, \mathrm t + \gamma (t, Y_) \, \mathrm W_, where ''W'' denotes standard ''r''-dimensional
Brownian motion Brownian motion, or pedesis (from grc, πήδησις "leaping"), is the random motion of particles suspended in a medium (a liquid or a gas). This pattern of motion typically consists of random fluctuations in a particle's position insi ...
, independent of ''B'' and the initial condition ''Y''0, and ''c'' : [0, +∞) × R''n'' → R''n'' and ''γ'' : [0, +∞) × R''n'' → R''n''×''r'' satisfy :\big, c (t, x) \big, + \big, \gamma (t, x) \big, \leq C \big( 1 + , x , \big) for all ''t'' and ''x'' and some constant ''C''. The filtering problem is the following: given observations ''Z''''s'' for 0 ≤ ''s'' ≤ ''t'', what is the best estimate ''Ŷ''''t'' of the true state ''Y''''t'' of the system based on those observations? By "based on those observations" it is meant that ''Ŷ''''t'' is measurable function, measurable with respect to the sigma algebra, ''σ''-algebra ''G''''t'' generated by the observations ''Z''''s'', 0 ≤ ''s'' ≤ ''t''. Denote by ''K'' = ''K''(''Z'', ''t'') the collection of all R''n''-valued random variables ''Y'' that are square-integrable and ''G''''t''-measurable: :K = K(Z, t) = L^ (\Omega, G_, \mathbf; \mathbf^). By "best estimate", it is meant that ''Ŷ''''t'' minimizes the mean-square distance between ''Y''''t'' and all candidates in ''K'': :\mathbf \left Y_ - \hat_ \big, ^ \right= \inf_ \mathbf \left Y_ - Y \big, ^ \right \qquad \mbox


Basic result: orthogonal projection

The space ''K''(''Z'', ''t'') of candidates is a
Hilbert space In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natu ...
, and the general theory of Hilbert spaces implies that the solution ''Ŷ''''t'' of the minimization problem (M) is given by :\hat_ = P_ \big( Y_ \big), where ''P''''K''(''Z'',''t'') denotes the
orthogonal projection In linear algebra and functional analysis, a projection is a linear transformation P from a vector space to itself (an endomorphism) such that P\circ P=P. That is, whenever P is applied twice to any vector, it gives the same result as if i ...
of ''L''2(Ω, Σ, P; R''n'') onto the linear subspace ''K''(''Z'', ''t'') = ''L''2(Ω, ''G''''t'', P; R''n''). Furthermore, it is a general fact about conditional expectations that if ''F'' is any sub-''σ''-algebra of Σ then the orthogonal projection :P_ : L^ (\Omega, \Sigma, \mathbf; \mathbf^) \to L^ (\Omega, F, \mathbf; \mathbf^) is exactly the conditional expectation operator E ''F'' i.e., :P_ (X) = \mathbf \big F \big Hence, :\hat_ = P_ \big( Y_ \big) = \mathbf \big G_ \big This elementary result is the basis for the general Fujisaki-Kallianpur-Kunita equation of filtering theory.


See also

* The smoothing problem, closely related to the filtering problem *
Filter (signal processing) In signal processing, a filter is a device or process that removes some unwanted components or features from a signal. Filtering is a class of signal processing, the defining feature of filters being the complete or partial suppression of some asp ...
*
Kalman filter For statistics and control theory, Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estima ...
, a well-known filtering algorithm related both to the filtering problem and the smoothing problem * Smoothing


References


Further reading

* * {{cite book , last = Øksendal , first = Bernt K. , authorlink = Bernt Øksendal , title = Stochastic Differential Equations: An Introduction with Applications , edition = Sixth , publisher=Springer , location = Berlin , year = 2003 , isbn = 3-540-04758-1 (See Section 6.1) Control theory Signal estimation Stochastic differential equations