In
numerical methods for
stochastic differential equations
A stochastic differential equation (SDE) is a differential equation in which one or more of the terms is a stochastic process, resulting in a solution which is also a stochastic process. SDEs are used to model various phenomena such as stock pr ...
, the Markov chain approximation method (MCAM) belongs to the several numerical (schemes) approaches used in
stochastic control theory. Regrettably the simple adaptation of the deterministic schemes for matching up to stochastic models such as the Runge–Kutta method does not work at all.
It is a powerful and widely usable set of ideas, due to the current infancy of stochastic control it might be even said 'insights.' for numerical and other approximations problems in
stochastic processes. They represent counterparts from deterministic control theory such as
optimal control theory
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...
.
[F. B. Hanson]
"Markov Chain Approximation"
in C. T. Leondes, ed., ''Stochastic Digital Control System Techniques'', Academic Press, 1996, .
The basic idea of the MCAM is to approximate the original
controlled process by a chosen
controlled markov process on a
finite state space. In case of need, one must as well approximate the
cost function for one that matches up the
Markov chain chosen to approximate the original stochastic process.
See also
*
Control theory
Control theory is a field of mathematics that deals with the control of dynamical systems in engineered processes and machines. The objective is to develop a model or algorithm governing the application of system inputs to drive the system to a ...
*
Optimal control
*
Stochastic differential equation
A stochastic differential equation (SDE) is a differential equation in which one or more of the terms is a stochastic process, resulting in a solution which is also a stochastic process. SDEs are used to model various phenomena such as stock p ...
*
Differential equation
In mathematics, a differential equation is an equation that relates one or more unknown functions and their derivatives. In applications, the functions generally represent physical quantities, the derivatives represent their rates of change, an ...
*
Numerical analysis
Numerical analysis is the study of algorithms that use numerical approximation (as opposed to symbolic manipulations) for the problems of mathematical analysis (as distinguished from discrete mathematics). It is the study of numerical methods ...
*
Stochastic process
References
{{Stochastic processes
Markov processes
de:Markow-Kette