Pontryagin's maximum principle is used in
optimal control
Optimal control theory is a branch of mathematical optimization that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized. It has numerous applications in science, engineering and ...
theory to find the best possible control for taking a
dynamical system
In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water i ...
from one state to another, especially in the presence of constraints for the state or input controls. It states that it is
necessary for any optimal control along with the optimal state trajectory to solve the so-called Hamiltonian system, which is a two-point
boundary value problem
In mathematics, in the field of differential equations, a boundary value problem is a differential equation together with a set of additional constraints, called the boundary conditions. A solution to a boundary value problem is a solution to t ...
, plus a maximum condition of the
control Hamiltonian. These necessary conditions become sufficient under certain convexity conditions on the objective and constraint functions.
The maximum principle was formulated in 1956 by the Russian mathematician
Lev Pontryagin
Lev Semenovich Pontryagin (russian: Лев Семёнович Понтрягин, also written Pontriagin or Pontrjagin) (3 September 1908 – 3 May 1988) was a Soviet mathematician. He was born in Moscow and lost his eyesight completely d ...
and his students, and its initial application was to the maximization of the terminal speed of a rocket. The result was derived using ideas from the classical
calculus of variations
The calculus of variations (or Variational Calculus) is a field of mathematical analysis that uses variations, which are small changes in functions
and functionals, to find maxima and minima of functionals: mappings from a set of functions t ...
. After a slight
perturbation
Perturbation or perturb may refer to:
* Perturbation theory, mathematical methods that give approximate solutions to problems that cannot be solved exactly
* Perturbation (geology), changes in the nature of alluvial deposits over time
* Perturbat ...
of the optimal control, one considers the first-order term of a
Taylor expansion with respect to the perturbation; sending the perturbation to zero leads to a variational inequality from which the maximum principle follows.
Widely regarded as a milestone in optimal control theory, the significance of the maximum principle lies in the fact that maximizing the Hamiltonian is much easier than the original infinite-dimensional control problem; rather than maximizing over a
function space
In mathematics, a function space is a set of functions between two fixed sets. Often, the domain and/or codomain will have additional structure which is inherited by the function space. For example, the set of functions from any set into a vect ...
, the problem is converted to a
pointwise In mathematics, the qualifier pointwise is used to indicate that a certain property is defined by considering each value f(x) of some function f. An important class of pointwise concepts are the ''pointwise operations'', that is, operations defined ...
optimization. A similar logic leads to
Bellman's principle of optimality, a related approach to optimal control problems which states that the optimal trajectory remains optimal at intermediate points in time.
The resulting
Hamilton–Jacobi–Bellman equation In optimal control theory, the Hamilton-Jacobi-Bellman (HJB) equation gives a necessary and sufficient condition for optimality of a control with respect to a loss function. It is, in general, a nonlinear partial differential equation in the val ...
provides a necessary and sufficient condition for an optimum, and admits
a straightforward extension to stochastic optimal control problems, whereas the maximum principle does not.
However in contrast to the Hamilton–Jacobi–Bellman equation, which needs to hold over the entire state space to be valid, Pontryagin's Maximum Principle is potentially more computationally efficient in that the conditions which it specifies only need to hold over a particular trajectory.
Notation
For set
and functions
and
we use the following notation:
:
:
:
:
:
Formal statement of necessary conditions for minimization problem
Here the necessary conditions are shown for minimization of a functional. Take
to be the state of the
dynamical system
In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in an ambient space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water i ...
with input
, such that
:
where
is the set of admissible controls and
is the terminal (i.e., final) time of the system. The control
must be chosen for all