
Optimal control theory is a branch of
control theory
Control theory is a field of control engineering and applied mathematics that deals with the control system, control of dynamical systems in engineered processes and machines. The objective is to develop a model or algorithm governing the applic ...
that deals with finding a
control for a
dynamical system
In mathematics, a dynamical system is a system in which a Function (mathematics), function describes the time dependence of a Point (geometry), point in an ambient space, such as in a parametric curve. Examples include the mathematical models ...
over a period of time such that an
objective function
In mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost ...
is optimized.
It has numerous applications in science, engineering and operations research. For example, the dynamical system might be a
spacecraft
A spacecraft is a vehicle that is designed spaceflight, to fly and operate in outer space. Spacecraft are used for a variety of purposes, including Telecommunications, communications, Earth observation satellite, Earth observation, Weather s ...
with controls corresponding to rocket thrusters, and the objective might be to reach the
Moon
The Moon is Earth's only natural satellite. It Orbit of the Moon, orbits around Earth at Lunar distance, an average distance of (; about 30 times Earth diameter, Earth's diameter). The Moon rotation, rotates, with a rotation period (lunar ...
with minimum fuel expenditure. Or the dynamical system could be a nation's
economy
An economy is an area of the Production (economics), production, Distribution (economics), distribution and trade, as well as Consumption (economics), consumption of Goods (economics), goods and Service (economics), services. In general, it is ...
, with the objective to minimize
unemployment; the controls in this case could be
fiscal and
monetary policy
Monetary policy is the policy adopted by the monetary authority of a nation to affect monetary and other financial conditions to accomplish broader objectives like high employment and price stability (normally interpreted as a low and stable rat ...
. A dynamical system may also be introduced to embed
operations research problems within the framework of optimal control theory.
Optimal control is an extension of the
calculus of variations
The calculus of variations (or variational calculus) is a field of mathematical analysis that uses variations, which are small changes in Function (mathematics), functions
and functional (mathematics), functionals, to find maxima and minima of f ...
, and is a mathematical optimization method for deriving
control policies. The method is largely due to the work of
Lev Pontryagin and
Richard Bellman in the 1950s, after contributions to calculus of variations by
Edward J. McShane. Optimal control can be seen as a
control strategy in
control theory
Control theory is a field of control engineering and applied mathematics that deals with the control system, control of dynamical systems in engineered processes and machines. The objective is to develop a model or algorithm governing the applic ...
.
General method
Optimal control deals with the problem of finding a control law for a given system such that a certain
optimality criterion is achieved. A control problem includes a
cost functional that is a
function of state and control variables. An optimal control is a set of
differential equations describing the paths of the control variables that minimize the cost function. The optimal control can be derived using
Pontryagin's maximum principle (a
necessary condition also known as Pontryagin's minimum principle or simply Pontryagin's principle), or by solving the
Hamilton–Jacobi–Bellman equation (a
sufficient condition).
We begin with a simple example. Consider a car traveling in a straight line on a hilly road. The question is, how should the driver press the accelerator pedal in order to ''minimize'' the total traveling time? In this example, the term ''control law'' refers specifically to the way in which the driver presses the accelerator and shifts the gears. The ''system'' consists of both the car and the road, and the ''optimality criterion'' is the minimization of the total traveling time. Control problems usually include ancillary
constraints. For example, the amount of available fuel might be limited, the accelerator pedal cannot be pushed through the floor of the car, speed limits, etc.
A proper cost function will be a mathematical expression giving the traveling time as a function of the speed, geometrical considerations, and
initial conditions of the system.
Constraints are often interchangeable with the cost function.
Another related optimal control problem may be to find the way to drive the car so as to minimize its fuel consumption, given that it must complete a given course in a time not exceeding some amount. Yet another related control problem may be to minimize the total monetary cost of completing the trip, given assumed monetary prices for time and fuel.
A more abstract framework goes as follows.
Minimize the continuous-time cost functional
subject to the first-order dynamic constraints (the state equation)
the algebraic ''path constraints''
and the
endpoint conditions
where
is the ''state'',
is the ''control'',
is the independent variable (generally speaking, time),
is the initial time, and
is the terminal time. The terms
and
are called the ''endpoint cost '' and the ''running cost'' respectively. In the calculus of variations,
and
are referred to as the Mayer term and the ''
Lagrangian'', respectively. Furthermore, it is noted that the path constraints are in general ''inequality'' constraints and thus may not be active (i.e., equal to zero) at the optimal solution. It is also noted that the optimal control problem as stated above may have multiple solutions (i.e., the solution may not be unique). Thus, it is most often the case that any solution