Numerical methods for ordinary differential equations are methods used to find
numerical approximations to the solutions of
ordinary differential equation
In mathematics, an ordinary differential equation (ODE) is a differential equation (DE) dependent on only a single independent variable (mathematics), variable. As with any other DE, its unknown(s) consists of one (or more) Function (mathematic ...
s (ODEs). Their use is also known as "
numerical integration
In analysis, numerical integration comprises a broad family of algorithms for calculating the numerical value of a definite integral.
The term numerical quadrature (often abbreviated to quadrature) is more or less a synonym for "numerical integr ...
", although this term can also refer to the computation of
integral
In mathematics, an integral is the continuous analog of a Summation, sum, which is used to calculate area, areas, volume, volumes, and their generalizations. Integration, the process of computing an integral, is one of the two fundamental oper ...
s.
Many differential equations cannot be solved exactly. For practical purposes, however – such as in engineering – a numeric approximation to the solution is often sufficient. The
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of Rigour#Mathematics, mathematically rigorous instructions, typically used to solve a class of specific Computational problem, problems or to perform a computation. Algo ...
s studied here can be used to compute such an approximation. An alternative method is to use techniques from
calculus
Calculus is the mathematics, mathematical study of continuous change, in the same way that geometry is the study of shape, and algebra is the study of generalizations of arithmetic operations.
Originally called infinitesimal calculus or "the ...
to obtain a
series expansion
In mathematics, a series expansion is a technique that expresses a Function (mathematics), function as an infinite sum, or Series (mathematics), series, of simpler functions. It is a method for calculating a Function (mathematics), function that ...
of the solution.
Ordinary differential equations occur in many scientific disciplines, including
physics
Physics is the scientific study of matter, its Elementary particle, fundamental constituents, its motion and behavior through space and time, and the related entities of energy and force. "Physical science is that department of knowledge whi ...
,
chemistry
Chemistry is the scientific study of the properties and behavior of matter. It is a physical science within the natural sciences that studies the chemical elements that make up matter and chemical compound, compounds made of atoms, molecules a ...
,
biology
Biology is the scientific study of life and living organisms. It is a broad natural science that encompasses a wide range of fields and unifying principles that explain the structure, function, growth, History of life, origin, evolution, and ...
, and
economics
Economics () is a behavioral science that studies the Production (economics), production, distribution (economics), distribution, and Consumption (economics), consumption of goods and services.
Economics focuses on the behaviour and interac ...
. In addition, some methods in
numerical partial differential equations
Numerical may refer to:
* Number
* Numerical digit
* Numerical analysis
Numerical analysis is the study of algorithms that use numerical approximation (as opposed to symbolic computation, symbolic manipulations) for the problems of mathematical ...
convert the
partial differential equation
In mathematics, a partial differential equation (PDE) is an equation which involves a multivariable function and one or more of its partial derivatives.
The function is often thought of as an "unknown" that solves the equation, similar to ho ...
into an ordinary differential equation, which must then be solved.
The problem
A first-order differential equation is an
Initial value problem
In multivariable calculus, an initial value problem (IVP) is an ordinary differential equation together with an initial condition which specifies the value of the unknown function at a given point in the domain. Modeling a system in physics or ...
(IVP) of the form,
where
is a function
, and the initial condition
is a given vector. ''First-order'' means that only the first derivative of ''y'' appears in the equation, and higher derivatives are absent.
Without loss of generality to higher-order systems, we restrict ourselves to ''first-order'' differential equations, because a higher-order ODE can be converted into a larger system of first-order equations by introducing extra variables. For example, the second-order equation can be rewritten as two first-order equations: and
In this section, we describe numerical methods for IVPs, and remark that ''boundary value problems'' (BVPs) require a different set of tools. In a BVP, one defines values, or components of the solution ''y'' at more than one point. Because of this, different methods need to be used to solve BVPs. For example, the
shooting method
In numerical analysis, the shooting method is a method for solving a boundary value problem by reducing it to an initial value problem. It involves finding solutions to the initial value problem for different initial conditions until one finds the ...
(and its variants) or global methods like
finite difference
A finite difference is a mathematical expression of the form . Finite differences (or the associated difference quotients) are often used as approximations of derivatives, such as in numerical differentiation.
The difference operator, commonly d ...
s,
[LeVeque, R. J. (2007). Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems (Vol. 98). SIAM.] Galerkin method
In mathematics, in the area of numerical analysis, Galerkin methods are a family of methods for converting a continuous operator problem, such as a differential equation, commonly in a weak formulation, to a discrete problem by applying linear c ...
s, or
collocation method
In mathematics, a collocation method is a method for the numerical solution of ordinary differential equations, partial differential equations and integral equations. The idea is to choose a finite-dimensional space of candidate solutions (usually ...
s are appropriate for that class of problems.
The
Picard–Lindelöf theorem
In mathematics, specifically the study of differential equations, the Picard–Lindelöf theorem gives a set of conditions under which an initial value problem has a unique solution. It is also known as Picard's existence theorem, the Cauchy– ...
states that there is a unique solution, provided ''f'' is
Lipschitz-continuous.
Methods
Numerical methods for solving first-order IVPs often fall into one of two large categories:
linear multistep method
Linear multistep methods are used for the numerical solution of ordinary differential equations. Conceptually, a numerical method starts from an initial point and then takes a short step forward in time to find the next solution point. The proce ...
s, or
Runge–Kutta methods
In numerical analysis, the Runge–Kutta methods ( ) are a family of Explicit and implicit methods, implicit and explicit iterative methods, List of Runge–Kutta methods, which include the Euler method, used in temporal discretization for the a ...
. A further division can be realized by dividing methods into those that are explicit and those that are implicit. For example, implicit
linear multistep method
Linear multistep methods are used for the numerical solution of ordinary differential equations. Conceptually, a numerical method starts from an initial point and then takes a short step forward in time to find the next solution point. The proce ...
s include
Adams-Moulton methods, and
backward differentiation methods (BDF), whereas
implicit Runge–Kutta methods
Implicit may refer to:
Mathematics
* Implicit function
* Implicit function theorem
* Implicit curve
* Implicit surface
* Implicit differential equation
Other uses
* Implicit assumption, in logic
* Implicit-association test, in social psychology ...
include diagonally implicit Runge–Kutta (DIRK), singly diagonally implicit Runge–Kutta (SDIRK), and Gauss–Radau (based on
Gaussian quadrature
In numerical analysis, an -point Gaussian quadrature rule, named after Carl Friedrich Gauss, is a quadrature rule constructed to yield an exact result for polynomials of degree or less by a suitable choice of the nodes and weights for .
Th ...
) numerical methods. Explicit examples from the
linear multistep family include the
Adams–Bashforth methods
Linear multistep methods are used for the numerical solution of ordinary differential equations. Conceptually, a numerical method starts from an initial point and then takes a short step forward in time to find the next solution point. The proce ...
, and any Runge–Kutta method with a lower diagonal
Butcher tableau
A butcher is a person who may slaughter animals, dress their flesh, sell their meat, or participate within any combination of these three tasks. They may prepare standard cuts of meat and poultry for sale in retail or wholesale food establishme ...
is
explicit
Explicit refers to something that is specific, clear, or detailed. It can also mean:
* Explicit knowledge, knowledge that can be readily articulated, codified and transmitted to others
* Explicit (text), the final words of a text; contrast with inc ...
. A loose rule of thumb dictates that
stiff differential equations require the use of implicit schemes, whereas non-stiff problems can be solved more efficiently with explicit schemes.
The so-called
general linear methods (GLMs) are a generalization of the above two large classes of methods.
Euler method
From any point on a curve, you can find an approximation of a nearby point on the curve by moving a short distance along a line
tangent
In geometry, the tangent line (or simply tangent) to a plane curve at a given point is, intuitively, the straight line that "just touches" the curve at that point. Leibniz defined it as the line through a pair of infinitely close points o ...
to the curve.
Starting with the differential equation (), we replace the derivative ''y''′ by the
finite difference
A finite difference is a mathematical expression of the form . Finite differences (or the associated difference quotients) are often used as approximations of derivatives, such as in numerical differentiation.
The difference operator, commonly d ...
approximation
which when re-arranged yields the following formula
:
and using () gives:
This formula is usually applied in the following way. We choose a step size ''h'', and we construct the sequence
We denote by
a numerical estimate of the exact solution
. Motivated by (), we compute these estimates by the following
recursive
Recursion occurs when the definition of a concept or process depends on a simpler or previous version of itself. Recursion is used in a variety of disciplines ranging from linguistics to logic. The most common application of recursion is in m ...
scheme
This is the ''
Euler method
In mathematics and computational science, the Euler method (also called the forward Euler method) is a first-order numerical analysis, numerical procedure for solving ordinary differential equations (ODEs) with a given Initial value problem, in ...
'' (or ''
forward Euler method'', in contrast with the ''backward Euler method'', to be described below). The method is named after
Leonhard Euler
Leonhard Euler ( ; ; ; 15 April 170718 September 1783) was a Swiss polymath who was active as a mathematician, physicist, astronomer, logician, geographer, and engineer. He founded the studies of graph theory and topology and made influential ...
who described it in 1768.
The Euler method is an example of an
''explicit'' method. This means that the new value ''y''
''n''+1 is defined in terms of things that are already known, like ''y''
''n''.
Backward Euler method
If, instead of (), we use the approximation
we get the ''backward Euler method'':
The backward Euler method is an
''implicit'' method, meaning that we have to solve an equation to find ''y''
''n''+1. One often uses
fixed-point iteration or (some modification of) the
Newton–Raphson method to achieve this.
It costs more time to solve this equation than explicit methods; this cost must be taken into consideration when one selects the method to use. The advantage of implicit methods such as () is that they are usually more stable for solving a
stiff equation
In mathematics, a stiff equation is a differential equation for which certain numerical methods for solving the equation are numerically unstable, unless the step size is taken to be extremely small. It has proven difficult to formulate a precise ...
, meaning that a larger step size ''h'' can be used.
First-order exponential integrator method
Exponential integrators describe a large class of integrators that have recently seen a lot of development.
[ This is a modern and extensive review paper for exponential integrators] They date back to at least the 1960s.
In place of (), we assume the differential equation is either of the form
or it has been locally linearized about a background state to produce a linear term
and a nonlinear term
.
Exponential integrators are constructed by multiplying () by
, and exactly integrating the result over
a time interval