Davidon–Fletcher–Powell formula

The Davidon–Fletcher–Powell formula (or DFP; named after William C. Davidon, Roger Fletcher, and Michael J. D. Powell) finds the solution to the secant equation that is closest to the current estimate and satisfies the curvature condition. It was the first quasi-Newton method to generalize the secant method to a multidimensional problem. This update maintains the symmetry and positive definiteness of the Hessian matrix.

Given a function $f(x)$, its gradient ($\nabla f$), and positive-definite Hessian matrix $B$, the Taylor series is

:$f(x_k+s_k) = f(x_k) + \nabla f(x_k)^T s_k + \frac s^T_k s_k + \dots,$

and the Taylor series of the gradient itself (secant equation)

:$\nabla f(x_k+s_k) = \nabla f(x_k) + B s_k + \dots$

is used to update $B$.

The DFP formula finds a solution that is symmetric, positive-definite and closest to the current approximate value of $B_k$:

:$B_= (I - \gamma_k y_k s_k^T) B_k (I - \gamma_k s_k y_k^T) + \gamma_k y_k y_k^T,$

where

:$y_k = \nabla f(x_k+s_k) - \nabla f(x_k),$
:$\gamma_k = \frac,$

and $B_k$ is a symmetric and positive-definite matrix.

The corresponding update to the inverse Hessian approximation $H_k = B_k^$ is given by

:$H_ = H_k - \frac + \frac.$

$B$ is assumed to be positive-definite, and the vectors $s_k^T$ and $y$ must satisfy the curvature condition

: $s_k^T y_k = s_k^T B s_k > 0.$

The DFP formula is quite effective, but it was soon superseded by the Broyden–Fletcher–Goldfarb–Shanno formula, which is its dual (interchanging the roles of ''y'' and ''s'').

See also

* Newton's method
* Newton's method in optimization
* Quasi-Newton method
* Broyden–Fletcher–Goldfarb–Shanno (BFGS) method
* Limited-memory BFGS method
* Symmetric rank-one formula
* Nelder–Mead method

References

Further reading

*
*
*
*
*

{{DEFAULTSORT:Davidon-Fletcher-Powell formula
Optimization algorithms and methods