The Truncated Newton Method originated in a paper by Ron Dembo and Trond Steihaug first published in Mathematical Programming (Dembo, R.S., Steihaug, T. Truncated-Newton algorithms for large-scale unconstrained optimization. Mathematical Programming 26, 190–212)'. Convergence results for this algorithm can be found in: Dembo, Ron S., Stanley C. Eisenstat, and Trond Steihaug. "Inexact newton methods." SIAM Journal on Numerical analysis 19.2 (1982): 400-408. Also known as Hessian-free optimization, are a family of

optimization algorithm Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfi ...

s designed for optimizing non-linear functions with large numbers of

independent variable Dependent and independent variables are variables in mathematical modeling, statistical modeling and experimental sciences. Dependent variables receive this name because, in an experiment, their values are studied under the supposition or demand ...

s. A truncated Newton method consists of repeated application of an iterative optimization algorithm to approximately solve Newton's equations, to determine an update to the function's parameters. The inner solver is ''truncated'', i.e., run for only a limited number of iterations. It follows that, for truncated Newton methods to work, the inner solver needs to produce a good approximation in a finite number of iterations;

conjugate gradient In mathematics, the conjugate gradient method is an algorithm for the numerical solution of particular systems of linear equations, namely those whose matrix is positive-definite. The conjugate gradient method is often implemented as an iterative ...

has been suggested and evaluated as a candidate inner loop. Another prerequisite is good

preconditioning In mathematics, preconditioning is the application of a transformation, called the preconditioner, that conditions a given problem into a form that is more suitable for numerical solving methods. Preconditioning is typically related to reducing ...

for the inner algorithm.

References

Further reading