Scoring algorithm, also known as Fisher's scoring, is a form of
Newton's method
In numerical analysis, Newton's method, also known as the Newton–Raphson method, named after Isaac Newton and Joseph Raphson, is a root-finding algorithm which produces successively better approximations to the roots (or zeroes) of a real-val ...
used in
statistics
Statistics (from German: ''Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, industri ...
to solve
maximum likelihood
In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statis ...
equations
numerically, named after
Ronald Fisher
Sir Ronald Aylmer Fisher (17 February 1890 – 29 July 1962) was a British polymath who was active as a mathematician, statistician, biologist, geneticist, and academic. For his work in statistics, he has been described as "a genius who ...
.
Sketch of derivation
Let
be
random variable
A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...
s, independent and identically distributed with twice differentiable
p.d.f. , and we wish to calculate the
maximum likelihood estimator
In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statisti ...
(M.L.E.)
of
. First, suppose we have a starting point for our algorithm
, and consider a
Taylor expansion of the
score function,
, about
:
:
where
:
is the
observed information matrix at
. Now, setting
, using that
and rearranging gives us:
:
We therefore use the algorithm
:
and under certain regularity conditions, it can be shown that
.
Fisher scoring
In practice,
is usually replaced by