Maximum likelihood

Prediction bias

Maximum likelihood estimates of parameters can be substituted into expressions for the probability density function, cumulative distribution function, or quantile function, to generate predictions of probabilities or quantiles of out-of-sample events. This method for predicting probabilities is recommended in statistics text-books and actuarial textbooks, and is widely used in the scientific literature. However, maximum likelihood prediction fails to propagate the uncertainty around the maximum likelihood parameter estimates into the prediction. As a result, the predicted probabilities are not well calibrated, and should not be expected to correspond to the frequencies of out-of-sample events. In particular, tail exceedance probabilities and tail exceedance quantiles are typically underestimated, sometimes dramatically. The underestimation is largest when there is little training data, many parameters being estimated, and for the far tail. For cases where this prediction bias is a problem, Bayesian predictions can provide a solution if the prior is chosen so as to reduce or eliminate the bias.