Leave-one-out error
   HOME

TheInfoList



OR:

For
mathematical analysis Analysis is the branch of mathematics dealing with continuous functions, limit (mathematics), limits, and related theories, such as Derivative, differentiation, Integral, integration, measure (mathematics), measure, infinite sequences, series (m ...
and statistics, Leave-one-out error can refer to the following: * Leave-one-out cross-validation Stability (CVloo, for ''stability of Cross Validation with leave one out''): An
algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
f has CVloo stability β with respect to the loss function V if the following holds: \forall i\in\, \mathbb_S\\geq1-\delta_ * Expected-to-leave-one-out error Stability (Eloo_, for ''Expected error from leaving one out''): An algorithm f has Eloo_ stability if for each n there exists a\beta_^m and a \delta_^m such that: \forall i\in\, \mathbb_S\\geq1-\delta_^m, with \beta_^mand \delta_^m going to zero for n\rightarrow\inf


Preliminary notations

With X and Y being a subset of the
real number In mathematics, a real number is a number that can be used to measure a ''continuous'' one-dimensional quantity such as a distance, duration or temperature. Here, ''continuous'' means that values can have arbitrarily small variations. Every ...
s R, or X and Y ⊂ R, being respectively an input space X and an output space Y, we consider a
training set In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data. Such algorithms function by making data-driven predictions or decisions, through building a mathematical model from ...
: S = \ of size m in Z = X \times Y drawn independently and identically distributed (i.i.d.) from an unknown distribution, here called "D". Then a learning algorithm is a function f from Z_m into F \subset YX which
maps A map is a symbolic depiction emphasizing relationships between elements of some space, such as objects, regions, or themes. Many maps are static, fixed to paper or some other durable medium, while others are dynamic or interactive. Although ...
a learning set S onto a function f_S from the input space X to the output space Y. To avoid complex notation, we consider only
deterministic algorithm In computer science, a deterministic algorithm is an algorithm that, given a particular input, will always produce the same output, with the underlying machine always passing through the same sequence of states. Deterministic algorithms are by far ...
s. It is also assumed that the algorithm f is symmetric with respect to S, i.e. it does not depend on the order of the elements in the
training set In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data. Such algorithms function by making data-driven predictions or decisions, through building a mathematical model from ...
. Furthermore, we assume that all functions are measurable and all sets are countable which does not limit the interest of the results presented here. The loss of an
hypothesis A hypothesis (plural hypotheses) is a proposed explanation for a phenomenon. For a hypothesis to be a scientific hypothesis, the scientific method requires that one can test it. Scientists generally base scientific hypotheses on previous obse ...
f with respect to an example z = (x,y) is then defined as V(f,z) = V(f(x),y). The empirical error of f can then be written as I_S = \frac\sum V(f,z_i). The true error of f is I = \mathbb_z V(f,z) Given a training set S of size m, we will build, for all i = 1....,m, modified training sets as follows: * By removing the i-th element S^ = \ * and/or by replacing the i-th element S^i = \{z_1 ,...,\ z_{i-1},\ z_i',\ z_{i+1},...,\ z_m\}


See also

*
Constructive analysis In mathematics, constructive analysis is mathematical analysis done according to some principles of constructive mathematics. This contrasts with ''classical analysis'', which (in this context) simply means analysis done according to the (more com ...
* History of calculus * Hypercomplex analysis *
Jackknife resampling In statistics, the jackknife (jackknife cross-validation) is a cross-validation technique and, therefore, a form of resampling. It is especially useful for bias and variance estimation. The jackknife pre-dates other common resampling methods suc ...
*
Statistical classification In statistics, classification is the problem of identifying which of a set of categories (sub-populations) an observation (or observations) belongs to. Examples are assigning a given email to the "spam" or "non-spam" class, and assigning a diagn ...
*
Timeline of calculus and mathematical analysis A timeline of calculus and mathematical analysis. 500BC to 1600 * 5th century BC - The Zeno's paradoxes, * 5th century BC - Antiphon attempts to square the circle, * 5th century BC - Democritus finds the volume of cone is 1/3 of volume of cylind ...


References

*S. Mukherjee, P. Niyogi, T. Poggio, and R. M. Rifkin. Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Adv. Comput. Math., 25(1-3):161–193, 2006 Machine learning