In
regression analysis
In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one ...
, an interval predictor model (IPM) is an approach to regression where bounds on the function to be approximated are obtained.
This differs from other techniques in
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
, where usually one wishes to estimate point values or an entire probability distribution.
Interval Predictor Models are sometimes referred to as a
nonparametric regression
Nonparametric regression is a category of regression analysis in which the predictor does not take a predetermined form but is constructed according to information derived from the data. That is, no parametric form is assumed for the relationship ...
technique, because a potentially infinite set of functions are contained by the IPM, and no specific distribution is implied for the regressed variables.
Multiple-input multiple-output IPMs for multi-point data commonly used to represent functions have been recently developed. These IPM prescribe the parameters of the model as a path-connected, semi-algebraic set using sliced-normal or sliced-exponential distributions. A key advantage of this approach is its ability to characterize complex parameter dependencies to varying fidelity levels. This practice enables the analyst to adjust the desired level of conservatism in the prediction.
As a consequence of the theory of
scenario optimization
The scenario approach or scenario optimization approach is a technique for obtaining solutions to robust optimization and chance-constrained optimization problems based on a sample of the constraints. It also relates to inductive reasoning in mod ...
, in many cases rigorous predictions can be made regarding the performance of the model at test time.
Hence an interval predictor model can be seen as a guaranteed bound on
quantile regression
Quantile regression is a type of regression analysis used in statistics and econometrics. Whereas the method of least squares estimates the conditional ''mean'' of the response variable across values of the predictor variables, quantile regressi ...
.
Interval predictor models can also be seen as a way to prescribe the
support
Support may refer to:
Arts, entertainment, and media
* Supporting character
Business and finance
* Support (technical analysis)
* Child support
* Customer support
* Income Support
Construction
* Support (structure), or lateral support, a ...
of random predictor models, of which a
Gaussian process
In probability theory and statistics, a Gaussian process is a stochastic process (a collection of random variables indexed by time or space), such that every finite collection of those random variables has a multivariate normal distribution, i.e. e ...
is a specific case
.
Convex interval predictor models
Typically the interval predictor model is created by specifying a parametric function, which is usually chosen to be the product of a parameter vector and a basis.
Usually the basis is made up of polynomial features or a radial basis is sometimes used.
Then a convex set is assigned to the parameter vector, and the size of the convex set is minimized such that every possible data point can be predicted by one possible value of the parameters.
Ellipsoidal parameters sets were used by Campi (2009), which yield a convex optimization program to train the IPM.
Crespo (2016) proposed the use of a hyperrectangular parameter set, which results in a convenient, linear form for the bounds of the IPM.
Hence the IPM can be trained with a linear optimization program:
:
where the training data examples are
and
, and the Interval Predictor Model bounds
and
are parameterised by the parameter vector
.
The reliability of such an IPM is obtained by noting that for a convex IPM the number of support constraints is less than the dimensionality of the
trainable parameters, and hence the scenario approach can be applied.
Lacerda (2017) demonstrated that this approach can be extended to situations where the training data is interval valued rather than point valued.
Non-convex interval predictor models
In Campi (2015) a non-convex theory of scenario optimization was proposed.
This involves measuring the number of support constraints,
, for the Interval Predictor Model after training and hence making predictions about the reliability of the model.
This enables non-convex IPMs to be created, such as a single layer neural network.
Campi (2015) demonstrates that an algorithm where the scenario optimization program is only solved
times which can determine the reliability of the model at test time without a prior evaluation on a validation set.
This is achieved by solving the optimisation program
:
where the interval predictor model center line
, and the model width
. This results in an IPM which makes predictions with homoscedastic uncertainty.
Sadeghi (2019) demonstrates that the non-convex scenario approach from Campi (2015) can be extended to train deeper neural networks which predict intervals with hetreoscedastic uncertainty on datasets with imprecision.
This is achieved by proposing generalizations to the max-error loss function given by
:
which is equivalent to solving the optimisation program proposed by Campi (2015).
Applications
Initially,
scenario optimization
The scenario approach or scenario optimization approach is a technique for obtaining solutions to robust optimization and chance-constrained optimization problems based on a sample of the constraints. It also relates to inductive reasoning in mod ...
was applied to robust control problems.
Crespo (2015) and (2021) applied Interval Predictor Models to the design of space radiation shielding
and to system identification.
In Patelli (2017), Faes (2019), and Crespo (2018), Interval Predictor models were applied to the
structural reliability analysis problem.
Brandt (2017) applies interval predictor models to fatigue damage estimation of offshore wind turbines jacket substructures.
Garatti (2019) proved that Chebyshev layers (i.e., the minimax layers around functions fitted by linear
-regression) belong to a particular class of Interval Predictor Models, for which the reliability is invariant with respect to the distribution of the data.
Software implementations
PyIPM provides an
open-source
Open source is source code that is made freely available for possible modification and redistribution. Products include permission to use the source code, design documents, or content of the product. The open-source model is a decentralized sof ...
Python implementation of the work of Crespo (2015).
OpenCOSSAN provides a Matlab implementation of the work of Crespo (2015).
References
{{Reflist