Relevance Vector Machine

In mathematics, a Relevance Vector Machine (RVM) is a machine learning technique that uses Bayesian inference to obtain parsimonious solutions for regression and probabilistic classification.
The RVM has an identical functional form to the support vector machine, but provides probabilistic classification.

It is actually equivalent to a Gaussian process model with covariance function:
:$k(\mathbf,\mathbf) = \sum_^N \frac \varphi(\mathbf,\mathbf_j)\varphi(\mathbf',\mathbf_j)$
where $\varphi$ is the kernel function (usually Gaussian), $\alpha_j$ are the variances of the prior on the weight vector
$w \sim N(0,\alpha^I)$, and $\mathbf_1,\ldots,\mathbf_N$ are the input vectors of the training set.

Compared to that of support vector machines (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations). However RVMs use an expectation maximization (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard sequential minimal optimization (SMO)-based algorithms employed by SVMs, which are guaranteed to find a global optimum (of the convex problem).

The relevance vector machine was patented in the United States by Microsoft (patent expired September 4, 2019).

See also

* Kernel trick
* Platt scaling: turns an SVM into a probability model

References

{{reflist

Software

dlib
C++ Library

The Kernel-Machine Library


R package for binary classification

scikit-rvm

fast-scikit-rvmrvm tutorial

External links

Tipping's webpage on Sparse Bayesian Models and the RVMA Tutorial on RVM by Tristan FletcherApplied tutorial on RVMComparison of RVM and SVM

Classification algorithms
Kernel methods for machine learning
Nonparametric Bayesian statistics