Minimum redundancy feature selection is an
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
frequently used in a method to accurately identify characteristics of
gene
In biology, the word gene (from , ; "... Wilhelm Johannsen coined the word gene to describe the Mendelian units of heredity..." meaning ''generation'' or ''birth'' or ''gender'') can have several different meanings. The Mendelian gene is a b ...
s and
phenotype
In genetics, the phenotype () is the set of observable characteristics or traits of an organism. The term covers the organism's morphology or physical form and structure, its developmental processes, its biochemical and physiological pr ...
s and narrow down their relevance and is usually described in its pairing with relevant feature selection as ''Minimum Redundancy Maximum Relevance'' (mRMR).
''
Feature selection
In machine learning and statistics, feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables, predictors) for use in model construc ...
'', one of the basic problems in
pattern recognition
Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphics ...
and
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
, identifies subsets of data that are relevant to the parameters used and is normally called ''
Maximum Relevance''. These subsets often contain material which is relevant but redundant and mRMR attempts to address this problem by removing those redundant subsets. mRMR has a variety of applications in many areas such as
cancer diagnosis and
speech recognition
Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers with the ...
.
Features can be selected in many different ways. One scheme is to select features that
correlate
In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistic ...
strongest to the
classification variable. This has been called maximum-relevance selection. Many
heuristic algorithm
In mathematical optimization and computer science, heuristic (from Greek εὑρίσκω "I find, discover") is a technique designed for solving a problem more quickly when classic methods are too slow for finding an approximate solution, or whe ...
s can be used, such as the sequential forward, backward, or floating selections.
On the other hand, features can be selected to be mutually far away from each other while still having "high" correlation to the classification variable. This scheme, termed as ''Minimum Redundancy Maximum Relevance'' (mRMR) selection has been found to be more powerful than the maximum relevance selection.
As a special case, the "correlation" can be replaced by the
statistical dependency between variables.
Mutual information
In probability theory and information theory, the mutual information (MI) of two random variables is a measure of the mutual dependence between the two variables. More specifically, it quantifies the " amount of information" (in units such ...
can be used to quantify the dependency. In this case, it is shown that mRMR is an approximation to maximizing the dependency between the
joint distribution
Given two random variables that are defined on the same probability space, the joint probability distribution is the corresponding probability distribution on all possible pairs of outputs. The joint distribution can just as well be considered ...
of the selected features and the classification variable.
Studies have tried different measures for redundancy and relevance measures. A recent study compared several measures within the context of biomedical images.
[Auffarth, B., Lopez, M., Cerquides, J. (2010). Comparison of redundancy and relevance measures for feature selection in tissue classification of CT images. Advances in Data Mining. Applications and Theoretical Aspects. p. 248--262. Springer. http://www.csc.kth.se/~auffarth/publications/redrel.pdf]
References
{{reflist
External links
* Peng, H.C., Long, F., and Ding, C.,
Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 8, pp. 1226–1238, 2005.
* Chris Ding and Hanchuan Peng,
Minimum Redundancy Feature Selection from Microarray Gene Expression Data. 2nd IEEE Computer Society Bioinformatics Conference (CSB 2003), 11–14 August 2003, Stanford, CA, USA. Pages 523–529.
Penglab mRMR
Machine learning algorithms
zh:最小冗余特征选择