Bioinformatics BI_J0008

Title : An Efficient Alternative to SVM Based Recursive Feature Elimination with Applications in Natural Language Processing and Bioinformatics
Author : Justin Bedo, Conrad Sanderson and Adam Kowalczyk
Year Publish : 2006
Place of Publish: Springer Berlin / Heidelberg
Abstract :

The SVM based Recursive Feature Elimination (RFE-SVM) algorithm is a popular technique for feature selection, used in natural language processing and bioinformatics. Recently it was demonstrated that a small regularisation constant C can considerably improve the performance of RFE-SVM on microarray datasets. In this paper we show that further improvements are possible if the explicitly computable limit C ?0 is used. We prove that in this limit most forms of SVM and ridge regression classifiers scaled by the factor \frac1C converge to a centroid classifier. As this classifier can be used directly for feature ranking, in the limit we can avoid the computationally demanding recursion and convex optimisation in RFE-SVM. Comparisons on two text based author verification tasks and on three genomic microarray classification tasks indicate that this straightforward method can surprisingly obtain comparable (at times superior) performance and is about an order of magnitude faster.