TY - GEN
T1 - Statistical SVMs for robust detection, supervised learning, and universal classification
AU - Huang, Dayu
AU - Unnikrishnan, Jayakrishnan
AU - Meyn, Sean
AU - Veeravalli, Venugopal Varadachari
AU - Surana, Amit
PY - 2009
Y1 - 2009
N2 - The support vector machine (SVM) has emerged as one of the most popular approaches to classification and supervised learning. It is a flexible approach for solving the problems posed in these areas, but the approach is not easily adapted to noisy data in which absolute discrimination is not possible. We address this issue in this paper by returning to the statistical setting. The main contribution is the introduction of a statistical support vector machine (SSVM) that captures all of the desirable features of the SVM, along with desirable statistical features of the classical likelihood ratio test. In particular, weestablish the following: (i) The SSVM can be designed so that it forms a continuous function of the data, yet also approximates the potentially discontinuous log likelihood ratio test. (ii) Extension to universal detection is developed, in which only one hypothesis is labeled (a semi-supervised learning problem). (iii) The SSVM generalizes the robust hypothesis testing problem based on a moment class. Motivation for the approach and analysis are each based on ideas from information theory. A detailed performance analysis is provided in the special case of i.i.d. observations. This research was partially supported by NSF under grant CCF 07- 29031, by UTRC, Motorola, and by the DARPA ITMANET program. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF, UTRC, Motorola, or DARPA.
AB - The support vector machine (SVM) has emerged as one of the most popular approaches to classification and supervised learning. It is a flexible approach for solving the problems posed in these areas, but the approach is not easily adapted to noisy data in which absolute discrimination is not possible. We address this issue in this paper by returning to the statistical setting. The main contribution is the introduction of a statistical support vector machine (SSVM) that captures all of the desirable features of the SVM, along with desirable statistical features of the classical likelihood ratio test. In particular, weestablish the following: (i) The SSVM can be designed so that it forms a continuous function of the data, yet also approximates the potentially discontinuous log likelihood ratio test. (ii) Extension to universal detection is developed, in which only one hypothesis is labeled (a semi-supervised learning problem). (iii) The SSVM generalizes the robust hypothesis testing problem based on a moment class. Motivation for the approach and analysis are each based on ideas from information theory. A detailed performance analysis is provided in the special case of i.i.d. observations. This research was partially supported by NSF under grant CCF 07- 29031, by UTRC, Motorola, and by the DARPA ITMANET program. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF, UTRC, Motorola, or DARPA.
UR - http://www.scopus.com/inward/record.url?scp=77950669950&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77950669950&partnerID=8YFLogxK
U2 - 10.1109/ITWNIT.2009.5158542
DO - 10.1109/ITWNIT.2009.5158542
M3 - Conference contribution
AN - SCOPUS:77950669950
SN - 9781424445363
T3 - Proceedings - 2009 IEEE Information Theory Workshop on Networking and Information Theory, ITW 2009
SP - 62
EP - 66
BT - Proceedings - 2009 IEEE Information Theory Workshop on Networking and Information Theory, ITW 2009
T2 - 2009 IEEE Information Theory Workshop on Networking and Information Theory, ITW 2009
Y2 - 10 June 2009 through 12 June 2009
ER -