TY - GEN

T1 - Statistical SVMs for robust detection, supervised learning, and universal classification

AU - Huang, Dayu

AU - Unnikrishnan, Jayakrishnan

AU - Meyn, Sean

AU - Veeravalli, Venugopal Varadachari

AU - Surana, Amit

PY - 2009/12/1

Y1 - 2009/12/1

N2 - The support vector machine (SVM) has emerged as one of the most popular approaches to classification and supervised learning. It is a flexible approach for solving the problems posed in these areas, but the approach is not easily adapted to noisy data in which absolute discrimination is not possible. We address this issue in this paper by returning to the statistical setting. The main contribution is the introduction of a statistical support vector machine (SSVM) that captures all of the desirable features of the SVM, along with desirable statistical features of the classical likelihood ratio test. In particular, weestablish the following: (i) The SSVM can be designed so that it forms a continuous function of the data, yet also approximates the potentially discontinuous log likelihood ratio test. (ii) Extension to universal detection is developed, in which only one hypothesis is labeled (a semi-supervised learning problem). (iii) The SSVM generalizes the robust hypothesis testing problem based on a moment class. Motivation for the approach and analysis are each based on ideas from information theory. A detailed performance analysis is provided in the special case of i.i.d. observations. This research was partially supported by NSF under grant CCF 07- 29031, by UTRC, Motorola, and by the DARPA ITMANET program. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF, UTRC, Motorola, or DARPA.

AB - The support vector machine (SVM) has emerged as one of the most popular approaches to classification and supervised learning. It is a flexible approach for solving the problems posed in these areas, but the approach is not easily adapted to noisy data in which absolute discrimination is not possible. We address this issue in this paper by returning to the statistical setting. The main contribution is the introduction of a statistical support vector machine (SSVM) that captures all of the desirable features of the SVM, along with desirable statistical features of the classical likelihood ratio test. In particular, weestablish the following: (i) The SSVM can be designed so that it forms a continuous function of the data, yet also approximates the potentially discontinuous log likelihood ratio test. (ii) Extension to universal detection is developed, in which only one hypothesis is labeled (a semi-supervised learning problem). (iii) The SSVM generalizes the robust hypothesis testing problem based on a moment class. Motivation for the approach and analysis are each based on ideas from information theory. A detailed performance analysis is provided in the special case of i.i.d. observations. This research was partially supported by NSF under grant CCF 07- 29031, by UTRC, Motorola, and by the DARPA ITMANET program. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF, UTRC, Motorola, or DARPA.

UR - http://www.scopus.com/inward/record.url?scp=77950669950&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77950669950&partnerID=8YFLogxK

U2 - 10.1109/ITWNIT.2009.5158542

DO - 10.1109/ITWNIT.2009.5158542

M3 - Conference contribution

AN - SCOPUS:77950669950

SN - 9781424445363

T3 - Proceedings - 2009 IEEE Information Theory Workshop on Networking and Information Theory, ITW 2009

SP - 62

EP - 66

BT - Proceedings - 2009 IEEE Information Theory Workshop on Networking and Information Theory, ITW 2009

T2 - 2009 IEEE Information Theory Workshop on Networking and Information Theory, ITW 2009

Y2 - 10 June 2009 through 12 June 2009

ER -