Generalized fisher score for feature selection

Quanquan Gu, Zhenhui Li, Jiawei Han

Research output: Contribution to conferencePaper

Abstract

Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims at finding an subset of features, which maximize the lower bound of traditional Fisher score. The resulting feature selection problem is a mixed integer programming, which can be reformulated as a quadratically constrained linear programming (QCLP). It is solved by cutting plane algorithm, in each iteration of which a multiple kernel learning problem is solved alternatively by multivariate ridge regression and projected gradient descent. Experiments on benchmark data sets indicate that the proposed method outperforms Fisher score as well as many other state-of-the-art feature selection methods.

Original languageEnglish (US)
Pages266-273
Number of pages8
StatePublished - Sep 29 2011
Event27th Conference on Uncertainty in Artificial Intelligence, UAI 2011 - Barcelona, Spain
Duration: Jul 14 2011Jul 17 2011

Other

Other27th Conference on Uncertainty in Artificial Intelligence, UAI 2011
CountrySpain
CityBarcelona
Period7/14/117/17/11

Fingerprint

Feature Selection
Feature extraction
Integer programming
Linear programming
Projected Gradient
Cutting Plane Algorithm
Ridge Regression
Subset
Multivariate Regression
Gradient Descent
Mixed Integer Programming
Maximise
Benchmark
Lower bound
kernel
Iteration
Experiments
Experiment

ASJC Scopus subject areas

  • Artificial Intelligence
  • Applied Mathematics

Cite this

Gu, Q., Li, Z., & Han, J. (2011). Generalized fisher score for feature selection. 266-273. Paper presented at 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, Barcelona, Spain.

Generalized fisher score for feature selection. / Gu, Quanquan; Li, Zhenhui; Han, Jiawei.

2011. 266-273 Paper presented at 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, Barcelona, Spain.

Research output: Contribution to conferencePaper

Gu, Q, Li, Z & Han, J 2011, 'Generalized fisher score for feature selection', Paper presented at 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, Barcelona, Spain, 7/14/11 - 7/17/11 pp. 266-273.
Gu Q, Li Z, Han J. Generalized fisher score for feature selection. 2011. Paper presented at 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, Barcelona, Spain.
Gu, Quanquan ; Li, Zhenhui ; Han, Jiawei. / Generalized fisher score for feature selection. Paper presented at 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011, Barcelona, Spain.8 p.
@conference{8a30feb400d541ba96abeb9579275a71,
title = "Generalized fisher score for feature selection",
abstract = "Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims at finding an subset of features, which maximize the lower bound of traditional Fisher score. The resulting feature selection problem is a mixed integer programming, which can be reformulated as a quadratically constrained linear programming (QCLP). It is solved by cutting plane algorithm, in each iteration of which a multiple kernel learning problem is solved alternatively by multivariate ridge regression and projected gradient descent. Experiments on benchmark data sets indicate that the proposed method outperforms Fisher score as well as many other state-of-the-art feature selection methods.",
author = "Quanquan Gu and Zhenhui Li and Jiawei Han",
year = "2011",
month = "9",
day = "29",
language = "English (US)",
pages = "266--273",
note = "27th Conference on Uncertainty in Artificial Intelligence, UAI 2011 ; Conference date: 14-07-2011 Through 17-07-2011",

}

TY - CONF

T1 - Generalized fisher score for feature selection

AU - Gu, Quanquan

AU - Li, Zhenhui

AU - Han, Jiawei

PY - 2011/9/29

Y1 - 2011/9/29

N2 - Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims at finding an subset of features, which maximize the lower bound of traditional Fisher score. The resulting feature selection problem is a mixed integer programming, which can be reformulated as a quadratically constrained linear programming (QCLP). It is solved by cutting plane algorithm, in each iteration of which a multiple kernel learning problem is solved alternatively by multivariate ridge regression and projected gradient descent. Experiments on benchmark data sets indicate that the proposed method outperforms Fisher score as well as many other state-of-the-art feature selection methods.

AB - Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims at finding an subset of features, which maximize the lower bound of traditional Fisher score. The resulting feature selection problem is a mixed integer programming, which can be reformulated as a quadratically constrained linear programming (QCLP). It is solved by cutting plane algorithm, in each iteration of which a multiple kernel learning problem is solved alternatively by multivariate ridge regression and projected gradient descent. Experiments on benchmark data sets indicate that the proposed method outperforms Fisher score as well as many other state-of-the-art feature selection methods.

UR - http://www.scopus.com/inward/record.url?scp=80053144252&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053144252&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:80053144252

SP - 266

EP - 273

ER -