Multi-stage convex relaxation for feature selection

Research output: Contribution to journalArticlepeer-review

Abstract

A number of recent work studied the effectiveness of feature selection using Lasso. It is known that under the restricted isometry properties (RIP), Lasso does not generally lead to the exact recovery of the set of nonzero coefficients, due to the looseness of convex relaxation. This paper considers the feature selection property of nonconvex regularization, where the solution is given by a multi-stage convex relaxation scheme. The nonconvex regularizer requires two tuning parameters (compared to one tuning parameter for Lasso). Although the method is more complex than Lasso, we show that under appropriate conditions including the dependence of a tuning parameter on the support set size, the local solution obtained by this procedure recovers the set of nonzero coefficients without suffering from the bias of Lasso relaxation, which complements parameter estimation results of this procedure in (J.Mach. Learn. Res. 11 (2011) 1087-1107).

Original languageEnglish (US)
Pages (from-to)2277-2293
Number of pages17
JournalBernoulli
Volume19
Issue number5 B
DOIs
StatePublished - Nov 2013
Externally publishedYes

Keywords

  • Multi-stage convex relaxation
  • Non-convex penalty
  • Variable selection

ASJC Scopus subject areas

  • Statistics and Probability

Fingerprint

Dive into the research topics of 'Multi-stage convex relaxation for feature selection'. Together they form a unique fingerprint.

Cite this