Variable selection in high-dimensional varying-coefficient models with global optimality

Lan Xue, Peiyong Qu

Research output: Contribution to journalArticlepeer-review

Abstract

The varying-coefficient model is flexible and powerful for modeling the dynamic changes of regression coefficients. It is important to identify significant covariates associated with response variables, especially for high-dimensional settings where the number of covariates can be larger than the sample size. We consider model selection in the high-dimensional setting and adopt difference convex programming to approximate the L0 penalty, and we investigate the global optimality properties of the varying-coefficient estimator. The challenge of the variable selection problem here is that the dimension of the nonparametric form for the varying-coefficient modeling could be infinite, in addition to dealing with the high-dimensional linear covariates. We show that the proposed varying-coefficient estimator is consistent, enjoys the oracle property and achieves an optimal convergence rate for the non-zero nonparametric components for high-dimensional data. Our simulations and numerical examples indicate that the difference convex algorithm is efficient using the coordinate decent algorithm, and is able to select the true model at a higher frequency than the least absolute shrinkage and selection operator (LASSO), the adaptive LASSO and the smoothly clipped absolute deviation (SCAD) approaches.

Original languageEnglish (US)
Pages (from-to)1973-1998
Number of pages26
JournalJournal of Machine Learning Research
Volume13
StatePublished - Jun 2012

Keywords

  • Coordinate decent algorithm
  • Difference convex programming
  • L0- regularization
  • Large-p small-n
  • Model selection
  • Nonparametric function
  • Oracle property
  • Truncated L1 penalty

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Statistics and Probability
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Variable selection in high-dimensional varying-coefficient models with global optimality'. Together they form a unique fingerprint.

Cite this