Nonparametric variable selection and its application to additive models

Zhenghui Feng, Lu Lin, Ruoqing Zhu, Lixing Zhu

Research output: Contribution to journalArticle

Abstract

Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

Original languageEnglish (US)
JournalAnnals of the Institute of Statistical Mathematics
DOIs
StatePublished - Jan 1 2019

Fingerprint

Additive Models
Variable Selection
Nonparametric Model
Approximation
Two-stage Procedure
Adaptive Estimation
Multivariate Regression
Curse of Dimensionality
Nonparametric Regression
Inaccurate
Smoothness
Regression Model
Objective function
Simulation Study
Model

Keywords

  • Adaptive estimation
  • Nonparametric additive model
  • Nonparametric regression
  • Variable selection

ASJC Scopus subject areas

  • Statistics and Probability

Cite this

Nonparametric variable selection and its application to additive models. / Feng, Zhenghui; Lin, Lu; Zhu, Ruoqing; Zhu, Lixing.

In: Annals of the Institute of Statistical Mathematics, 01.01.2019.

Research output: Contribution to journalArticle

@article{90389810f2d74d20a1a04ac94cdcbf19,
title = "Nonparametric variable selection and its application to additive models",
abstract = "Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.",
keywords = "Adaptive estimation, Nonparametric additive model, Nonparametric regression, Variable selection",
author = "Zhenghui Feng and Lu Lin and Ruoqing Zhu and Lixing Zhu",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s10463-019-00711-9",
language = "English (US)",
journal = "Annals of the Institute of Statistical Mathematics",
issn = "0020-3157",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - Nonparametric variable selection and its application to additive models

AU - Feng, Zhenghui

AU - Lin, Lu

AU - Zhu, Ruoqing

AU - Zhu, Lixing

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

AB - Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

KW - Adaptive estimation

KW - Nonparametric additive model

KW - Nonparametric regression

KW - Variable selection

UR - http://www.scopus.com/inward/record.url?scp=85064272623&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064272623&partnerID=8YFLogxK

U2 - 10.1007/s10463-019-00711-9

DO - 10.1007/s10463-019-00711-9

M3 - Article

AN - SCOPUS:85064272623

JO - Annals of the Institute of Statistical Mathematics

JF - Annals of the Institute of Statistical Mathematics

SN - 0020-3157

ER -