Consistent model selection for marginal generalized additive model for correlated data

Lan Xue, Annie Qu, Jianhui Zhou

Research output: Contribution to journalArticlepeer-review


We consider the generalized additive model when responses from the same cluster are correlated. Incorporating correlation in the estimation of nonparametric components for the generalized additive model is important because it improves estimation efficiency and increases statistical power for model selection. In our setting, there is no specified likelihood function for the generalized additive model, because the outcomes could be nonnormal and discrete, which makes estimation and model selection very challenging problems. We propose consistent estimation and model selection that incorporate the correlation structure. We establish an asymptotic property with L 2-norm consistency for the nonparametric components, which achieves the optimal rate of convergence. In addition, the proposed model selection strategy is able to select the correct generalized additive model consistently. That is, with probability approaching to 1, the estimators for the zero function components converge to 0 almost surely. We illustrate our method using numerical studies with both continuous and binary responses, along with a real data application of binary periodontal data. Supplemental materials including technical details are available online.

Original languageEnglish (US)
Pages (from-to)1518-1530
Number of pages13
JournalJournal of the American Statistical Association
Issue number492
StatePublished - Dec 2010


  • L-norm consistency
  • Model selection
  • Nonparametric function
  • Oracle property
  • Polynomial spline
  • Quadratic inference function
  • SCAD

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty


Dive into the research topics of 'Consistent model selection for marginal generalized additive model for correlated data'. Together they form a unique fingerprint.

Cite this