TY - JOUR
T1 - Asymptotic analysis of robust LASSOs in the presence of noise with large variance
AU - Chen, Xiaohui
AU - Wang, Z. Jane
AU - McKeown, Martin J.
N1 - Funding Information:
Manuscript received September 11, 2009; revised March 22, 2010. Date of current version September 15, 2010. This work was supported in part by a Pacific Alzheimer’s Research Foundation (PARF) Centre Grant Award. X. Chen was supported in part by a University Graduate Fellowship of BC. X. Chen and Z. J. Wang are with the Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4 Canada (e-mail: xiaohuic@ece.ubc.ca; zjanew@ece.ubc.ca). M. J. McKeown is with the Department of Medicine (Neurology), The University of British Columbia, Vancouver, BC V6T 2B5 Canada (e-mail: mmck-eown@interchange.ubc.ca). Communicated by J. Romberg, Associate Editor for Signal Processing. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIT.2010.2059770
PY - 2010/10
Y1 - 2010/10
N2 - In the context of linear regression, the least absolute shrinkage and selection operator (LASSO) is probably the most popular supervised-learning technique proposed to recover sparse signals from high-dimensional measurements. Prior literature has mainly concerned itself with independent, identically distributed noise with moderate variance. In many real applications, however, the measurement errors may have heavy-tailed distributions or suffer from severe outliers, making the LASSO poorly estimate the coefficients due to its sensitivity to large error variance. To address this concern, a robust version of the LASSO is proposed, and the limiting distribution of its estimator is derived. Model selection consistency is established for the proposed robust LASSO under an adaptation procedure of the penalty weight. A parallel asymptotic analysis is derived for the Huberized LASSO, a previously proposed robust LASSO, and it is shown that the Huberized LASSO estimator preserves similar asymptotics even with a Cauchy error distribution. We show that asymptotic variances of the two robust LASSO estimators are stabilized in the presence of large variance noise, compared with the unbounded asymptotic variance of the ordinary LASSO estimator. The asymptotic analysis from the nonstochastic design is extended to the case of random design. Simulations further confirm our theoretical results.
AB - In the context of linear regression, the least absolute shrinkage and selection operator (LASSO) is probably the most popular supervised-learning technique proposed to recover sparse signals from high-dimensional measurements. Prior literature has mainly concerned itself with independent, identically distributed noise with moderate variance. In many real applications, however, the measurement errors may have heavy-tailed distributions or suffer from severe outliers, making the LASSO poorly estimate the coefficients due to its sensitivity to large error variance. To address this concern, a robust version of the LASSO is proposed, and the limiting distribution of its estimator is derived. Model selection consistency is established for the proposed robust LASSO under an adaptation procedure of the penalty weight. A parallel asymptotic analysis is derived for the Huberized LASSO, a previously proposed robust LASSO, and it is shown that the Huberized LASSO estimator preserves similar asymptotics even with a Cauchy error distribution. We show that asymptotic variances of the two robust LASSO estimators are stabilized in the presence of large variance noise, compared with the unbounded asymptotic variance of the ordinary LASSO estimator. The asymptotic analysis from the nonstochastic design is extended to the case of random design. Simulations further confirm our theoretical results.
KW - Asymptotic normality
KW - Huber loss
KW - least absolute shrinkage and selection operator (LASSO)
KW - model selection consistency
KW - random designs
KW - robustness
KW - signal recovery
KW - sparse linear regression
UR - http://www.scopus.com/inward/record.url?scp=77956711576&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77956711576&partnerID=8YFLogxK
U2 - 10.1109/TIT.2010.2059770
DO - 10.1109/TIT.2010.2059770
M3 - Article
AN - SCOPUS:77956711576
SN - 0018-9448
VL - 56
SP - 5131
EP - 5149
JO - IEEE Transactions on Information Theory
JF - IEEE Transactions on Information Theory
IS - 10
M1 - 5571842
ER -