TY - JOUR
T1 - Convex optimization, shape constraints, compound decisions, and empirical bayes rules
AU - Koenker, Roger
AU - Mizera, Ivan
N1 - Funding Information:
Roger Koenker is McKinley Professor of Economics and Professor of Statistics, University of Illinois, Urbana, IL 61801 (E-mail: rkoenker@uiuc.edu). Ivan Mizera is Professor of Statistics, University of Alberta, Edmonton, Alberta T6G 2G1, Canada (E-mail: imizera@ualberta.ca). This research was partially supported by NSF grant SES-11-53548 and the NSERC of Canada. The authors thank Larry Brown for pointing out the relevance of shape constrained density estimation to compound decision problems, and Steve Portnoy for extensive related discussions.
Publisher Copyright:
© 2014 American Statistical Association.
PY - 2014
Y1 - 2014
N2 - Estimation of mixture densities for the classical Gaussian compound decision problem and their associated (empirical) Bayes rules is considered from two new perspectives. The first, motivated by Brown and Greenshtein, introduces a nonparametric maximum likelihood estimator of the mixture density subject to a monotonicity constraint on the resulting Bayes rule. The second, motivated by Jiang and Zhang, proposes a new approach to computing the Kiefer-Wolfowitz nonparametric maximum likelihood estimator for mixtures. In contrast to prior methods for these problems, our new approaches are cast as convex optimization problems that can be efficiently solved by modern interior point methods. In particular, we show that the reformulation of the Kiefer-Wolfowitz estimator as a convex optimization problem reduces the computational effort by several orders of magnitude for typical problems, by comparison to prior EM-algorithm based methods, and thus greatly expands the practical applicability of the resulting methods. Our new procedures are compared with several existing empirical Bayes methods in simulations employing the well-established design of Johnstone and Silverman. Some further comparisons are made based on prediction of baseball batting averages. A Bernoulli mixture application is briefly considered in the penultimate section.
AB - Estimation of mixture densities for the classical Gaussian compound decision problem and their associated (empirical) Bayes rules is considered from two new perspectives. The first, motivated by Brown and Greenshtein, introduces a nonparametric maximum likelihood estimator of the mixture density subject to a monotonicity constraint on the resulting Bayes rule. The second, motivated by Jiang and Zhang, proposes a new approach to computing the Kiefer-Wolfowitz nonparametric maximum likelihood estimator for mixtures. In contrast to prior methods for these problems, our new approaches are cast as convex optimization problems that can be efficiently solved by modern interior point methods. In particular, we show that the reformulation of the Kiefer-Wolfowitz estimator as a convex optimization problem reduces the computational effort by several orders of magnitude for typical problems, by comparison to prior EM-algorithm based methods, and thus greatly expands the practical applicability of the resulting methods. Our new procedures are compared with several existing empirical Bayes methods in simulations employing the well-established design of Johnstone and Silverman. Some further comparisons are made based on prediction of baseball batting averages. A Bernoulli mixture application is briefly considered in the penultimate section.
KW - Empirical Bayes
KW - Kiefer-Wolfowitz maximum likelihood estimator
KW - Mixture models
UR - http://www.scopus.com/inward/record.url?scp=84907503852&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84907503852&partnerID=8YFLogxK
U2 - 10.1080/01621459.2013.869224
DO - 10.1080/01621459.2013.869224
M3 - Article
AN - SCOPUS:84907503852
VL - 109
SP - 674
EP - 685
JO - Journal of the American Statistical Association
JF - Journal of the American Statistical Association
SN - 0162-1459
IS - 506
ER -