Communication-Efficient Distributed Statistical Inference

Michael I. Jordan, Jason D. Lee, Yun Yang

Research output: Contribution to journalArticle

Abstract

We present a communication-efficient surrogate likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation, and Bayesian inference. For low-dimensional estimation, CSL provably improves upon naive averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax-optimal estimator with controlled communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of Markov chain Monte Carlo (MCMC) algorithms even in a nondistributed setting. We present both theoretical analysis and experiments to explore the properties of the CSL approximation. Supplementary materials for this article are available online.

Original languageEnglish (US)
Pages (from-to)668-681
Number of pages14
JournalJournal of the American Statistical Association
Volume114
Issue number526
DOIs
StatePublished - Apr 3 2019

Keywords

  • Communication efficiency
  • Distributed inference
  • Likelihood approximation

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Communication-Efficient Distributed Statistical Inference'. Together they form a unique fingerprint.

  • Cite this