Information projection and approximate inference for structured sparse variables

Rajiv Khanna, Joydeep Ghosh, Russell Poldrack, Oluwasanmi Oluseye Koyejo

Research output: Contribution to conferencePaper

Abstract

Approximate inference via information projection has been recently introduced as a general-purpose technique for efficient probabilistic inference given sparse variables. This manuscript goes beyond classical sparsity by proposing efficient algorithms for approximate inference via information projection that are applicable to any structure on the set of variables that admits enumeration using matroid or knapsack constraints. Further, leveraging recent advances in submodular optimization, we provide an efficient greedy algorithm with strong optimization-theoretic guarantees. The class of probabilistic models that can be expressed in this way is quite broad and, as we show, includes group sparse regression, group sparse principal components analysis and sparse collective matrix factorization, among others. Empirical results on simulated data and high dimensional neuroimaging data highlight the superior performance of the information projection approach as compared to established baselines for a range of probabilistic models.

Original languageEnglish (US)
StatePublished - Jan 1 2017
Event20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017 - Fort Lauderdale, United States
Duration: Apr 20 2017Apr 22 2017

Conference

Conference20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017
CountryUnited States
CityFort Lauderdale
Period4/20/174/22/17

Fingerprint

Projection
Neuroimaging
Factorization
Probabilistic Model
Principal component analysis
Efficient Algorithms
Probabilistic Inference
Knapsack
Matrix Factorization
Optimization
Greedy Algorithm
Matroid
Sparsity
Enumeration
Principal Component Analysis
Baseline
High-dimensional
Regression
Statistical Models
Range of data

ASJC Scopus subject areas

  • Artificial Intelligence
  • Statistics and Probability

Cite this

Khanna, R., Ghosh, J., Poldrack, R., & Koyejo, O. O. (2017). Information projection and approximate inference for structured sparse variables. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.

Information projection and approximate inference for structured sparse variables. / Khanna, Rajiv; Ghosh, Joydeep; Poldrack, Russell; Koyejo, Oluwasanmi Oluseye.

2017. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.

Research output: Contribution to conferencePaper

Khanna, R, Ghosh, J, Poldrack, R & Koyejo, OO 2017, 'Information projection and approximate inference for structured sparse variables' Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States, 4/20/17 - 4/22/17, .
Khanna R, Ghosh J, Poldrack R, Koyejo OO. Information projection and approximate inference for structured sparse variables. 2017. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.
Khanna, Rajiv ; Ghosh, Joydeep ; Poldrack, Russell ; Koyejo, Oluwasanmi Oluseye. / Information projection and approximate inference for structured sparse variables. Paper presented at 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, United States.
@conference{92e9faa21a6d4c9aba5ca2851f68fbed,
title = "Information projection and approximate inference for structured sparse variables",
abstract = "Approximate inference via information projection has been recently introduced as a general-purpose technique for efficient probabilistic inference given sparse variables. This manuscript goes beyond classical sparsity by proposing efficient algorithms for approximate inference via information projection that are applicable to any structure on the set of variables that admits enumeration using matroid or knapsack constraints. Further, leveraging recent advances in submodular optimization, we provide an efficient greedy algorithm with strong optimization-theoretic guarantees. The class of probabilistic models that can be expressed in this way is quite broad and, as we show, includes group sparse regression, group sparse principal components analysis and sparse collective matrix factorization, among others. Empirical results on simulated data and high dimensional neuroimaging data highlight the superior performance of the information projection approach as compared to established baselines for a range of probabilistic models.",
author = "Rajiv Khanna and Joydeep Ghosh and Russell Poldrack and Koyejo, {Oluwasanmi Oluseye}",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
note = "20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017 ; Conference date: 20-04-2017 Through 22-04-2017",

}

TY - CONF

T1 - Information projection and approximate inference for structured sparse variables

AU - Khanna, Rajiv

AU - Ghosh, Joydeep

AU - Poldrack, Russell

AU - Koyejo, Oluwasanmi Oluseye

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Approximate inference via information projection has been recently introduced as a general-purpose technique for efficient probabilistic inference given sparse variables. This manuscript goes beyond classical sparsity by proposing efficient algorithms for approximate inference via information projection that are applicable to any structure on the set of variables that admits enumeration using matroid or knapsack constraints. Further, leveraging recent advances in submodular optimization, we provide an efficient greedy algorithm with strong optimization-theoretic guarantees. The class of probabilistic models that can be expressed in this way is quite broad and, as we show, includes group sparse regression, group sparse principal components analysis and sparse collective matrix factorization, among others. Empirical results on simulated data and high dimensional neuroimaging data highlight the superior performance of the information projection approach as compared to established baselines for a range of probabilistic models.

AB - Approximate inference via information projection has been recently introduced as a general-purpose technique for efficient probabilistic inference given sparse variables. This manuscript goes beyond classical sparsity by proposing efficient algorithms for approximate inference via information projection that are applicable to any structure on the set of variables that admits enumeration using matroid or knapsack constraints. Further, leveraging recent advances in submodular optimization, we provide an efficient greedy algorithm with strong optimization-theoretic guarantees. The class of probabilistic models that can be expressed in this way is quite broad and, as we show, includes group sparse regression, group sparse principal components analysis and sparse collective matrix factorization, among others. Empirical results on simulated data and high dimensional neuroimaging data highlight the superior performance of the information projection approach as compared to established baselines for a range of probabilistic models.

UR - http://www.scopus.com/inward/record.url?scp=85067557282&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067557282&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85067557282

ER -