On prior distributions and approximate inference for structured variables

Oluwasanmi Oluseye Koyejo, Rajiv Khanna, Joydeep Ghosh, Russell A. Poldrack

Research output: Contribution to journalConference article

Abstract

We present a general framework for constructing prior distributions with structured variables. The prior is defined as the information projection of a base distribution onto distributions supported on the constraint set of interest. In cases where this projection is intractable, we propose a family of parameterized approximations indexed by subsets of the domain. We further analyze the special case of sparse structure. While the optimal prior is intractable in general, we show that approximate inference using convex subsets is tractable, and is equivalent to maximizing a submodular function subject to cardinality constraints. As a result, inference using greedy forward selection provably achieves within a factor of (1-1/e) of the optimal objective value. Our work is motivated by the predictive modeling of high-dimensional functional neuroimaging data. For this task, we employ the Gaussian base distribution induced by local partial correlations and consider the design of priors to capture the domain knowledge of sparse support. Experimental results on simulated data and high dimensional neuroimaging data show the effectiveness of our approach in terms of support recovery and predictive accuracy.

Original languageEnglish (US)
Pages (from-to)676-684
Number of pages9
JournalAdvances in Neural Information Processing Systems
Volume1
Issue numberJanuary
StatePublished - Jan 1 2014
Externally publishedYes
Event28th Annual Conference on Neural Information Processing Systems 2014, NIPS 2014 - Montreal, Canada
Duration: Dec 8 2014Dec 13 2014

Fingerprint

Functional neuroimaging
Neuroimaging
Recovery

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

On prior distributions and approximate inference for structured variables. / Koyejo, Oluwasanmi Oluseye; Khanna, Rajiv; Ghosh, Joydeep; Poldrack, Russell A.

In: Advances in Neural Information Processing Systems, Vol. 1, No. January, 01.01.2014, p. 676-684.

Research output: Contribution to journalConference article

Koyejo, Oluwasanmi Oluseye ; Khanna, Rajiv ; Ghosh, Joydeep ; Poldrack, Russell A. / On prior distributions and approximate inference for structured variables. In: Advances in Neural Information Processing Systems. 2014 ; Vol. 1, No. January. pp. 676-684.
@article{a81712a17a88461f855fdcdb3cf133ca,
title = "On prior distributions and approximate inference for structured variables",
abstract = "We present a general framework for constructing prior distributions with structured variables. The prior is defined as the information projection of a base distribution onto distributions supported on the constraint set of interest. In cases where this projection is intractable, we propose a family of parameterized approximations indexed by subsets of the domain. We further analyze the special case of sparse structure. While the optimal prior is intractable in general, we show that approximate inference using convex subsets is tractable, and is equivalent to maximizing a submodular function subject to cardinality constraints. As a result, inference using greedy forward selection provably achieves within a factor of (1-1/e) of the optimal objective value. Our work is motivated by the predictive modeling of high-dimensional functional neuroimaging data. For this task, we employ the Gaussian base distribution induced by local partial correlations and consider the design of priors to capture the domain knowledge of sparse support. Experimental results on simulated data and high dimensional neuroimaging data show the effectiveness of our approach in terms of support recovery and predictive accuracy.",
author = "Koyejo, {Oluwasanmi Oluseye} and Rajiv Khanna and Joydeep Ghosh and Poldrack, {Russell A.}",
year = "2014",
month = "1",
day = "1",
language = "English (US)",
volume = "1",
pages = "676--684",
journal = "Advances in Neural Information Processing Systems",
issn = "1049-5258",
number = "January",

}

TY - JOUR

T1 - On prior distributions and approximate inference for structured variables

AU - Koyejo, Oluwasanmi Oluseye

AU - Khanna, Rajiv

AU - Ghosh, Joydeep

AU - Poldrack, Russell A.

PY - 2014/1/1

Y1 - 2014/1/1

N2 - We present a general framework for constructing prior distributions with structured variables. The prior is defined as the information projection of a base distribution onto distributions supported on the constraint set of interest. In cases where this projection is intractable, we propose a family of parameterized approximations indexed by subsets of the domain. We further analyze the special case of sparse structure. While the optimal prior is intractable in general, we show that approximate inference using convex subsets is tractable, and is equivalent to maximizing a submodular function subject to cardinality constraints. As a result, inference using greedy forward selection provably achieves within a factor of (1-1/e) of the optimal objective value. Our work is motivated by the predictive modeling of high-dimensional functional neuroimaging data. For this task, we employ the Gaussian base distribution induced by local partial correlations and consider the design of priors to capture the domain knowledge of sparse support. Experimental results on simulated data and high dimensional neuroimaging data show the effectiveness of our approach in terms of support recovery and predictive accuracy.

AB - We present a general framework for constructing prior distributions with structured variables. The prior is defined as the information projection of a base distribution onto distributions supported on the constraint set of interest. In cases where this projection is intractable, we propose a family of parameterized approximations indexed by subsets of the domain. We further analyze the special case of sparse structure. While the optimal prior is intractable in general, we show that approximate inference using convex subsets is tractable, and is equivalent to maximizing a submodular function subject to cardinality constraints. As a result, inference using greedy forward selection provably achieves within a factor of (1-1/e) of the optimal objective value. Our work is motivated by the predictive modeling of high-dimensional functional neuroimaging data. For this task, we employ the Gaussian base distribution induced by local partial correlations and consider the design of priors to capture the domain knowledge of sparse support. Experimental results on simulated data and high dimensional neuroimaging data show the effectiveness of our approach in terms of support recovery and predictive accuracy.

UR - http://www.scopus.com/inward/record.url?scp=84937918949&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84937918949&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:84937918949

VL - 1

SP - 676

EP - 684

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

IS - January

ER -