Compression of noisy signals with information bottlenecks

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider a novel approach to the information bottleneck problem where the goal is to perform compression of a noisy signal, while retaining a significant amount of information about a correlated auxiliary signal. To facilitate analysis, we cast compression with side information as an optimization problem involving an information measure, which for jointly Gaussian random variables equals the classical mutual information. We provide closed form expressions for locally optimal linear compression schemes; in particular, we show that the optimal solutions are of the form of the product of an arbitrary full-rank matrix and the left eigenvectors corresponding to smallest eigenvalues of a matrix related to the signals' covariance matrices. In addition, we study the influence of the sparsity level of the Bernoulli-Gaussian noise on the compression rate. We also highlight the similarities and differences between the noisy bottleneck problem and canonical correlation analysis (CCA), as well as the Gaussian information bottleneck problem.

Original languageEnglish (US)
Title of host publication2013 IEEE Information Theory Workshop, ITW 2013
DOIs
StatePublished - Dec 1 2013
Event2013 IEEE Information Theory Workshop, ITW 2013 - Seville, Spain
Duration: Sep 9 2013Sep 13 2013

Publication series

Name2013 IEEE Information Theory Workshop, ITW 2013

Other

Other2013 IEEE Information Theory Workshop, ITW 2013
CountrySpain
CitySeville
Period9/9/139/13/13

ASJC Scopus subject areas

  • Information Systems

Fingerprint Dive into the research topics of 'Compression of noisy signals with information bottlenecks'. Together they form a unique fingerprint.

Cite this