Mutual information and posterior estimates in channels of exponential family type

Maxim Raginsky, Todd P. Coleman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recently, there has been a lot of interest in the connections between information-theoretic and estimation-theoretic properties of various noisy channel models. For example, Guo, Shamai, and Verdú have shown that mutual information in Gaussian channels is related in a simple way to minimum meansquare error, regardless of the input distribution. In this paper, we consider the class of E-type channels, i.e., additive noise channels induced by an exponential family of distributions. We derive several differential and integral representations of the mutual information and the posterior information gain that are valid for any E-type channel regardless of input distribution. Next, we establish an extremal property of E-type channels that connects the Bayesian concept of a posterior estimate with a natural rate-distortion problem and makes precise a qualitative observation made by Mitter and Newton concerning informationtheoretic properties of optimal nonlinear filters. Finally, we indicate how our results may be used to show monotonicity of the mutual information in E-type channels as a function of a "channel quality" parameter without assuming stochastic degradation.

Original languageEnglish (US)
Title of host publication2009 IEEE Information Theory Workshop, ITW 2009
Pages399-403
Number of pages5
DOIs
StatePublished - Dec 7 2009
Externally publishedYes
Event2009 IEEE Information Theory Workshop, ITW 2009 - Taormina, Sicily, Italy
Duration: Oct 11 2009Oct 16 2009

Publication series

Name2009 IEEE Information Theory Workshop, ITW 2009

Other

Other2009 IEEE Information Theory Workshop, ITW 2009
CountryItaly
CityTaormina, Sicily
Period10/11/0910/16/09

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Networks and Communications
  • Information Systems
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Mutual information and posterior estimates in channels of exponential family type'. Together they form a unique fingerprint.

Cite this