MMSE dimension

Yihong Wu; Sergio Verdú

doi:10.1109/TIT.2011.2158905

MMSE dimension

Yihong Wu, Sergio Verdú

Research output: Contribution to journal › Article › peer-review

Abstract

If N is standard Gaussian, the minimum mean-square error (MMSE) of estimating a random variable X based on √snrX + N vanishes at least as fast as 1/snr as snr → ∞. We define the MMSE dimension of X as the limit as snr → ∞ of the product of snr and the MMSE. MMSE dimension is also shown to be the asymptotic ratio of nonlinear MMSE to linear MMSE. For discrete, absolutely continuous or mixed distribution we show that MMSE dimension equals Rnyi's information dimension. However, for a class of self-similar singular X (e.g., Cantor distribution), we show that the product of snr and MMSE oscillates around information dimension periodically in snr (dB). We also show that these results extend considerably beyond Gaussian noise under various technical conditions.

Original language	English (US)
Article number	5961853
Pages (from-to)	4857-4879
Number of pages	23
Journal	IEEE Transactions on Information Theory
Volume	57
Issue number	8
DOIs	https://doi.org/10.1109/TIT.2011.2158905
State	Published - Aug 2011
Externally published	Yes

Keywords

Additive noise
Bayesian statistics
Gaussian noise
Rnyi information dimension
high-SNR asymptotics
minimum mean-square error (MMSE)
mutual information
non-Gaussian noise

ASJC Scopus subject areas

Information Systems
Computer Science Applications
Library and Information Sciences

Online availability

10.1109/TIT.2011.2158905

Library availability

Discover UIUC Full Text

Cite this

@article{64399437877446979ed2825f594e3321,

title = "MMSE dimension",

abstract = "If N is standard Gaussian, the minimum mean-square error (MMSE) of estimating a random variable X based on √snrX + N vanishes at least as fast as 1/snr as snr → ∞. We define the MMSE dimension of X as the limit as snr → ∞ of the product of snr and the MMSE. MMSE dimension is also shown to be the asymptotic ratio of nonlinear MMSE to linear MMSE. For discrete, absolutely continuous or mixed distribution we show that MMSE dimension equals Rnyi's information dimension. However, for a class of self-similar singular X (e.g., Cantor distribution), we show that the product of snr and MMSE oscillates around information dimension periodically in snr (dB). We also show that these results extend considerably beyond Gaussian noise under various technical conditions.",

keywords = "Additive noise, Bayesian statistics, Gaussian noise, Rnyi information dimension, high-SNR asymptotics, minimum mean-square error (MMSE), mutual information, non-Gaussian noise",

author = "Yihong Wu and Sergio Verd{\'u}",

note = "Funding Information: Manuscript received December 22, 2009; revised November 11, 2010; accepted February 18, 2011. Date of current version July 29, 2011. This work was supported in part by the National Science Foundation (NSF) under Grant CCF-1016625 and by the Center for Science of Information (CSoI), an NSF Science and Technology Center, under Grant CCF-0939370. The material in this paper was presented in part at the IEEE International Symposium on Information Theory, Austin, TX, June 2010.",

year = "2011",

month = aug,

doi = "10.1109/TIT.2011.2158905",

language = "English (US)",

volume = "57",

pages = "4857--4879",

journal = "IEEE Transactions on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "8",

}

TY - JOUR

T1 - MMSE dimension

AU - Wu, Yihong

AU - Verdú, Sergio

N1 - Funding Information: Manuscript received December 22, 2009; revised November 11, 2010; accepted February 18, 2011. Date of current version July 29, 2011. This work was supported in part by the National Science Foundation (NSF) under Grant CCF-1016625 and by the Center for Science of Information (CSoI), an NSF Science and Technology Center, under Grant CCF-0939370. The material in this paper was presented in part at the IEEE International Symposium on Information Theory, Austin, TX, June 2010.

PY - 2011/8

Y1 - 2011/8

N2 - If N is standard Gaussian, the minimum mean-square error (MMSE) of estimating a random variable X based on √snrX + N vanishes at least as fast as 1/snr as snr → ∞. We define the MMSE dimension of X as the limit as snr → ∞ of the product of snr and the MMSE. MMSE dimension is also shown to be the asymptotic ratio of nonlinear MMSE to linear MMSE. For discrete, absolutely continuous or mixed distribution we show that MMSE dimension equals Rnyi's information dimension. However, for a class of self-similar singular X (e.g., Cantor distribution), we show that the product of snr and MMSE oscillates around information dimension periodically in snr (dB). We also show that these results extend considerably beyond Gaussian noise under various technical conditions.

AB - If N is standard Gaussian, the minimum mean-square error (MMSE) of estimating a random variable X based on √snrX + N vanishes at least as fast as 1/snr as snr → ∞. We define the MMSE dimension of X as the limit as snr → ∞ of the product of snr and the MMSE. MMSE dimension is also shown to be the asymptotic ratio of nonlinear MMSE to linear MMSE. For discrete, absolutely continuous or mixed distribution we show that MMSE dimension equals Rnyi's information dimension. However, for a class of self-similar singular X (e.g., Cantor distribution), we show that the product of snr and MMSE oscillates around information dimension periodically in snr (dB). We also show that these results extend considerably beyond Gaussian noise under various technical conditions.

KW - Additive noise

KW - Bayesian statistics

KW - Gaussian noise

KW - Rnyi information dimension

KW - high-SNR asymptotics

KW - minimum mean-square error (MMSE)

KW - mutual information

KW - non-Gaussian noise

UR - http://www.scopus.com/inward/record.url?scp=79960980740&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79960980740&partnerID=8YFLogxK

U2 - 10.1109/TIT.2011.2158905

DO - 10.1109/TIT.2011.2158905

M3 - Article

AN - SCOPUS:79960980740

SN - 0018-9448

VL - 57

SP - 4857

EP - 4879

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 8

M1 - 5961853

ER -

MMSE dimension

Abstract

Keywords

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this