Estimation in Gaussian noise: Properties of the minimum mean-square error

Dongning Guo, Yihong Wu, Shlomo Shamai, Sergio Verdú

Research output: Contribution to journalArticle

Abstract

Consider the minimum mean-square error (MMSE) of estimating an arbitrary random variable from its observation contaminated by Gaussian noise. The MMSE can be regarded as a function of the signal-to-noise ratio (SNR) as well as a functional of the input distribution (of the random variable to be estimated). It is shown that the MMSE is concave in the input distribution at any given SNR. For a given input distribution, the MMSE is found to be infinitely differentiable at all positive SNR, and in fact a real analytic function in SNR under mild conditions. The key to these regularity results is that the posterior distribution conditioned on the observation through Gaussian channels always decays at least as quickly as some Gaussian density. Furthermore, simple expressions for the first three derivatives of the MMSE with respect to the SNR are obtained. It is also shown that, as functions of the SNR, the curves for the MMSE of a Gaussian input and that of a non-Gaussian input cross at most once over all SNRs. These properties lead to simple proofs of the facts that Gaussian inputs achieve both the secrecy capacity of scalar Gaussian wiretap channels and the capacity of scalar Gaussian broadcast channels, as well as a simple proof of the entropy power inequality in the special case where one of the variables is Gaussian.

Original languageEnglish (US)
Article number5730572
Pages (from-to)2371-2385
Number of pages15
JournalIEEE Transactions on Information Theory
Volume57
Issue number4
DOIs
StatePublished - Apr 1 2011

Keywords

  • Entropy
  • Gaussian broadcast channel
  • Gaussian noise
  • Gaussian wiretap channel
  • estimation
  • minimum mean square error (MMSE)
  • mutual information

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Fingerprint Dive into the research topics of 'Estimation in Gaussian noise: Properties of the minimum mean-square error'. Together they form a unique fingerprint.

  • Cite this