L1 Estimation: On the Optimality of Linear Estimators

Leighton P. Barnes, Alex Dytso, Jingbo Liu, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

Abstract

Consider the problem of estimating a random variable X from noisy observations Y = X+ Z , where Z is standard normal, under the L1 fidelity criterion. It is well known that the optimal Bayesian estimator in this setting is the conditional median. This work shows that the only prior distribution on X that induces linearity in the conditional median is Gaussian. Along the way, several other results are presented. In particular, it is demonstrated that if the conditional distribution P_{X|Y=y} is symmetric for all y, then X must follow a Gaussian distribution. Additionally, we consider other Lp losses and observe the following phenomenon: for p in [1,2] , Gaussian is the only prior distribution that induces a linear optimal Bayesian estimator, and for p /in (2,∞) , infinitely many prior distributions on X can induce linearity. Finally, extensions are provided to encompass noise models leading to conditional distributions from certain exponential families.

Original languageEnglish (US)
Pages (from-to)8026-8039
Number of pages14
JournalIEEE Transactions on Information Theory
Volume70
Issue number11
DOIs
StatePublished - 2024
Externally publishedYes

Keywords

  • additive noise
  • conditional mean
  • Conditional median
  • exponential family
  • Fourier transform
  • Gaussian noise
  • input distribution
  • mean absolute error
  • mean square error
  • poisson distribution
  • posterior probability
  • random variables
  • tempered distributions

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Fingerprint

Dive into the research topics of 'L1 Estimation: On the Optimality of Linear Estimators'. Together they form a unique fingerprint.

Cite this