Information theoretic validity of penalized likelihood

Sabyasachi Chatterjee, Andrew Barron

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Building upon past work, which developed information theoretic notions of when a penalized likelihood procedure can be interpreted as codelengths arising from a two stage code and when the statistical risk of the procedure has a redundancy risk bound, we present new results and risk bounds showing that the l1 penalty in Gaussian Graphical Models fits the above story. We also show how the traditional l0 penalty times plus lower order terms which stay bounded on the whole parameter space has a conditional two stage description length interpretation.

Original languageEnglish (US)
Title of host publication2014 IEEE International Symposium on Information Theory, ISIT 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Print)9781479951864
StatePublished - Jan 1 2014
Externally publishedYes
Event2014 IEEE International Symposium on Information Theory, ISIT 2014 - Honolulu, HI, United States
Duration: Jun 29 2014Jul 4 2014

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
ISSN (Print)2157-8095


Other2014 IEEE International Symposium on Information Theory, ISIT 2014
Country/TerritoryUnited States
CityHonolulu, HI

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics


Dive into the research topics of 'Information theoretic validity of penalized likelihood'. Together they form a unique fingerprint.

Cite this