TY - GEN
T1 - On Bayesian bounds
AU - Banerjee, Arindam
N1 - Copyright:
Copyright 2011 Elsevier B.V., All rights reserved.
PY - 2006
Y1 - 2006
N2 - We show that several important Bayesian bounds studied in machine learning, both in the batch as well as the online setting, arise by an application of a simple compression lemma. In particular, we derive (i) PAC-Bayesian bounds in the batch setting, (ii) Bayesian log-loss bounds and (iii) Bayesian bounded-loss bounds in the online setting using the compression lemma. Although every setting has different semantics for prior, posterior and loss, we show that the core bound argument is the same. The paper simplifies our understanding of several important and apparently disparate results, as well as brings to light a powerful tool for developing similar arguments for other methods.
AB - We show that several important Bayesian bounds studied in machine learning, both in the batch as well as the online setting, arise by an application of a simple compression lemma. In particular, we derive (i) PAC-Bayesian bounds in the batch setting, (ii) Bayesian log-loss bounds and (iii) Bayesian bounded-loss bounds in the online setting using the compression lemma. Although every setting has different semantics for prior, posterior and loss, we show that the core bound argument is the same. The paper simplifies our understanding of several important and apparently disparate results, as well as brings to light a powerful tool for developing similar arguments for other methods.
UR - http://www.scopus.com/inward/record.url?scp=34250706326&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=34250706326&partnerID=8YFLogxK
U2 - 10.1145/1143844.1143855
DO - 10.1145/1143844.1143855
M3 - Conference contribution
AN - SCOPUS:33749262485
SN - 1595933832
SN - 9781595933836
T3 - ACM International Conference Proceeding Series
SP - 81
EP - 88
BT - ACM International Conference Proceeding Series - Proceedings of the 23rd International Conference on Machine Learning, ICML 2006
T2 - 23rd International Conference on Machine Learning, ICML 2006
Y2 - 25 June 2006 through 29 June 2006
ER -