TY - GEN
T1 - On additive-combinatorial affine inequalities for Shannon entropy and differential entropy
AU - Makkuva, Ashok Vardhan
AU - Wu, Yihong
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2016/8/10
Y1 - 2016/8/10
N2 - To be considered for the 2016 IEEE Jack Keil Wolf ISIT Student Paper Award. This paper addresses the question of to what extent do discrete entropy inequalities for weighted sums of independent group-valued random variables continue to hold for differential entropies. We show that all balanced affine inequalities (with the sum of coefficients being zero) of Shannon entropy extend to differential entropy; conversely, any affine inequality for differential entropy must be balanced. In particular, this result recovers recently proved differential entropy inequalities by Kontoyiannis and Madiman [1] from their discrete counterparts due to Tao [2] in a unified manner. Our proof relies on a result of Rényi which relates the Shannon entropy of a finely discretized random variable to its differential entropy and also helps in establishing the entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum.
AB - To be considered for the 2016 IEEE Jack Keil Wolf ISIT Student Paper Award. This paper addresses the question of to what extent do discrete entropy inequalities for weighted sums of independent group-valued random variables continue to hold for differential entropies. We show that all balanced affine inequalities (with the sum of coefficients being zero) of Shannon entropy extend to differential entropy; conversely, any affine inequality for differential entropy must be balanced. In particular, this result recovers recently proved differential entropy inequalities by Kontoyiannis and Madiman [1] from their discrete counterparts due to Tao [2] in a unified manner. Our proof relies on a result of Rényi which relates the Shannon entropy of a finely discretized random variable to its differential entropy and also helps in establishing the entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum.
UR - http://www.scopus.com/inward/record.url?scp=84986000749&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84986000749&partnerID=8YFLogxK
U2 - 10.1109/ISIT.2016.7541460
DO - 10.1109/ISIT.2016.7541460
M3 - Conference contribution
AN - SCOPUS:84986000749
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 1053
EP - 1057
BT - Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2016 IEEE International Symposium on Information Theory, ISIT 2016
Y2 - 10 July 2016 through 15 July 2016
ER -