TY - JOUR
T1 - On the existence and characterization of the maxent distribution under general moment inequality constraints
AU - Ishwar, Prakash
AU - Moulin, Pierre
N1 - Manuscript received November 14, 2001; revised December 16, 2004. This work was supported by the National Science Foundation under Grants MIP-9707633 and CDA-9624396. P. Ishwar was with the Beckman Institute and the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA. He is now with the Department of Electrical and Computer Engineering, Boston University, Boston, MA 02215 USA (e-mail: [email protected]). P. Moulin is with the Beckman Institute, Coordinated Science Laboratory, and the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA (e-mail: [email protected]). Communicated by A. Kavcˇić, Associate Editor for Detection and Estimation. Digital Object Identifier 10.1109/TIT.2005.853317
PY - 2005/9
Y1 - 2005/9
N2 - A broad set of sufficient conditions that guarantees the existence of the maximum entropy (maxent) distribution consistent with specified bounds on certain generalized moments is derived. Most results in the literature are either focused on the minimum cross-entropy distribution or apply only to distributions with a bounded-volume support or address only equality constraints. The results of this work hold for general moment inequality constraints for probability distributions with possibly unbounded support, and the technical conditions are explicitly on the underlying generalized moment functions. An analytical characterization of the maxent distribution is also derived using results from the theory of constrained optimization in infinite-dimensional normed linear spaces. Several auxiliary results of independent interest pertaining to certain properties of convex coercive functions are also presented.
AB - A broad set of sufficient conditions that guarantees the existence of the maximum entropy (maxent) distribution consistent with specified bounds on certain generalized moments is derived. Most results in the literature are either focused on the minimum cross-entropy distribution or apply only to distributions with a bounded-volume support or address only equality constraints. The results of this work hold for general moment inequality constraints for probability distributions with possibly unbounded support, and the technical conditions are explicitly on the underlying generalized moment functions. An analytical characterization of the maxent distribution is also derived using results from the theory of constrained optimization in infinite-dimensional normed linear spaces. Several auxiliary results of independent interest pertaining to certain properties of convex coercive functions are also presented.
KW - Coercive functions
KW - Constrained optimization
KW - Convex analysis
KW - Cross-entropy
KW - Differential entropy
KW - Maximum entropy methods
UR - http://www.scopus.com/inward/record.url?scp=26444478560&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=26444478560&partnerID=8YFLogxK
U2 - 10.1109/TIT.2005.853317
DO - 10.1109/TIT.2005.853317
M3 - Article
AN - SCOPUS:26444478560
SN - 0018-9448
VL - 51
SP - 3322
EP - 3333
JO - IEEE Transactions on Information Theory
JF - IEEE Transactions on Information Theory
IS - 9
ER -