Supervised learning of quantizer codebooks by information loss minimization

Research output: Contribution to journalArticlepeer-review


This paper proposes a technique for jointly quantizing continuous features and the posterior distributions of their class labels based on minimizing empirical information loss such that the quantizer index of a given feature vector approximates a sufficient statistic for its class label. Informally, the quantized representation retains as much information as possible for classifying the feature vector correctly. We derive an alternating minimization procedure for simultaneously learning codebooks in the euclidean feature space and in the simplex of posterior class distributions. The resulting quantizer can be used to encode unlabeled points outside the training set and to predict their posterior class distributions, and has an elegant interpretation in terms of lossless source coding. The proposed method is validated on synthetic and real data sets and is applied to two diverse problems: Learning discriminative visual vocabularies for bag-of-features image classification and image segmentation.

Original languageEnglish (US)
Pages (from-to)1294-1309
Number of pages16
JournalIEEE transactions on pattern analysis and machine intelligence
Issue number7
StatePublished - 2009
Externally publishedYes


  • Clustering
  • Computer vision
  • Information theory
  • Pattern recognition
  • Quantization
  • Scene analysis
  • Segmentation

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Supervised learning of quantizer codebooks by information loss minimization'. Together they form a unique fingerprint.

Cite this