TY - JOUR
T1 - Model Enforcement
T2 - A Unified Feature Transformation Framework for Classification and Recognition
AU - Omar, Mohamed Kamal
AU - Hasegawa-Johnson, Mark
N1 - Manuscript received March 15, 2003; revised September 29, 2003. This work was supported by the National Science Foundation under Award 0 132 900. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The associate editor coordinating the review of this paper and approving it for publication was Prof. Gregori Vazquez.
PY - 2004/10
Y1 - 2004/10
N2 - Bayesian classifiers rely on models of the a priori and class-conditional feature distributions; the classifier is trained by optimizing these models to best represent features observed in a training corpus according to certain criterion. In many problems of interest, the true class-conditional feature probability density function (PDF) is not a member of the set of PDFs the classifier can represent. Previous research has shown that the effect of this problem may be reduced either by improving the models or by transforming the features used in the classifier. This paper addresses this model mismatch problem in statistical identification, classification, and recognition systems. We formulate the problem as the problem of minimizing the relative entropy, which is also known as the Kullback-Leibler distance, between the true conditional PDF and the hypothesized probabilistic model. Based on this formulation, we provide a computationally efficient solution to the problem based on volume-preserving maps; existing linear transform designs are shown to be special cases of the proposed solution. Using this result, we propose the symplectic maximum likelihood transform (SMLT), which is a nonlinear volume-preserving extension of the maximum likelihood linear transform (MLLT). This approach has many applications in statistical modeling, classification, and recognition. We apply it to the maximum likelihood estimation (MLE) of the joint PDF of order statistics and show a significant increase in the likelihood for the same number of parameters. We provide also phoneme recognition experiments that show recognition accuracy improvement compared with using the baseline Mel-Frequency Cepstrum Coefficient (MFCC) features or using MLLT. We present an iterative algorithm to jointly estimate the parameters of the symplectic map and the probabilistic model for both applications.
AB - Bayesian classifiers rely on models of the a priori and class-conditional feature distributions; the classifier is trained by optimizing these models to best represent features observed in a training corpus according to certain criterion. In many problems of interest, the true class-conditional feature probability density function (PDF) is not a member of the set of PDFs the classifier can represent. Previous research has shown that the effect of this problem may be reduced either by improving the models or by transforming the features used in the classifier. This paper addresses this model mismatch problem in statistical identification, classification, and recognition systems. We formulate the problem as the problem of minimizing the relative entropy, which is also known as the Kullback-Leibler distance, between the true conditional PDF and the hypothesized probabilistic model. Based on this formulation, we provide a computationally efficient solution to the problem based on volume-preserving maps; existing linear transform designs are shown to be special cases of the proposed solution. Using this result, we propose the symplectic maximum likelihood transform (SMLT), which is a nonlinear volume-preserving extension of the maximum likelihood linear transform (MLLT). This approach has many applications in statistical modeling, classification, and recognition. We apply it to the maximum likelihood estimation (MLE) of the joint PDF of order statistics and show a significant increase in the likelihood for the same number of parameters. We provide also phoneme recognition experiments that show recognition accuracy improvement compared with using the baseline Mel-Frequency Cepstrum Coefficient (MFCC) features or using MLLT. We present an iterative algorithm to jointly estimate the parameters of the symplectic map and the probabilistic model for both applications.
UR - http://www.scopus.com/inward/record.url?scp=85008042402&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85008042402&partnerID=8YFLogxK
U2 - 10.1109/TSP.2004.834344
DO - 10.1109/TSP.2004.834344
M3 - Article
AN - SCOPUS:85008042402
SN - 1053-587X
VL - 52
SP - 2701
EP - 2710
JO - IEEE Transactions on Signal Processing
JF - IEEE Transactions on Signal Processing
IS - 10
ER -