A Fused Hidden Markov Model with Application to Bimodal Speech Processing

Hao Pan, Stephen E Levinson, Thomas S Huang, Zhi-Pei Liang

Research output: Contribution to journalArticlepeer-review


This paper presents a novel fused hidden Markov model (fused HMM) for integrating tightly coupled time series, such as audio and visual features of speech. In this model, the time series are first modeled by two conventional HMMs separately. The resulting HMMs are then fused together using a probabilistic fusion model, which is optimal according to the maximum entropy principle and a maximum mutual information criterion. Simulations and bimodal speaker verification experiments show that the proposed model can significantly reduce the recognition errors in noiseless or noisy environments.

Original languageEnglish (US)
Pages (from-to)573-581
Number of pages9
JournalIEEE Transactions on Signal Processing
Issue number3
StatePublished - Mar 2004


  • Bimodal speech processing
  • Hidden Markov model
  • Information fusion

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing


Dive into the research topics of 'A Fused Hidden Markov Model with Application to Bimodal Speech Processing'. Together they form a unique fingerprint.

Cite this