A Fused Hidden Markov Model with Application to Bimodal Speech Processing

Hao Pan; Stephen E Levinson; Thomas S Huang; Zhi-Pei Liang

doi:10.1109/TSP.2003.822353

A Fused Hidden Markov Model with Application to Bimodal Speech Processing

Hao Pan, Stephen E Levinson, Thomas S Huang, Zhi-Pei Liang

Research output: Contribution to journal › Article

Abstract

This paper presents a novel fused hidden Markov model (fused HMM) for integrating tightly coupled time series, such as audio and visual features of speech. In this model, the time series are first modeled by two conventional HMMs separately. The resulting HMMs are then fused together using a probabilistic fusion model, which is optimal according to the maximum entropy principle and a maximum mutual information criterion. Simulations and bimodal speaker verification experiments show that the proposed model can significantly reduce the recognition errors in noiseless or noisy environments.

Original language	English (US)
Pages (from-to)	573-581
Number of pages	9
Journal	IEEE Transactions on Signal Processing
Volume	52
Issue number	3
DOIs	https://doi.org/10.1109/TSP.2003.822353
State	Published - Mar 1 2004

Fingerprint

Speech processing

Hidden Markov models

Time series

Entropy

Fusion reactions

Experiments

Keywords

Bimodal speech processing
Hidden Markov model
Information fusion

ASJC Scopus subject areas

Electrical and Electronic Engineering
Signal Processing

Cite this

Pan, H., Levinson, S. E., Huang, T. S., & Liang, Z-P. (2004). A Fused Hidden Markov Model with Application to Bimodal Speech Processing. IEEE Transactions on Signal Processing, 52(3), 573-581. https://doi.org/10.1109/TSP.2003.822353

A Fused Hidden Markov Model with Application to Bimodal Speech Processing. / Pan, Hao; Levinson, Stephen E ; Huang, Thomas S ; Liang, Zhi-Pei.

In: IEEE Transactions on Signal Processing, Vol. 52, No. 3, 01.03.2004, p. 573-581.

Research output: Contribution to journal › Article

Pan, H, Levinson, SE , Huang, TS & Liang, Z-P 2004, 'A Fused Hidden Markov Model with Application to Bimodal Speech Processing', IEEE Transactions on Signal Processing, vol. 52, no. 3, pp. 573-581. https://doi.org/10.1109/TSP.2003.822353

Pan H, Levinson SE , Huang TS , Liang Z-P. A Fused Hidden Markov Model with Application to Bimodal Speech Processing. IEEE Transactions on Signal Processing. 2004 Mar 1;52(3):573-581. https://doi.org/10.1109/TSP.2003.822353

Pan, Hao ; Levinson, Stephen E ; Huang, Thomas S ; Liang, Zhi-Pei. / A Fused Hidden Markov Model with Application to Bimodal Speech Processing. In: IEEE Transactions on Signal Processing. 2004 ; Vol. 52, No. 3. pp. 573-581.

@article{41121a88ac544f53a92b2e93b09320ca,

title = "A Fused Hidden Markov Model with Application to Bimodal Speech Processing",

abstract = "This paper presents a novel fused hidden Markov model (fused HMM) for integrating tightly coupled time series, such as audio and visual features of speech. In this model, the time series are first modeled by two conventional HMMs separately. The resulting HMMs are then fused together using a probabilistic fusion model, which is optimal according to the maximum entropy principle and a maximum mutual information criterion. Simulations and bimodal speaker verification experiments show that the proposed model can significantly reduce the recognition errors in noiseless or noisy environments.",

keywords = "Bimodal speech processing, Hidden Markov model, Information fusion",

author = "Hao Pan and Levinson, {Stephen E} and Huang, {Thomas S} and Zhi-Pei Liang",

year = "2004",

month = "3",

day = "1",

doi = "10.1109/TSP.2003.822353",

language = "English (US)",

volume = "52",

pages = "573--581",

journal = "IEEE Transactions on Signal Processing",

issn = "1053-587X",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "3",

}

TY - JOUR

T1 - A Fused Hidden Markov Model with Application to Bimodal Speech Processing

AU - Pan, Hao

AU - Levinson, Stephen E

AU - Huang, Thomas S

AU - Liang, Zhi-Pei

PY - 2004/3/1

Y1 - 2004/3/1

N2 - This paper presents a novel fused hidden Markov model (fused HMM) for integrating tightly coupled time series, such as audio and visual features of speech. In this model, the time series are first modeled by two conventional HMMs separately. The resulting HMMs are then fused together using a probabilistic fusion model, which is optimal according to the maximum entropy principle and a maximum mutual information criterion. Simulations and bimodal speaker verification experiments show that the proposed model can significantly reduce the recognition errors in noiseless or noisy environments.

AB - This paper presents a novel fused hidden Markov model (fused HMM) for integrating tightly coupled time series, such as audio and visual features of speech. In this model, the time series are first modeled by two conventional HMMs separately. The resulting HMMs are then fused together using a probabilistic fusion model, which is optimal according to the maximum entropy principle and a maximum mutual information criterion. Simulations and bimodal speaker verification experiments show that the proposed model can significantly reduce the recognition errors in noiseless or noisy environments.

KW - Bimodal speech processing

KW - Hidden Markov model

KW - Information fusion

UR - http://www.scopus.com/inward/record.url?scp=1542303714&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=1542303714&partnerID=8YFLogxK

U2 - 10.1109/TSP.2003.822353

DO - 10.1109/TSP.2003.822353

M3 - Article

AN - SCOPUS:1542303714

VL - 52

SP - 573

EP - 581

JO - IEEE Transactions on Signal Processing

JF - IEEE Transactions on Signal Processing

SN - 1053-587X

IS - 3

ER -

Access to Document

10.1109/TSP.2003.822353