TY - GEN
T1 - Multi-stream confidence analysis for audio-visual affect recognition
AU - Zeng, Zhihong
AU - Tu, Juin
AU - Liu, Ming
AU - Huang, Thomas S.
PY - 2005
Y1 - 2005
N2 - Changes in a speaker's emotion are a fundamental component in human communication. Some emotions motivate human actions while others add deeper meaning and richness to human interactions. In this paper, we explore the development of a computing algorithm that uses audio and visual sensors to recognize a speaker's affective state. Within the framework of Multi-stream Hidden Markov Model (MHMM), we analyze audio and visual observations to detect 11 cognitive/emotive states. We investigate the use of individual modality confidence measures as a means of estimating weights when combining likelihoods in the audio-visual decision fusion. Person-independent experimental results from 20 subjects in 660 sequences suggest that the use of stream exponents estimated on training data results in classification accuracy improvement of audio-visual affect recognition.
AB - Changes in a speaker's emotion are a fundamental component in human communication. Some emotions motivate human actions while others add deeper meaning and richness to human interactions. In this paper, we explore the development of a computing algorithm that uses audio and visual sensors to recognize a speaker's affective state. Within the framework of Multi-stream Hidden Markov Model (MHMM), we analyze audio and visual observations to detect 11 cognitive/emotive states. We investigate the use of individual modality confidence measures as a means of estimating weights when combining likelihoods in the audio-visual decision fusion. Person-independent experimental results from 20 subjects in 660 sequences suggest that the use of stream exponents estimated on training data results in classification accuracy improvement of audio-visual affect recognition.
UR - http://www.scopus.com/inward/record.url?scp=33646795673&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=33646795673&partnerID=8YFLogxK
U2 - 10.1007/11573548_123
DO - 10.1007/11573548_123
M3 - Conference contribution
AN - SCOPUS:33646795673
SN - 3540296212
SN - 9783540296218
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 964
EP - 971
BT - Affective Computing and Intelligent Interaction - First International Conference, ACII 2005, Proceedings
T2 - 1st International Conference on ffective Computing and Intelligent Interaction, ACII 2005
Y2 - 22 October 2005 through 24 October 2005
ER -