Audio-visual affective expression recognition

Thomas S Huang, Zhihong Zeng

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Automatic affective expression recognition has attracted more and more attention of researchers from different disciplines, which will significantly contribute to a new paradigm for human computer interaction (affect-sensitive interfaces, socially intelligent environments) and advance the research in the affect-related fields including psychology, psychiatry, and education. Multimodal information integration is a process that enables human to assess affective states robustly and flexibly. In order to understand the richness and subtleness of human emotion behavior, the computer should be able to integrate information from multiple sensors. We introduce in this paper our efforts toward machine understanding of audio-visual affective behavior, based on both deliberate and spontaneous displays. Some promising methods are presented to integrate information from both audio and visual modalities. Our experiments show the advantage of audio-visual fusion in affective expression recognition over audio-only or visual-only approaches.

Original languageEnglish (US)
Title of host publicationMIPPR 2007
Subtitle of host publicationPattern Recognition and Computer Vision
StatePublished - Dec 1 2007
EventMIPPR 2007: Pattern Recognition and Computer Vision - Wuhan, China
Duration: Nov 15 2007Nov 17 2007

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
ISSN (Print)0277-786X


OtherMIPPR 2007: Pattern Recognition and Computer Vision


  • Affect recognition
  • Affective computing
  • Emotion recognition
  • Human computing
  • Multimodal human computer interaction

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Audio-visual affective expression recognition'. Together they form a unique fingerprint.

Cite this