Real time speech driven facial animation using formant analysis

Zhen Wen, Pengyu Hong, T. S. Huang

Research output: Contribution to journalArticlepeer-review


Formant analysis is a technique widely used for speech analysis and synthesis. In this paper, we present a simple, fast and effective method for real time speech driven facial animation based on formant analysis. Speech signal is first processed by a formant analyzer. Since the resulting formants are known to be correlated with vocal tract shape, the formants can be directly mapped to mouth shapes. In addition, median filter and energy modulation is used to smooth the mouth shape sequence. The smoothed mouth shape sequence is used to animate our synthetic 3D head model with synchronized audio. The proposed method is simple and does not rely on contextual information. Thus it is good for real time two-way communication applications. Since the method extracts mouth shapes from acoustic features, it is language independent. In speaker-independent case, the proposed method is also shown to work well.

Original languageEnglish (US)
Article number1237847
Pages (from-to)817-820
Number of pages4
JournalProceedings - IEEE International Conference on Multimedia and Expo
StatePublished - 2001

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications


Dive into the research topics of 'Real time speech driven facial animation using formant analysis'. Together they form a unique fingerprint.

Cite this