Real time speech driven facial animation using formant analysis

Formant analysis is a technique widely used for speech analysis and synthesis. In this paper, we present a simple, fast and effective method for real time speech driven facial animation based on formant analysis. Speech signal is first processed by a formant analyzer. Since the resulting formants are known to be correlated with vocal tract shape, the formants can be directly mapped to mouth shapes. In addition, median filter and energy modulation is used to smooth the mouth shape sequence. The smoothed mouth shape sequence is used to animate our synthetic 3D head model with synchronized audio. The proposed method is simple and does not rely on contextual information. Thus it is good for real time two-way communication applications. Since the method extracts mouth shapes from acoustic features, it is language independent. In speaker-independent case, the proposed method is also shown to work well.