TY - GEN
T1 - Real-time conversion from a single 2D face image to a 3D text-driven emotive audio-visual avatar
AU - Tang, Hao
AU - Hu, Yuxiao
AU - Fu, Yun
AU - Hasegawa-Johnson, Mark
AU - Huang, Thomas S.
PY - 2008
Y1 - 2008
N2 - In this paper, we propose a complete pipeline of efficient and lowcost techniques to construct a realistic 3D text-driven emotive audio-visual avatar from a single 2D frontal-view face image of any person on the fly. This real-time conversion is achieved through three steps. First, a personalized 3D face model is built based on the 2D face image using a fully automatic 3D face shape and texture reconstruction framework. Second, using standard MPEG-4 FAPs (Facial Animation Parameters), the face model is animated by the viseme and expression channels and is complemented by the visual prosody channel that controls head, eye and eyelid movements. Finally, the facial animation is combined and synchronized with the emotive synthetic speech generated by incorporating an emotion transformer into a Festival-MBROLA text to neutral speech synthesizer.
AB - In this paper, we propose a complete pipeline of efficient and lowcost techniques to construct a realistic 3D text-driven emotive audio-visual avatar from a single 2D frontal-view face image of any person on the fly. This real-time conversion is achieved through three steps. First, a personalized 3D face model is built based on the 2D face image using a fully automatic 3D face shape and texture reconstruction framework. Second, using standard MPEG-4 FAPs (Facial Animation Parameters), the face model is animated by the viseme and expression channels and is complemented by the visual prosody channel that controls head, eye and eyelid movements. Finally, the facial animation is combined and synchronized with the emotive synthetic speech generated by incorporating an emotion transformer into a Festival-MBROLA text to neutral speech synthesizer.
KW - 3D face reconstruction
KW - Facial animation
KW - MPEG-4
KW - Text-to-speech
KW - Viseme
UR - http://www.scopus.com/inward/record.url?scp=54049142575&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=54049142575&partnerID=8YFLogxK
U2 - 10.1109/ICME.2008.4607657
DO - 10.1109/ICME.2008.4607657
M3 - Conference contribution
AN - SCOPUS:54049142575
SN - 9781424425716
T3 - 2008 IEEE International Conference on Multimedia and Expo, ICME 2008 - Proceedings
SP - 1205
EP - 1208
BT - 2008 IEEE International Conference on Multimedia and Expo, ICME 2008 - Proceedings
T2 - 2008 IEEE International Conference on Multimedia and Expo, ICME 2008
Y2 - 23 June 2008 through 26 June 2008
ER -