Real-time humanoid avatar for multimodal human-machine interaction

Yun Fu; Renxiang Li; Thomas S. Huang; Mike Danielsen

doi:10.1109/icme.2007.4284819

Real-time humanoid avatar for multimodal human-machine interaction

Yun Fu, Renxiang Li, Thomas S. Huang, Mike Danielsen

Coordinated Science Lab

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

A novel framework of multimodal human-machine or human-human interaction via real-time humanoid avatar communication is proposed for real-world mobile application. It integrates audio-visual analysis and synthesis modules to realize real-time head tracking, multichannel and runtime animations, visual TTS and real-time viseme detection and rendering. The 3-D avatar provides customized modeling for low-bit rate virtual communication by adopting M3G standard and supports MPEG-4 FAPs. A robust user head tracker and the associated head pose and motion estimation scheme are developed for real-time avatar animation control at remote locations. The framework is recognized as an effective design for realistic industrial products of human-to-human mobile communication.

Original language	English (US)
Title of host publication	Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007
Publisher	IEEE Computer Society
Pages	991-994
Number of pages	4
ISBN (Print)	1424410177, 9781424410170
DOIs	https://doi.org/10.1109/icme.2007.4284819
State	Published - 2007
Event	IEEE International Conference onMultimedia and Expo, ICME 2007 - Beijing, China Duration: Jul 2 2007 → Jul 5 2007

Publication series

Name	Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007

Other

Other	IEEE International Conference onMultimedia and Expo, ICME 2007
Country/Territory	China
City	Beijing
Period	7/2/07 → 7/5/07

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Software

Online availability

10.1109/icme.2007.4284819

Library availability

Discover UIUC Full Text

Cite this

Fu, Y., Li, R., Huang, T. S., & Danielsen, M. (2007). Real-time humanoid avatar for multimodal human-machine interaction. In Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007 (pp. 991-994). Article 4284819 (Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007). IEEE Computer Society. https://doi.org/10.1109/icme.2007.4284819

Real-time humanoid avatar for multimodal human-machine interaction. / Fu, Yun; Li, Renxiang; Huang, Thomas S. et al.
Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007. IEEE Computer Society, 2007. p. 991-994 4284819 (Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Fu, Y, Li, R, Huang, TS & Danielsen, M 2007, Real-time humanoid avatar for multimodal human-machine interaction. in Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007., 4284819, Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007, IEEE Computer Society, pp. 991-994, IEEE International Conference onMultimedia and Expo, ICME 2007, Beijing, China, 7/2/07. https://doi.org/10.1109/icme.2007.4284819

@inproceedings{b29c7fb764d5442aa4c3ea15f5f215ca,

title = "Real-time humanoid avatar for multimodal human-machine interaction",

abstract = "A novel framework of multimodal human-machine or human-human interaction via real-time humanoid avatar communication is proposed for real-world mobile application. It integrates audio-visual analysis and synthesis modules to realize real-time head tracking, multichannel and runtime animations, visual TTS and real-time viseme detection and rendering. The 3-D avatar provides customized modeling for low-bit rate virtual communication by adopting M3G standard and supports MPEG-4 FAPs. A robust user head tracker and the associated head pose and motion estimation scheme are developed for real-time avatar animation control at remote locations. The framework is recognized as an effective design for realistic industrial products of human-to-human mobile communication.",

author = "Yun Fu and Renxiang Li and Huang, {Thomas S.} and Mike Danielsen",

year = "2007",

doi = "10.1109/icme.2007.4284819",

language = "English (US)",

isbn = "1424410177",

series = "Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007",

publisher = "IEEE Computer Society",

pages = "991--994",

booktitle = "Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007",

note = "IEEE International Conference onMultimedia and Expo, ICME 2007 ; Conference date: 02-07-2007 Through 05-07-2007",

}

TY - GEN

T1 - Real-time humanoid avatar for multimodal human-machine interaction

AU - Fu, Yun

AU - Li, Renxiang

AU - Huang, Thomas S.

AU - Danielsen, Mike

PY - 2007

Y1 - 2007

N2 - A novel framework of multimodal human-machine or human-human interaction via real-time humanoid avatar communication is proposed for real-world mobile application. It integrates audio-visual analysis and synthesis modules to realize real-time head tracking, multichannel and runtime animations, visual TTS and real-time viseme detection and rendering. The 3-D avatar provides customized modeling for low-bit rate virtual communication by adopting M3G standard and supports MPEG-4 FAPs. A robust user head tracker and the associated head pose and motion estimation scheme are developed for real-time avatar animation control at remote locations. The framework is recognized as an effective design for realistic industrial products of human-to-human mobile communication.

AB - A novel framework of multimodal human-machine or human-human interaction via real-time humanoid avatar communication is proposed for real-world mobile application. It integrates audio-visual analysis and synthesis modules to realize real-time head tracking, multichannel and runtime animations, visual TTS and real-time viseme detection and rendering. The 3-D avatar provides customized modeling for low-bit rate virtual communication by adopting M3G standard and supports MPEG-4 FAPs. A robust user head tracker and the associated head pose and motion estimation scheme are developed for real-time avatar animation control at remote locations. The framework is recognized as an effective design for realistic industrial products of human-to-human mobile communication.

UR - http://www.scopus.com/inward/record.url?scp=46449119257&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=46449119257&partnerID=8YFLogxK

U2 - 10.1109/icme.2007.4284819

DO - 10.1109/icme.2007.4284819

M3 - Conference contribution

AN - SCOPUS:46449119257

SN - 1424410177

SN - 9781424410170

T3 - Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007

SP - 991

EP - 994

BT - Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, ICME 2007

PB - IEEE Computer Society

T2 - IEEE International Conference onMultimedia and Expo, ICME 2007

Y2 - 2 July 2007 through 5 July 2007

ER -

Real-time humanoid avatar for multimodal human-machine interaction

Abstract

Publication series

Other

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this