Speech/gesture interface to a visual computing environment for molecular biologists

Rajeev Sharma, Thomas S. Huang, Vladimir I. Pavlovic, Yunxin Zhao, Zion Lo, Stephen Chu, Klaus Schulten, Andrew Dalke, Jim Phillips, Michael Zeller, William Humphrey

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Recent progress in 3-D, immersive display and virtual reality (VR) technologies has made possible many exciting applications, for example interactive visualization of complex scientific data. To fully exploit this potential there is a need for "natural" interfaces that allow the manipulation of such displays without cumbersome attachments. In this paper we describe the use of visual hand gesture analysis and speech recognition for developing a speech/gesture interface for controlling a 3-D display. The interface enhances an existing application, VMD, which is a VR visual computing environment far molecular biologists. The free hand gestures are used for manipulating the 3-D graphical display together with a set of speech commands. We describe the visual gesture analysis and the speech analysis techniques used in developing this interface. The dual modality of speech/gesture is found to greatly aid the interaction capability.

Original languageEnglish (US)
Title of host publicationTrack C
Subtitle of host publicationApplications and Robotic Systems
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Print)081867282X, 9780818672828
StatePublished - 1996
Event13th International Conference on Pattern Recognition, ICPR 1996 - Vienna, Austria
Duration: Aug 25 1996Aug 29 1996

Publication series

NameProceedings - International Conference on Pattern Recognition
ISSN (Print)1051-4651


Other13th International Conference on Pattern Recognition, ICPR 1996

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Speech/gesture interface to a visual computing environment for molecular biologists'. Together they form a unique fingerprint.

Cite this