Multimodal speech and audio user interfaces for K-12 outreach

Mark Hasegawa-Johnson, Camille Goudeseune, Jennifer Cole, Hank Kaczmarski, Heejin Kim, Sarah King, Timothy Mahrt, Jui Ting Huang, Xiaodan Zhuang, Kai Hsiang Lin, Harsh Vardhan Sharma, Zhen Li, Thomas S. Huang

Research output: Contribution to conferencePaper

Abstract

Elementary school children have short attention spans. This paper describes three multimodal speech and audio user interfaces that captured and held the attention of a few dozen elementary-school and high-school children during the course of a two-day university open house. The Speech Recognition Game demonstrated an isolated word recognizer with a rapidly-won game, in which children were challenged to get ten words in a row correctly recognized. The Audio Easter Egg Hunt demonstrated our timeliner multimedia analytics platform with a faster-than-real-time search through orchestral music for audio anomalies (cuckoo clocks, motorcycles, etc). Finally, at the Intonation Station, children had to pick the pitch contour that would help a friendly troll to successfully hunt dragons in the city of Champaign. Results suggest that competition, collaboration, and other forms of social interaction may motivate children more than prizes.

Original languageEnglish (US)
Pages526-531
Number of pages6
StatePublished - Dec 1 2011
EventAsia-Pacific Signal and Information Processing Association Annual Summit and Conference 2011, APSIPA ASC 2011 - Xi'an, China
Duration: Oct 18 2011Oct 21 2011

Other

OtherAsia-Pacific Signal and Information Processing Association Annual Summit and Conference 2011, APSIPA ASC 2011
CountryChina
CityXi'an
Period10/18/1110/21/11

    Fingerprint

ASJC Scopus subject areas

  • Information Systems
  • Signal Processing

Cite this

Hasegawa-Johnson, M., Goudeseune, C., Cole, J., Kaczmarski, H., Kim, H., King, S., Mahrt, T., Huang, J. T., Zhuang, X., Lin, K. H., Sharma, H. V., Li, Z., & Huang, T. S. (2011). Multimodal speech and audio user interfaces for K-12 outreach. 526-531. Paper presented at Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2011, APSIPA ASC 2011, Xi'an, China.