A mixed reality system for teaching STEM content using embodied learning and whole-body metaphors

Remo Pillat, Arjun Nagendran, Robb Lindgren

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper describes the development of an MR environment that can be used in teaching STEM (Science, Technology, Engineering, and Mathematics) topics. Specifically we seek to create a space for facilitating whole-body metaphors where learners use the physical movement and positioning of their entire bodies to enact their understanding of complex concepts. A rigorous technical approach comprised of virtual elements, real users, spatial audio, and an integrated sensor network is presented that fulfills the requirements of an embodied learning environment. An algorithm that uses homography-based multi-projector blending is used to create a large, seamless projection on the floor that affords a human-scale interaction environment. To further improve the immersive quality, projectors are strategically overlapped to minimize user shadows on the projected surface. A hybrid sensor solution using a Kinect and a laser scanner is developed that tracks users' physical movements and extracts relevant game parameters such as position and velocity. Requiring no pre-training or props, this tracking setup is adaptable and shows high performance over a wide range of users, from children to adults. An exhibit employing this MR system was field-tested at the Museum of Science and Industry in Tampa, FL.

Original languageEnglish (US)
Title of host publicationProceedings - VRCAI 2012
Subtitle of host publication11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry
Pages295-302
Number of pages8
DOIs
StatePublished - Dec 1 2012
Externally publishedYes
Event11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI 2012 - Singapore, Singapore
Duration: Dec 2 2012Dec 4 2012

Publication series

NameProceedings - VRCAI 2012: 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry

Other

Other11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI 2012
CountrySingapore
CitySingapore
Period12/2/1212/4/12

Keywords

  • educational game
  • embodied learning
  • mixed reality
  • sensor fusion
  • whole-body interaction
  • whole-body metaphors

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications

Fingerprint Dive into the research topics of 'A mixed reality system for teaching STEM content using embodied learning and whole-body metaphors'. Together they form a unique fingerprint.

  • Cite this

    Pillat, R., Nagendran, A., & Lindgren, R. (2012). A mixed reality system for teaching STEM content using embodied learning and whole-body metaphors. In Proceedings - VRCAI 2012: 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (pp. 295-302). (Proceedings - VRCAI 2012: 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry). https://doi.org/10.1145/2407516.2407584