Design requirements for using embodied learning and whole-body metaphors in a mixed reality simulation game

This paper describes the development of a mixed reality (MR) environment that can be used in teaching STEM (Science, Technology, Engineering, and Mathematics) topics. Specifically we seek to create a space for facilitating whole-body metaphors, where learners use the physical movement and positioning of their entire bodies to enact their understanding of complex concepts. A technical approach is presented that fulfills the requirements of this type of embodied learning environment. Multi-projector blending is used to create a large, seamless, and shadow-minimizing projection on the floor that affords a human-scale interaction environment. A hybrid sensor solution using a Kinect and a laser scanner is developed that tracks users' physical movements. Requiring no pre-training or props, this tracking setup is adaptable and shows high performance over a wide range of users, from children to adults. To illustrate the application of the MR system to a real-world learning scenario, an example implementation that was designed to teach physics concepts to middle school kids is described.