Examining modalities of embodied interactions and the effects on learning from a collaborative science simulation

Advances in motion tracking and gesture recognition technologies have opened the door to new lines of investigation around leveraging physical body movements to enhance the learning of critical STEM concepts. While significant progress has been made, there remain several challenges to supporting productive embodied learning with technology. This paper describes the design, implementation, and initial results from a study comparing two approaches to whole-body gestural input framing: personalized gestures or prescribed gestures based on their alignment with the target content. Two participants engage collaboratively with two science simulations linked by the “rates of change” NGSS crosscutting concept. Data collected shows significant individual and dyad gains in learning after engaging with the simulations, and video transcript samples show the effective facilitation of collaborative discourse.