Considerations for the Design of Gesture-Augmented Learning Environments

A new genre of learning technologies is emerging that integrates computer simulations with physical or “embodied” interactions such as hand gestures. While this genre presents new opportunities for innovative digital environments that physically engage learners, there is very little guidance on how to design these environments to optimize learning. This chapter presents considerations specifically for the design of gesture-augmented learning environments. Design considerations are discussed in three main areas related to (1) what gestural interactions are used, (2) constraints of the learning environment, and (3) what social and contextual supports are offered. The term considerations is used rather than principles or guidelines to highlight the real tradeoffs and legitimate decisions to be made when designing gesture-based technologies for learning. These considerations are illustrated with detailed examples from a project that implements students’ gestures as the primary method of interaction with digital science simulations. Although the examples specifically pertain to learning in science, the considerations are framed such that they can be applied to a broad range of domains.