The challenge of using research-based instructional strategies: Insights from an effectiveness study of the intrinsic motivation course conversion

As awareness grows about how difficult it is to increase the use of research-based instructional strategies (RBIS) among engineering faculty, we must develop a better understanding of not only what RBIS work, but also how they might fail when used by inexperienced practitioners. We conducted an effectiveness study to explore whether the course design and targeted outcomes of one RBIS - an intrinsic motivation course conversion - were maintained after control of a converted course was turned over from the designing research team to a new set of instructors. The study was conducted as an amplified secondary analysis, combining data collected from multiple offerings of the converted course. The study triangulates data from course artifacts such as course syllabi and assignments, motivation surveys from 352 students, and post-course interviews with 27 students. Analysis of course artifacts revealed a high fidelity of implementation. Analysis of surveys and interviews revealed that small changes in the team formation process (a shift of project selection from before teams are formed to after teams are formed) for students' projects significantly undermined students' intrinsic motivation to learn - a core targeted outcome of this RBIS. We found that small changes in implementation can undermine the targeted outcomes for the intrinsic motivation course conversion. The documentation and promotion of RBIS must carefully consider how to support faculty through what may be the inevitable shortcomings in achieving the targeted outcomes of RBIS as the result of small changes in implementation.