On Students' Ability to Resolve their own Tracing Errors through Code Execution

When students attempt to solve code-tracing problems, sometimes students make mistakes as they read code that get in the way of correctly solving the problem. In this paper, we explore the degree to which students can correct their misunderstandings by executing the provided code on a computer. Specifically, we performed a qualitative between-subjects think-aloud study to compare what kinds of errors students can resolve by just executing the code versus which they can resolve by using a line-by-line debugger. From observing our participants, two factors appear to be necessary for them to independently resolve their misunderstandings. First, they need to be using a tool that provides visibility into the error itself. When using a tool that provided only the output of the code, our participants could only resolve dataflow-oriented errors. In contrast, when given the ability to step through the code, some of our participants could additionally resolve control-flow errors. Second, the error must affect the output. In all of the cases where students arrived at the correct answer in spite of having errors in their understanding of the code, none corrected their error independent of the tool they were using. Presumably, they were not forced to confront their error because of an incorrect answer. Finally, while necessary, these conditions appear not to be sufficient, as students still need to be able to correctly interpret the information that the tool provides.