Applying Sequence Analysis to Understand the Debugging Process of Novice Programmers

Debugging is a challenging task for novice programmers that requires diverse skills as well as iterative practice to find and fix the cause of errors. In this study, we analyzed submission log data to investigate the temporal aspects of the debugging process. We extracted debugging episodes and interpretable debugging constituents - components of experts' interpretations of debugging behaviors - from the data collected from an undergraduate CS1 course. We first applied sequential pattern mining and state transition metrics to examine how debugging constituents occur one after another. We further applied temporal interestingness techniques to reveal the occurrence of debugging constituents within each episode and compared the difference in constituent patterns across the semester. We further investigated how the sequential ordering of debugging constituents changed over the course of the semester. Our findings suggest that novice programmers exhibit frequent printing and submission undo no matter in which phase of the semester. As they proceeded toward the end of the semester, they employed more repetitive printing to understand the problem and long planning before implementation.