Comparing the Impacts of Visually Grouped and Jumbled Distractors on Parsons Problems in CS1 Assessments

Parsons problems are a commonly used problem type typically used in introductory computer science courses. They involve organizing blocks containing segments of code to form a program. These questions often use ''distractors'' which are plausible, but incorrect, blocks of code. In Parsons problems distractors are often included either by jumbling them in alongside the correct response options or visually grouping them with their correct alternatives. In this study, we investigate the impact of both jumbled and visually grouped distractors on: 1) student performance, 2) the amount of time students spent on the question, and 3) the item's quality in exams and quizzes in a CS1 Python course. Our findings indicate that the inclusion of distractors, both visually grouped and jumbled, have a marginal impact on item quality while increasing the amount of time needed to solve the problem and reducing students' performance. Though visually grouped distractors appear to mediate the amount of time students spend relative to their jumbled counterparts, their effect is not so large as to fully alleviate concerns related to the increase in duration.