Algorithmic grading strategies for computerized drawing assessments

Introductory mechanics courses have important learning objectives focusing on students' ability to accurately draw or sketch particular types of diagrams, such as free body diagrams and graphs of shear forces and bending moments in beams. To achieve mastery of these drawing skills it is essential that students have many opportunities to practice and that they receive rapid and accurate feedback on whether they are drawing the correct diagram for a given mechanical problem. With the growing student enrollment in many engineering programs, however, it becomes increasingly difficult to provide prompt and accurate grading using the traditional approach of having students submit hand-drawn diagrams which are graded by a teaching assistant or grader. Instead, an appealing alternative is to use automated computer-based systems for presenting problems to students, in which they draw a diagram or graph on the computer and this can then be immediately graded algorithmically and feedback returned.

The central question addressed by this paper is as follows. When receiving a computer-drawn mechanical diagram or sketch from a student, what algorithmic procedure should be used to grade the submission? We present a general algorithmic framework for grading diagrams that addresses five key functionality requirements: (1) the algorithm should be able to provide students with meaningful feedback about errors in their diagram, (2) the algorithm should be easy to understand for problem authors, and require only data which is readily available to authors, (3) the algorithm should be adaptable to different types of drawings or sketches, (4) the algorithm should be fast to execute, and (5) the algorithm should be robust to unexpected or unusual inputs.

The algorithm we describe satisfies all of these requirements and it was implemented in a computerized drawing system used for a large introductory mechanics course at Midwestern University with approximately 200 students. We present results from three evaluation sources: (1) student interaction data with the system, (2) student affect data reported via survey and anonymous written feedback, and (3) instructor feedback. The results indicate that the system was able to efficiently and robustly grade student diagrams and provide formative feedback, and that students significantly increased the number of practice problems that they solved using this system when compared to the traditional pen-and-paper method previously used.