Autograding

Max Fowler, Binglin Chen, Sushmita Azad, Matthew West, Craig Zilles

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Previous research suggests that "Explain in Plain English"(EiPE) code reading activities could play an important role in the development of novice programmers, but EiPE questions aren't heavily used in introductory programming courses because they (traditionally) required manual grading. We present what we believe to be the first automatic grader for EiPE questions and its deployment in a large-enrollment introductory programming course. Based on a set of questions deployed on a computer-based exam, we find that our implementation has an accuracy of 87-89%, which is similar in performance to course teaching assistants trained to perform this task and compares favorably to automatic short answer grading algorithms developed for other domains. In addition, we briefly characterize the kinds of answers that the current autograder fails to score correctly and the kinds of errors made by students.

Original languageEnglish (US)
Title of host publicationSIGCSE 2021 - Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
PublisherAssociation for Computing Machinery, Inc
Pages1163-1169
Number of pages7
ISBN (Electronic)9781450380621
DOIs
StatePublished - Mar 3 2021
Event52nd ACM Technical Symposium on Computer Science Education, SIGCSE 2021 - Virtual, Online, United States
Duration: Mar 13 2021Mar 20 2021

Publication series

NameSIGCSE 2021 - Proceedings of the 52nd ACM Technical Symposium on Computer Science Education

Conference

Conference52nd ACM Technical Symposium on Computer Science Education, SIGCSE 2021
CountryUnited States
CityVirtual, Online
Period3/13/213/20/21

Keywords

  • asag
  • code reading
  • cs1
  • explain in plain english
  • nlp

ASJC Scopus subject areas

  • Computer Science(all)
  • Education

Fingerprint Dive into the research topics of 'Autograding'. Together they form a unique fingerprint.

Cite this