Peer-grading "explain in Plain English" Questions: A Bayesian Calibration Method for Categorical Answers

Binglin Chen, Matthew West, Craig Zilles

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

"Explain in plain English'' (EipE) questions have been proposed as an important activity and assessment for studying novice programmers' grasp of programming knowledge and their ability to communicate their understanding. However, EipE questions aren't widely used in introductory programming courses in part because of the large grading effort required. In this paper, we present our experience of using peer grading for EipE questions in a large-enrollment introductory programming course, where students were asked to categorize other students' responses. We developed a novel Bayesian algorithm for performing calibrated peer grading on categorical data, and we used a heuristic grade assignment method based on the Bayesian estimates. The peer-grading exercises served both as a way to coach students on what is expected from EipE questions and as a way to alleviate the grading load for the course staff. Based on four rounds of peer-grading activities, we found that students are generally capable of categorizing responses to EiPE questions and that our proposed Bayesian method is more robust than unweighted voting.

Original languageEnglish (US)
Title of host publicationSIGCSE 2022 - Proceedings of the 53rd ACM Technical Symposium on Computer Science Education
PublisherAssociation for Computing Machinery, Inc
Pages133-139
Number of pages7
ISBN (Electronic)9781450390705
DOIs
StatePublished - Feb 22 2022
Event53rd Annual ACM Technical Symposium on Computer Science Education, SIGCSE 2022 - Virtual, Online, United States
Duration: Mar 3 2022Mar 5 2022

Publication series

NameSIGCSE 2022 - Proceedings of the 53rd ACM Technical Symposium on Computer Science Education
Volume1

Conference

Conference53rd Annual ACM Technical Symposium on Computer Science Education, SIGCSE 2022
Country/TerritoryUnited States
CityVirtual, Online
Period3/3/223/5/22

Keywords

  • cs1
  • eipe
  • experience report
  • explain in plain english
  • peer grading

ASJC Scopus subject areas

  • Computer Science(all)
  • Education

Fingerprint

Dive into the research topics of 'Peer-grading "explain in Plain English" Questions: A Bayesian Calibration Method for Categorical Answers'. Together they form a unique fingerprint.

Cite this