Grading-based test suite augmentation

Jonathan Osei-Owusu, Angello Astorga, Liia Butler, Tao Xie, Geoffrey Challen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Enrollment in introductory programming (CS1) courses continues to surge and hundreds of CS1 students can produce thousands of submissions for a single problem, all requiring timely and accurate grading. One way that instructors can efficiently grade is to construct a custom instructor test suite that compares a student submission to a reference solution over randomly generated or hand-crafted inputs. However, such test suite is often insufficient, causing incorrect submissions to be marked as correct. To address this issue, we propose the Grasa (GRAding-based test Suite Augmentation) approach consisting of two techniques. Grasa first detects and clusters incorrect submissions by approximating their behavioral equivalence to each other. To augment the existing instructor test suite, Grasa generates a minimal set of additional tests that help detect the incorrect submissions. We evaluate our Grasa approach on a dataset of CS1 student submissions for three programming problems. Our preliminary results show that Grasa can effectively identify incorrect student submissions and minimally augment the instructor test suite.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages226-229
Number of pages4
ISBN (Electronic)9781728125084
DOIs
StatePublished - Nov 2019
Event34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019 - San Diego, United States
Duration: Nov 10 2019Nov 15 2019

Publication series

NameProceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019

Conference

Conference34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019
CountryUnited States
CitySan Diego
Period11/10/1911/15/19

Keywords

  • Clustering
  • Programming education
  • Testing

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software
  • Control and Optimization

Fingerprint Dive into the research topics of 'Grading-based test suite augmentation'. Together they form a unique fingerprint.

  • Cite this

    Osei-Owusu, J., Astorga, A., Butler, L., Xie, T., & Challen, G. (2019). Grading-based test suite augmentation. In Proceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019 (pp. 226-229). [8952332] (Proceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASE.2019.00030