Superficial Code-guise: Investigating the Impact of Surface Feature Changes on Students' Programming Question Scores

Max Fowler, Craig Zilles

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Assessing student performance on programming questions is important for introductory computer science courses, both for student learning and for ensuring students demonstrate competence. Part of being a competent programmer includes the ability to transfer learning from solved to analogous problems. Additionally, particularly in computer-based and online assessment, mitigating cheating efforts is another important consideration. One way to mitigate cheating is by randomly selecting from large pools of equivalent questions. In order to produce large pools of questions quickly, we used a permutation strategy to rapidly make new question variants by altering existing questions' surface features. In this work, we present the results of our first set of surface feature permuted questions in an introductory Python course. We find surface feature permutations to be an effective way to produce questions of a similar difficulty to other new questions for students while mitigating potential cheating. However, we also see permutations expose potential student knowledge fragility and transfer concerns, as performance on permutations of homework questions is not strictly better than performance on questions that are entirely new on assessments

Original languageEnglish (US)
Title of host publicationSIGCSE 2021 - Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
PublisherAssociation for Computing Machinery, Inc
Pages3-9
Number of pages7
ISBN (Electronic)9781450380621
DOIs
StatePublished - Mar 3 2021
Event52nd ACM Technical Symposium on Computer Science Education, SIGCSE 2021 - Virtual, Online, United States
Duration: Mar 13 2021Mar 20 2021

Publication series

NameSIGCSE 2021 - Proceedings of the 52nd ACM Technical Symposium on Computer Science Education

Conference

Conference52nd ACM Technical Symposium on Computer Science Education, SIGCSE 2021
CountryUnited States
CityVirtual, Online
Period3/13/213/20/21

Keywords

  • CS1
  • introductory computer science
  • online assessment
  • programming surface features

ASJC Scopus subject areas

  • Computer Science(all)
  • Education

Fingerprint Dive into the research topics of 'Superficial Code-guise: Investigating the Impact of Surface Feature Changes on Students' Programming Question Scores'. Together they form a unique fingerprint.

Cite this