Data-Driven Investigation into Variants of Code Writing Questions

Liia Butler, Geoffrey Challen, Tao Xie

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

To defend against collaborative cheating in code writing questions, instructors of courses with online, asynchronous exams can use the strategy of question variants. These question variants are manually written questions to be selected at random during exam time to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring that variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose data-driven investigation into these variants. We apply our data-driven investigation into a dataset of three midterm exams from a large introductory programming course. Our results show that (1) observable inequalities of student performance exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our data-driven investigation can be used to provide recommendations for improving design of future variants.

Original languageEnglish (US)
Title of host publication2020 IEEE 32nd Conference on Software Engineering Education and Training, CSEE and T 2020
EditorsMarian Daun, Elke Hochmuller, Stephan Krusche, Bernd Brugge, Bastian Tenbergen
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages75-84
Number of pages10
ISBN (Electronic)9781728168074
DOIs
StatePublished - Nov 2020
Event32nd IEEE Conference on Software Engineering Education and Training, CSEE and T 2020 - Munich, Germany
Duration: Nov 9 2020Nov 12 2020

Publication series

Name2020 IEEE 32nd Conference on Software Engineering Education and Training, CSEE and T 2020

Conference

Conference32nd IEEE Conference on Software Engineering Education and Training, CSEE and T 2020
CountryGermany
CityMunich
Period11/9/2011/12/20

Keywords

  • Question variants
  • assessment
  • code writing
  • exam questions

ASJC Scopus subject areas

  • Education
  • Software

Fingerprint Dive into the research topics of 'Data-Driven Investigation into Variants of Code Writing Questions'. Together they form a unique fingerprint.

Cite this