Measuring the score advantage on asynchronous exams in an undergraduate cs course

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents the results of a controlled crossover experiment designed to measure the score advantage that students have when taking exams asynchronously (i.e., the students can select a time to take the exam in a multi-day window) compared to synchronous exams (i.e., all students take the exam at the same time). The study was performed in an upper-division undergraduate computer science course with 321 students. Stratified sampling was used to randomly assign the students to two groups that alternated between the two treatments (synchronous versus asynchronous exams) across a series of four exams during the semester. These non-programming exams consisted of a mix of multiple choice, checkbox, and numeric input questions. For some questions, the parameters were randomized so that students received different versions of the question and some questions were identical for all students. In our results, students taking the exams asynchronously had scores that were on average only 3% higher (0.2 of a standard deviation). Furthermore, we found that the score advantage was decreased by the use of randomized questions, and it did not significantly differ based on the type of question. Thus, our results suggest that asynchronous exams can be a compelling alternative to synchronous exams.

Original languageEnglish (US)
Title of host publicationSIGCSE 2020 - Proceedings of the 51st ACM Technical Symposium on Computer Science Education
PublisherAssociation for Computing Machinery
Pages873-879
Number of pages7
ISBN (Electronic)9781450367936
DOIs
StatePublished - Feb 26 2020
Event51st ACM SIGCSE Technical Symposium on Computer Science Education, SIGCSE 2020 - Portland, United States
Duration: Mar 11 2020Mar 14 2020

Publication series

NameAnnual Conference on Innovation and Technology in Computer Science Education, ITiCSE
ISSN (Print)1942-647X

Conference

Conference51st ACM SIGCSE Technical Symposium on Computer Science Education, SIGCSE 2020
CountryUnited States
CityPortland
Period3/11/203/14/20

Keywords

  • Asynchronous exams
  • Cheating
  • Computer-based testing
  • Randomized questions

ASJC Scopus subject areas

  • Management of Technology and Innovation
  • Education

Fingerprint Dive into the research topics of 'Measuring the score advantage on asynchronous exams in an undergraduate cs course'. Together they form a unique fingerprint.

  • Cite this

    Silva, M., West, M., & Zilles, C. (2020). Measuring the score advantage on asynchronous exams in an undergraduate cs course. In SIGCSE 2020 - Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 873-879). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE). Association for Computing Machinery. https://doi.org/10.1145/3328778.3366859