Analyzing the decline of student scores over time in self-scheduled asynchronous exams

Binglin Chen, Matthew West, Craig Zilles

Research output: Contribution to journalArticle

Abstract

Background: When students are given a choice of when to take an exam in engineering and computing courses, it has been previously observed that average exam scores generally decline over the exam period. This trend may have implications both for the design of interventions to improve student learning and for data analysis to detect collaborative cheating. Purpose/Hypothesis: We hypothesize that average exam scores decline over the exam period primarily due to self-selection effects, where weaker students tend to choose exam times later in the exam period, while stronger students are more likely to choose earlier times. Design/Method: We collected 31,673 exam records over four semesters from six undergraduate engineering and computing courses that had both synchronous exams (all students at the same time) and asynchronous exams (students choose a time). We analyzed student exam time choice and asynchronous exam scores, using synchronous exam scores in the same course as a control variable. Results: We find that students with lower scores on synchronous exams generally elect to take asynchronous exams later and that controlling for student ability (via synchronous exams) removes 70% of the decline observed in average asynchronous exam scores over the exam period but does not eliminate the downward trend with time. Conclusions: We conclude that self-selection effects are primarily responsible for exam score declines over time, that exam time selection is unlikely to be a useful target for interventions to improve performance, and that there is no evidence for widespread collaborative cheating in the dataset used in this research.

Original languageEnglish (US)
Pages (from-to)574-594
Number of pages21
JournalJournal of Engineering Education
Volume108
Issue number4
DOIs
StatePublished - Oct 1 2019

Fingerprint

Students
student
engineering
time
trend
semester
data analysis
ability
learning
performance
evidence

Keywords

  • asynchronous
  • automated grading
  • test format (syn: Exam format)
  • undergraduate

ASJC Scopus subject areas

  • Education
  • Engineering(all)

Cite this

Analyzing the decline of student scores over time in self-scheduled asynchronous exams. / Chen, Binglin; West, Matthew; Zilles, Craig.

In: Journal of Engineering Education, Vol. 108, No. 4, 01.10.2019, p. 574-594.

Research output: Contribution to journalArticle

@article{95f4f1587a1348bfb52f2f36345fc29d,
title = "Analyzing the decline of student scores over time in self-scheduled asynchronous exams",
abstract = "Background: When students are given a choice of when to take an exam in engineering and computing courses, it has been previously observed that average exam scores generally decline over the exam period. This trend may have implications both for the design of interventions to improve student learning and for data analysis to detect collaborative cheating. Purpose/Hypothesis: We hypothesize that average exam scores decline over the exam period primarily due to self-selection effects, where weaker students tend to choose exam times later in the exam period, while stronger students are more likely to choose earlier times. Design/Method: We collected 31,673 exam records over four semesters from six undergraduate engineering and computing courses that had both synchronous exams (all students at the same time) and asynchronous exams (students choose a time). We analyzed student exam time choice and asynchronous exam scores, using synchronous exam scores in the same course as a control variable. Results: We find that students with lower scores on synchronous exams generally elect to take asynchronous exams later and that controlling for student ability (via synchronous exams) removes 70{\%} of the decline observed in average asynchronous exam scores over the exam period but does not eliminate the downward trend with time. Conclusions: We conclude that self-selection effects are primarily responsible for exam score declines over time, that exam time selection is unlikely to be a useful target for interventions to improve performance, and that there is no evidence for widespread collaborative cheating in the dataset used in this research.",
keywords = "asynchronous, automated grading, test format (syn: Exam format), undergraduate",
author = "Binglin Chen and Matthew West and Craig Zilles",
year = "2019",
month = "10",
day = "1",
doi = "10.1002/jee.20292",
language = "English (US)",
volume = "108",
pages = "574--594",
journal = "Journal of Engineering Education",
issn = "1069-4730",
publisher = "American Society for Engineering Education",
number = "4",

}

TY - JOUR

T1 - Analyzing the decline of student scores over time in self-scheduled asynchronous exams

AU - Chen, Binglin

AU - West, Matthew

AU - Zilles, Craig

PY - 2019/10/1

Y1 - 2019/10/1

N2 - Background: When students are given a choice of when to take an exam in engineering and computing courses, it has been previously observed that average exam scores generally decline over the exam period. This trend may have implications both for the design of interventions to improve student learning and for data analysis to detect collaborative cheating. Purpose/Hypothesis: We hypothesize that average exam scores decline over the exam period primarily due to self-selection effects, where weaker students tend to choose exam times later in the exam period, while stronger students are more likely to choose earlier times. Design/Method: We collected 31,673 exam records over four semesters from six undergraduate engineering and computing courses that had both synchronous exams (all students at the same time) and asynchronous exams (students choose a time). We analyzed student exam time choice and asynchronous exam scores, using synchronous exam scores in the same course as a control variable. Results: We find that students with lower scores on synchronous exams generally elect to take asynchronous exams later and that controlling for student ability (via synchronous exams) removes 70% of the decline observed in average asynchronous exam scores over the exam period but does not eliminate the downward trend with time. Conclusions: We conclude that self-selection effects are primarily responsible for exam score declines over time, that exam time selection is unlikely to be a useful target for interventions to improve performance, and that there is no evidence for widespread collaborative cheating in the dataset used in this research.

AB - Background: When students are given a choice of when to take an exam in engineering and computing courses, it has been previously observed that average exam scores generally decline over the exam period. This trend may have implications both for the design of interventions to improve student learning and for data analysis to detect collaborative cheating. Purpose/Hypothesis: We hypothesize that average exam scores decline over the exam period primarily due to self-selection effects, where weaker students tend to choose exam times later in the exam period, while stronger students are more likely to choose earlier times. Design/Method: We collected 31,673 exam records over four semesters from six undergraduate engineering and computing courses that had both synchronous exams (all students at the same time) and asynchronous exams (students choose a time). We analyzed student exam time choice and asynchronous exam scores, using synchronous exam scores in the same course as a control variable. Results: We find that students with lower scores on synchronous exams generally elect to take asynchronous exams later and that controlling for student ability (via synchronous exams) removes 70% of the decline observed in average asynchronous exam scores over the exam period but does not eliminate the downward trend with time. Conclusions: We conclude that self-selection effects are primarily responsible for exam score declines over time, that exam time selection is unlikely to be a useful target for interventions to improve performance, and that there is no evidence for widespread collaborative cheating in the dataset used in this research.

KW - asynchronous

KW - automated grading

KW - test format (syn: Exam format)

KW - undergraduate

UR - http://www.scopus.com/inward/record.url?scp=85075194193&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075194193&partnerID=8YFLogxK

U2 - 10.1002/jee.20292

DO - 10.1002/jee.20292

M3 - Article

AN - SCOPUS:85075194193

VL - 108

SP - 574

EP - 594

JO - Journal of Engineering Education

JF - Journal of Engineering Education

SN - 1069-4730

IS - 4

ER -