TY - GEN

T1 - How much randomization is needed to deter collaborative cheating on asynchronous exams?

AU - Chen, Binglin

AU - West, Matthew

AU - Zilles, Craig

N1 - Funding Information:
This work was partially supported by NSF DUE-1347722, NSF CMMI-1150490, and the College of Engineering at the University of Illinois at Urbana-Champaign under the Strategic Instructional Initiatives Program (SIIP).

PY - 2018/6/26

Y1 - 2018/6/26

N2 - This paper investigates randomization on asynchronous exams as a defense against collaborative cheating. Asynchronous exams are those for which students take the exam at different times, potentially across a multi-day exam period. Collaborative cheating occurs when one student (the information producer) takes the exam early and passes information about the exam to other students (the information consumers) that are taking the exam later. Using a dataset of computerized exam and homework problems in a single course with 425 students, we identified 5.5% of students (on average) as information consumers by their disproportionate studying of problems that were on the exam. These information consumers ("cheaters") had a significant advantage (13 percentage points on average) when every student was given the same exam problem (even when the parameters are randomized for each student), but that advantage dropped to almost negligible levels (2-3 percentage points) when students were given a random problem from a pool of two or four problems. We conclude that randomization with pools of four (or even three) problems, which also contain randomized parameters, is an effective mitigation for collaborative cheating. Our analysis suggests that this mitigation is in part explained by cheating students having less complete information about larger pools.

AB - This paper investigates randomization on asynchronous exams as a defense against collaborative cheating. Asynchronous exams are those for which students take the exam at different times, potentially across a multi-day exam period. Collaborative cheating occurs when one student (the information producer) takes the exam early and passes information about the exam to other students (the information consumers) that are taking the exam later. Using a dataset of computerized exam and homework problems in a single course with 425 students, we identified 5.5% of students (on average) as information consumers by their disproportionate studying of problems that were on the exam. These information consumers ("cheaters") had a significant advantage (13 percentage points on average) when every student was given the same exam problem (even when the parameters are randomized for each student), but that advantage dropped to almost negligible levels (2-3 percentage points) when students were given a random problem from a pool of two or four problems. We conclude that randomization with pools of four (or even three) problems, which also contain randomized parameters, is an effective mitigation for collaborative cheating. Our analysis suggests that this mitigation is in part explained by cheating students having less complete information about larger pools.

KW - Asynchronous Exams

KW - Collaborative Cheating

KW - Computerized Testing

KW - Problem Randomization

UR - http://www.scopus.com/inward/record.url?scp=85051569396&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85051569396&partnerID=8YFLogxK

U2 - 10.1145/3231644.3231664

DO - 10.1145/3231644.3231664

M3 - Conference contribution

AN - SCOPUS:85051569396

T3 - Proceedings of the 5th Annual ACM Conference on Learning at Scale, L at S 2018

BT - Proceedings of the 5th Annual ACM Conference on Learning at Scale, L at S 2018

PB - Association for Computing Machinery, Inc

T2 - 5th Annual ACM Conference on Learning at Scale, L at S 2018

Y2 - 26 June 2018 through 28 June 2018

ER -