TY - GEN
T1 - How much randomization is needed to deter collaborative cheating on asynchronous exams?
AU - Chen, Binglin
AU - West, Matthew
AU - Zilles, Craig
N1 - Publisher Copyright:
© 2017 Association for Computing Machinery. All rights reserved.
PY - 2018/6/26
Y1 - 2018/6/26
N2 - This paper investigates randomization on asynchronous exams as a defense against collaborative cheating. Asynchronous exams are those for which students take the exam at different times, potentially across a multi-day exam period. Collaborative cheating occurs when one student (the information producer) takes the exam early and passes information about the exam to other students (the information consumers) that are taking the exam later. Using a dataset of computerized exam and homework problems in a single course with 425 students, we identified 5.5% of students (on average) as information consumers by their disproportionate studying of problems that were on the exam. These information consumers ("cheaters") had a significant advantage (13 percentage points on average) when every student was given the same exam problem (even when the parameters are randomized for each student), but that advantage dropped to almost negligible levels (2-3 percentage points) when students were given a random problem from a pool of two or four problems. We conclude that randomization with pools of four (or even three) problems, which also contain randomized parameters, is an effective mitigation for collaborative cheating. Our analysis suggests that this mitigation is in part explained by cheating students having less complete information about larger pools.
AB - This paper investigates randomization on asynchronous exams as a defense against collaborative cheating. Asynchronous exams are those for which students take the exam at different times, potentially across a multi-day exam period. Collaborative cheating occurs when one student (the information producer) takes the exam early and passes information about the exam to other students (the information consumers) that are taking the exam later. Using a dataset of computerized exam and homework problems in a single course with 425 students, we identified 5.5% of students (on average) as information consumers by their disproportionate studying of problems that were on the exam. These information consumers ("cheaters") had a significant advantage (13 percentage points on average) when every student was given the same exam problem (even when the parameters are randomized for each student), but that advantage dropped to almost negligible levels (2-3 percentage points) when students were given a random problem from a pool of two or four problems. We conclude that randomization with pools of four (or even three) problems, which also contain randomized parameters, is an effective mitigation for collaborative cheating. Our analysis suggests that this mitigation is in part explained by cheating students having less complete information about larger pools.
KW - Asynchronous Exams
KW - Collaborative Cheating
KW - Computerized Testing
KW - Problem Randomization
UR - http://www.scopus.com/inward/record.url?scp=85051569396&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85051569396&partnerID=8YFLogxK
U2 - 10.1145/3231644.3231664
DO - 10.1145/3231644.3231664
M3 - Conference contribution
AN - SCOPUS:85051569396
T3 - Proceedings of the 5th Annual ACM Conference on Learning at Scale, L at S 2018
BT - Proceedings of the 5th Annual ACM Conference on Learning at Scale, L at S 2018
PB - Association for Computing Machinery
T2 - 5th Annual ACM Conference on Learning at Scale, L at S 2018
Y2 - 26 June 2018 through 28 June 2018
ER -