Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing

Edison M. Choe, Jinming Zhang, Hua Hua Chang

Research output: Contribution to journalArticle

Abstract

Item compromise persists in undermining the integrity of testing, even secure administrations of computerized adaptive testing (CAT) with sophisticated item exposure controls. In ongoing efforts to tackle this perennial security issue in CAT, a couple of recent studies investigated sequential procedures for detecting compromised items, in which a significant increase in the proportion of correct responses for each item in the pool is monitored in real time using moving averages. In addition to actual responses, response times are valuable information with tremendous potential to reveal items that may have been leaked. Specifically, examinees that have preknowledge of an item would likely respond more quickly to it than those who do not. Therefore, the current study proposes several augmented methods for the detection of compromised items, all involving simultaneous monitoring of changes in both the proportion correct and average response time for every item using various moving average strategies. Simulation results with an operational item pool indicate that, compared to the analysis of responses alone, utilizing response times can afford marked improvements in detection power with fewer false positives.

Original languageEnglish (US)
Pages (from-to)650-673
Number of pages24
JournalPsychometrika
Volume83
Issue number3
DOIs
StatePublished - Sep 1 2018

Keywords

  • change-point detection
  • computerized adaptive testing
  • repeated significance tests
  • response time
  • sequential analysis
  • test security

ASJC Scopus subject areas

  • Psychology(all)
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing'. Together they form a unique fingerprint.

  • Cite this