Computerized adaptive testing that allows for response revision: Design and asymptotic theory

Research output: Contribution to journalArticlepeer-review

Abstract

In Computerized Adaptive Testing (CAT), items are selected in real time and are adjusted to the test-taker's ability. While CAT has become popular for many measurement tasks, such as educational testing and patient reported outcomes, it has been criticized for not allowing examinees to review and revise their answers. In this work, we propose a novel CAT design that preserves the efficiency of a conventional CAT, but allows test-takers to revise their previous answers at any time during the test. The proposed method relies on a polytomous Item Response model that describes the first response to each item, as well as any subsequent responses to it. Each item is selected in order to maximize the Fisher information of the model at the current ability estimate, which is given by the maximizer of a partial likelihood function. We establish the strong consistency and asymptotic normality of the final ability estimator under minimal conditions on the test-taker's revision behavior. We present the findings of two simulation studies that illustrate our theoretical results, as well as the behavior of the proposed design in a realistic item pool.

Original languageEnglish (US)
Pages (from-to)1987-2010
Number of pages24
JournalStatistica Sinica
Volume27
Issue number4
DOIs
StatePublished - Oct 2017

Keywords

  • Asymptotic normality
  • Computerized adaptive testing
  • Consistency
  • Item response theory
  • Martingale limit theory
  • Nominal response model
  • Sequential design

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Computerized adaptive testing that allows for response revision: Design and asymptotic theory'. Together they form a unique fingerprint.

Cite this