Parameter Drift Detection in Multidimensional Computerized Adaptive Testing Based on Informational Distance/Divergence Measures

Hyeon Ah Kang, Hua Hua Chang

Research output: Contribution to journalArticlepeer-review

Abstract

An informational distance/divergence-based approach is proposed to detect the presence of parameter drift in multidimensional computerized adaptive testing (MCAT). The study presents significance testing procedures for identifying changes in multidimensional item response functions (MIRFs) over time based on informational distance/divergence measures that capture the discrepancy between two probability functions. To approximate the MIRFs from the observed response data, the k-nearest neighbors algorithm is used with the random search method. A simulation study suggests that the distance/divergence-based drift measures perform effectively in identifying the instances of parameter drift in MCAT. They showed moderate power with small samples of 500 examinees and excellent power when the sample size was as large as 1,000. The proposed drift measures also adequately controlled for Type I error at the nominal level under the null hypothesis.

Original languageEnglish (US)
Pages (from-to)534-550
Number of pages17
JournalApplied Psychological Measurement
Volume40
Issue number7
DOIs
StatePublished - Oct 1 2016

Keywords

  • item parameter drift
  • multidimensional computerized adaptive testing
  • multidimensional item response function

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Psychology (miscellaneous)

Fingerprint Dive into the research topics of 'Parameter Drift Detection in Multidimensional Computerized Adaptive Testing Based on Informational Distance/Divergence Measures'. Together they form a unique fingerprint.

Cite this