Developing new online calibration methods for multidimensional computerized adaptive testing

Ping Chen, Chun Wang, Tao Xin, Hua Hua Chang

Research output: Contribution to journalArticlepeer-review

Abstract

Multidimensional computerized adaptive testing (MCAT) has received increasing attention over the past few years in educational measurement. Like all other formats of CAT, item replenishment is an essential part of MCAT for its item bank maintenance and management, which governs retiring overexposed or obsolete items over time and replacing them with new ones. Moreover, calibration precision of the new items will directly affect the estimation accuracy of examinees’ ability vectors. In unidimensional CAT (UCAT) and cognitive diagnostic CAT, online calibration techniques have been developed to effectively calibrate new items. However, there has been very little discussion of online calibration in MCAT in the literature. Thus, this paper proposes new online calibration methods for MCAT based upon some popular methods used in UCAT. Three representative methods, Method A, the ‘one EM cycle’ method and the ‘multiple EM cycles’ method, are generalized to MCAT. Three simulation studies were conducted to compare the three new methods by manipulating three factors (test length, item bank design, and level of correlation between coordinate dimensions). The results showed that all the new methods were able to recover the item parameters accurately, and the adaptive online calibration designs showed some improvements compared to the random design under most conditions.

Original languageEnglish (US)
Pages (from-to)81-117
Number of pages37
JournalBritish Journal of Mathematical and Statistical Psychology
Volume70
Issue number1
DOIs
StatePublished - Feb 1 2017

Keywords

  • item bank construction
  • item replenishment
  • multidimensional computerized adaptive testing
  • multidimensional two-parameter logistic model
  • online calibration

ASJC Scopus subject areas

  • Statistics and Probability
  • Arts and Humanities (miscellaneous)
  • General Psychology

Fingerprint

Dive into the research topics of 'Developing new online calibration methods for multidimensional computerized adaptive testing'. Together they form a unique fingerprint.

Cite this