TY - JOUR
T1 - Developing new online calibration methods for multidimensional computerized adaptive testing
AU - Chen, Ping
AU - Wang, Chun
AU - Xin, Tao
AU - Chang, Hua Hua
N1 - This study was partially supported by the National Natural Science Foundation of China (Grant No. 31300862, 31660278), the Specialized Research Fund for the Doctoral Program of Higher Education (Grant No. 20130003120002), and KLAS (Grant No. 130028614). Part of the paper was originally presented in 2013 annual meeting of the Psychometric Society, Arnhem, the Netherlands. The authors are indebted to the editor, associate editor and two anonymous reviewers for their suggestions and comments on the earlier manuscript.
PY - 2017/2/1
Y1 - 2017/2/1
N2 - Multidimensional computerized adaptive testing (MCAT) has received increasing attention over the past few years in educational measurement. Like all other formats of CAT, item replenishment is an essential part of MCAT for its item bank maintenance and management, which governs retiring overexposed or obsolete items over time and replacing them with new ones. Moreover, calibration precision of the new items will directly affect the estimation accuracy of examinees’ ability vectors. In unidimensional CAT (UCAT) and cognitive diagnostic CAT, online calibration techniques have been developed to effectively calibrate new items. However, there has been very little discussion of online calibration in MCAT in the literature. Thus, this paper proposes new online calibration methods for MCAT based upon some popular methods used in UCAT. Three representative methods, Method A, the ‘one EM cycle’ method and the ‘multiple EM cycles’ method, are generalized to MCAT. Three simulation studies were conducted to compare the three new methods by manipulating three factors (test length, item bank design, and level of correlation between coordinate dimensions). The results showed that all the new methods were able to recover the item parameters accurately, and the adaptive online calibration designs showed some improvements compared to the random design under most conditions.
AB - Multidimensional computerized adaptive testing (MCAT) has received increasing attention over the past few years in educational measurement. Like all other formats of CAT, item replenishment is an essential part of MCAT for its item bank maintenance and management, which governs retiring overexposed or obsolete items over time and replacing them with new ones. Moreover, calibration precision of the new items will directly affect the estimation accuracy of examinees’ ability vectors. In unidimensional CAT (UCAT) and cognitive diagnostic CAT, online calibration techniques have been developed to effectively calibrate new items. However, there has been very little discussion of online calibration in MCAT in the literature. Thus, this paper proposes new online calibration methods for MCAT based upon some popular methods used in UCAT. Three representative methods, Method A, the ‘one EM cycle’ method and the ‘multiple EM cycles’ method, are generalized to MCAT. Three simulation studies were conducted to compare the three new methods by manipulating three factors (test length, item bank design, and level of correlation between coordinate dimensions). The results showed that all the new methods were able to recover the item parameters accurately, and the adaptive online calibration designs showed some improvements compared to the random design under most conditions.
KW - item bank construction
KW - item replenishment
KW - multidimensional computerized adaptive testing
KW - multidimensional two-parameter logistic model
KW - online calibration
UR - http://www.scopus.com/inward/record.url?scp=85010903364&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85010903364&partnerID=8YFLogxK
U2 - 10.1111/bmsp.12083
DO - 10.1111/bmsp.12083
M3 - Article
C2 - 28130937
AN - SCOPUS:85010903364
SN - 0007-1102
VL - 70
SP - 81
EP - 117
JO - British Journal of Mathematical and Statistical Psychology
JF - British Journal of Mathematical and Statistical Psychology
IS - 1
ER -