Stratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing

Jing Yang, Hua Hua Chang, Jian Tao, Ningzhong Shi

Research output: Contribution to journalArticlepeer-review

Abstract

Cognitive diagnostic computerized adaptive testing (CD-CAT) aims to obtain more useful diagnostic information by taking advantages of computerized adaptive testing (CAT). Cognitive diagnosis models (CDMs) have been developed to classify examinees into the correct proficiency classes so as to get more efficient remediation, whereas CAT tailors optimal items to the examinee’s mastery profile. The item selection method is the key factor of the CD-CAT procedure. In recent years, a large number of parametric/nonparametric item selection methods have been proposed. In this article, the authors proposed a series of stratified item selection methods in CD-CAT, which are combined with posterior-weighted Kullback–Leibler (PWKL), nonparametric item selection (NPS), and weighted nonparametric item selection (WNPS) methods, and named S-PWKL, S-NPS, and S-WNPS, respectively. Two different types of stratification indices were used: original versus novel. The performances of the proposed item selection methods were evaluated via simulation studies and compared with the PWKL, NPS, and WNPS methods without stratification. Manipulated conditions included calibration sample size, item quality, number of attributes, number of strata, and data generation models. Results indicated that the S-WNPS and S-NPS methods performed similarly, and both outperformed the S-PWKL method. And item selection methods with novel stratification indices performed slightly better than the ones with original stratification indices, and those without stratification performed the worst.

Original languageEnglish (US)
Pages (from-to)346-361
Number of pages16
JournalApplied Psychological Measurement
Volume44
Issue number5
DOIs
StatePublished - Jul 1 2020
Externally publishedYes

Keywords

  • cognitive diagnostic assessment
  • computerized adaptive testing
  • nonparametric item selection method
  • stratification indices

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Psychology (miscellaneous)

Fingerprint Dive into the research topics of 'Stratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing'. Together they form a unique fingerprint.

Cite this