TY - JOUR
T1 - Stratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing
AU - Yang, Jing
AU - Chang, Hua Hua
AU - Tao, Jian
AU - Shi, Ningzhong
N1 - Funding Information:
The authors would like to thank the Editor in Chief, the Associate Editor, Dr. Xue Zhang, and two anonymous reviewers for their helpful comments on earlier drafts of this article. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Natural Science and Social Science Foundations of China (Grant No. 11571069).
Funding Information:
https://orcid.org/0000-0001-7453-4353 Yang Jing 1 Chang Hua-Hua 2 https://orcid.org/0000-0002-0343-1426 Tao Jian 1 Shi Ningzhong 1 1 Northeast Normal University, Changchun, China 2 Purdue University, West Lafayette, IN, USA Jing Yang, Department of Statistics, School of Mathematics and Statistics, Northeast Normal University, 5268 Renmin Street, Changchun, Jilin Province 130024, China. Email: yangj014@nenu.edu.cn Jian Tao, Department of Statistics, School of Mathematics and Statistics, Northeast Normal University, 5268 Renmin Street, Changchun, Jilin Province 130024, China. Email: taoj@nenu.edu.cn 12 2019 0146621619893783 © The Author(s) 2019 2019 SAGE Publications Cognitive diagnostic computerized adaptive testing (CD-CAT) aims to obtain more useful diagnostic information by taking advantages of computerized adaptive testing (CAT). Cognitive diagnosis models (CDMs) have been developed to classify examinees into the correct proficiency classes so as to get more efficient remediation, whereas CAT tailors optimal items to the examinee’s mastery profile. The item selection method is the key factor of the CD-CAT procedure. In recent years, a large number of parametric/nonparametric item selection methods have been proposed. In this article, the authors proposed a series of stratified item selection methods in CD-CAT, which are combined with posterior-weighted Kullback–Leibler (PWKL), nonparametric item selection (NPS), and weighted nonparametric item selection (WNPS) methods, and named S-PWKL, S-NPS, and S-WNPS, respectively. Two different types of stratification indices were used: original versus novel. The performances of the proposed item selection methods were evaluated via simulation studies and compared with the PWKL, NPS, and WNPS methods without stratification. Manipulated conditions included calibration sample size, item quality, number of attributes, number of strata, and data generation models. Results indicated that the S-WNPS and S-NPS methods performed similarly, and both outperformed the S-PWKL method. And item selection methods with novel stratification indices performed slightly better than the ones with original stratification indices, and those without stratification performed the worst. cognitive diagnostic assessment computerized adaptive testing nonparametric item selection method stratification indices National Natural Science and Social Science Foundations of China 11571069 edited-state corrected-proof typesetter ts1 The authors would like to thank the Editor in Chief, the Associate Editor, Dr. Xue Zhang, and two anonymous reviewers for their helpful comments on earlier drafts of this article. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Natural Science and Social Science Foundations of China (Grant No. 11571069). ORCID iDs Jing Yang https://orcid.org/0000-0001-7453-4353 Jian Tao https://orcid.org/0000-0002-0343-1426 Supplemental Material Supplemental material for this article is available online.
Publisher Copyright:
© The Author(s) 2019.
PY - 2020/7/1
Y1 - 2020/7/1
N2 - Cognitive diagnostic computerized adaptive testing (CD-CAT) aims to obtain more useful diagnostic information by taking advantages of computerized adaptive testing (CAT). Cognitive diagnosis models (CDMs) have been developed to classify examinees into the correct proficiency classes so as to get more efficient remediation, whereas CAT tailors optimal items to the examinee’s mastery profile. The item selection method is the key factor of the CD-CAT procedure. In recent years, a large number of parametric/nonparametric item selection methods have been proposed. In this article, the authors proposed a series of stratified item selection methods in CD-CAT, which are combined with posterior-weighted Kullback–Leibler (PWKL), nonparametric item selection (NPS), and weighted nonparametric item selection (WNPS) methods, and named S-PWKL, S-NPS, and S-WNPS, respectively. Two different types of stratification indices were used: original versus novel. The performances of the proposed item selection methods were evaluated via simulation studies and compared with the PWKL, NPS, and WNPS methods without stratification. Manipulated conditions included calibration sample size, item quality, number of attributes, number of strata, and data generation models. Results indicated that the S-WNPS and S-NPS methods performed similarly, and both outperformed the S-PWKL method. And item selection methods with novel stratification indices performed slightly better than the ones with original stratification indices, and those without stratification performed the worst.
AB - Cognitive diagnostic computerized adaptive testing (CD-CAT) aims to obtain more useful diagnostic information by taking advantages of computerized adaptive testing (CAT). Cognitive diagnosis models (CDMs) have been developed to classify examinees into the correct proficiency classes so as to get more efficient remediation, whereas CAT tailors optimal items to the examinee’s mastery profile. The item selection method is the key factor of the CD-CAT procedure. In recent years, a large number of parametric/nonparametric item selection methods have been proposed. In this article, the authors proposed a series of stratified item selection methods in CD-CAT, which are combined with posterior-weighted Kullback–Leibler (PWKL), nonparametric item selection (NPS), and weighted nonparametric item selection (WNPS) methods, and named S-PWKL, S-NPS, and S-WNPS, respectively. Two different types of stratification indices were used: original versus novel. The performances of the proposed item selection methods were evaluated via simulation studies and compared with the PWKL, NPS, and WNPS methods without stratification. Manipulated conditions included calibration sample size, item quality, number of attributes, number of strata, and data generation models. Results indicated that the S-WNPS and S-NPS methods performed similarly, and both outperformed the S-PWKL method. And item selection methods with novel stratification indices performed slightly better than the ones with original stratification indices, and those without stratification performed the worst.
KW - cognitive diagnostic assessment
KW - computerized adaptive testing
KW - nonparametric item selection method
KW - stratification indices
UR - http://www.scopus.com/inward/record.url?scp=85077164322&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85077164322&partnerID=8YFLogxK
U2 - 10.1177/0146621619893783
DO - 10.1177/0146621619893783
M3 - Article
C2 - 32879535
AN - SCOPUS:85077164322
SN - 0146-6216
VL - 44
SP - 346
EP - 361
JO - Applied Psychological Measurement
JF - Applied Psychological Measurement
IS - 5
ER -