Item selection in computerized adaptive testing: Improving the a-stratified design with the sympson-hetter algorithm

Chi Keung Leung, Hua Hua Chang, Kit Tal Hau

Research output: Contribution to journalArticlepeer-review

Abstract

Item exposure control, test-overlap minimization, and the efficient use of item pool are some of the important issues in computerized adaptive testing (CAT) designs. The overexposure of some items and high test-overlap rate may cause both item and test security problems. Previously these problems associated with the maximum information (Max-I) item selection method have been partially solved by incorporating the Sympson-Hetter (SH) probabilistic procedure that attempts to control the actual usage of the popular high-discrimination items. On a different line of thought, Chang and Ying proposed the a-stratified design (STR) and advocated the use of less discriminating items in the earlier stages of testing. The method has been demonstrated to be effective in improving the utilization of the entire pool without sacrificing the efficiency in ability estimation. Nevertheless, the problem of item overexposure still persists when the ratio of pool size to test length is small. In four simulation studies, the possible benefits of incorporating the SH procedure in the STR method were investigated. The performance of such an enhanced stratified method (STR-SH) was compared with that of STR as well as Max-I-SH. The results indicated the potential advantages of the STR-SH design over the original STR in yielding a more balanced item exposure distribution, lowering the test-overlap rate, and remarkably reducing the number of overexposed items.

Original languageEnglish (US)
Pages (from-to)376-392
Number of pages17
JournalApplied Psychological Measurement
Volume26
Issue number4
DOIs
StatePublished - Dec 2002
Externally publishedYes

Keywords

  • A-stratified design
  • Computerized adaptive testing
  • Item exposure control
  • Item pool utilization
  • Sympson-Hetter algorithm
  • Test-overlap control

ASJC Scopus subject areas

  • Psychology(all)
  • Psychology (miscellaneous)
  • Social Sciences (miscellaneous)

Fingerprint Dive into the research topics of 'Item selection in computerized adaptive testing: Improving the a-stratified design with the sympson-hetter algorithm'. Together they form a unique fingerprint.

Cite this