Classification Accuracy of Mixed Format Tests: A Bi-Factor Item Response Theory Approach

Wei Wang, Fritz Drasgow, Liwen Liu

Research output: Contribution to journalArticlepeer-review

Abstract

Mixed format tests (e.g., a test consisting of multiple-choice [MC] items and constructed response [CR] items) have become increasingly popular. However, the latent structure of item pools consisting of the two formats is still equivocal. Moreover, the implications of this latent structure are unclear: For example, do constructed response items tap reasoning skills that cannot be assessed with multiple choice items? This study explored the dimensionality of mixed format tests by applying bi-factor models to 10 tests of various subjects from the College Board's Advanced Placement (AP) Program and compared the accuracy of scores based on the bi-factor analysis with scores derived from a unidimensional analysis. More importantly, this study focused on a practical and important question—classification accuracy of the overall grade on a mixed format test. Our findings revealed that the degree of multidimensionality resulting from the mixed item format varied from subject to subject, depending on the disattenuated correlation between scores from MC and CR subtests. Moreover, remarkably small decrements in classification accuracy were found for the unidimensional analysis when the disattenuated correlations exceeded 0.90.

Original languageEnglish (US)
Article number270
JournalFrontiers in Psychology
Volume7
DOIs
StatePublished - Feb 29 2016

Keywords

  • bi-factor model
  • classification accuracy
  • constructed response items
  • item response theory
  • mixed format test

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint Dive into the research topics of 'Classification Accuracy of Mixed Format Tests: A Bi-Factor Item Response Theory Approach'. Together they form a unique fingerprint.

Cite this