Heuristic cognitive diagnosis when the Q-matrix is unknown

Hans Friedrich Koehn, Chia Yi Chiu, Michael J. Brusco

Research output: Contribution to journalArticle

Abstract

Cognitive diagnosis models of educational test performance rely on a binary Q-matrix that specifies the associations between individual test items and the cognitive attributes (skills) required to answer those items correctly. Current methods for fitting cognitive diagnosis models to educational test data and assigning examinees to proficiency classes are based on parametric estimation methods such as expectation maximization (EM) and Markov chain Monte Carlo (MCMC) that frequently encounter difficulties in practical applications. In response to these difficulties, non-parametric classification techniques (cluster analysis) have been proposed as heuristic alternatives to parametric procedures. These non-parametric classification techniques first aggregate each examinee's test item scores into a profile of attribute sum scores, which then serve as the basis for clustering examinees into proficiency classes. Like the parametric procedures, the non-parametric classification techniques require that the Q-matrix underlying a given test be known. Unfortunately, in practice, the Q-matrix for most tests is not known and must be estimated to specify the associations between items and attributes, risking a misspecified Q-matrix that may then result in the incorrect classification of examinees. This paper demonstrates that clustering examinees into proficiency classes based on their item scores rather than on their attribute sum-score profiles does not require knowledge of the Q-matrix, and results in a more accurate classification of examinees.

Original languageEnglish (US)
Pages (from-to)268-291
Number of pages24
JournalBritish Journal of Mathematical and Statistical Psychology
Volume68
Issue number2
DOIs
StatePublished - May 1 2015

Keywords

  • Asymptotic theory of cognitive diagnosis
  • Classification
  • Clustering
  • Cognitive diagnosis
  • Consistency
  • Heuristic

ASJC Scopus subject areas

  • Statistics and Probability
  • Arts and Humanities (miscellaneous)
  • Psychology(all)

Fingerprint Dive into the research topics of 'Heuristic cognitive diagnosis when the Q-matrix is unknown'. Together they form a unique fingerprint.

  • Cite this