How to Build a Complete Q-Matrix for a Cognitively Diagnostic Test

Hans Friedrich Koehn, Chia Yi Chiu

Research output: Contribution to journalArticle

Abstract

The Q-matrix of a cognitively diagnostic test is said to be complete if it guarantees the identifiability of all possible proficiency classes among examinees. An incomplete Q-matrix causes examinees to be assigned to proficiency classes to which they do not belong. Completeness of the Q-matrix is therefore a key requirement of any cognitively diagnostic test. The importance of the completeness property of the Q-matrix of a test as a fundamental condition to guarantee a reliable estimate of an examinee’s attribute profile has only recently been realized by researchers. In fact, inspection of extant assessments based on the cognitive diagnosis framework often revealed that, in hindsight, the Q-matrices used with these tests were not complete. Thus, the availability of rules for building a complete Q-matrix at the early stages of test development is perhaps at least as desirable as rules for identifying the completeness of a given Q-matrix. This article presents procedures for constructing Q-matrices that are complete. The famous Fraction-Subtraction test problems by K. K. Tatsuoka (1984) are used throughout for illustration.

Original languageEnglish (US)
Pages (from-to)273-299
Number of pages27
JournalJournal of Classification
Volume35
Issue number2
DOIs
StatePublished - Jul 1 2018

Fingerprint

Q-matrix
Diagnostic Tests
Routine Diagnostic Tests
diagnostic
Research Personnel
Completeness
guarantee
Diagnostic tests
Identifiability
Subtraction
Test Problems
Inspection
cause
Availability
Attribute
Requirements

Keywords

  • Cognitive Diagnosis
  • Completeness
  • Diagnostic Classification Models (DCMs)
  • General DCMs
  • Q-matrix

ASJC Scopus subject areas

  • Mathematics (miscellaneous)
  • Psychology (miscellaneous)
  • Statistics, Probability and Uncertainty
  • Library and Information Sciences

Cite this

How to Build a Complete Q-Matrix for a Cognitively Diagnostic Test. / Koehn, Hans Friedrich; Chiu, Chia Yi.

In: Journal of Classification, Vol. 35, No. 2, 01.07.2018, p. 273-299.

Research output: Contribution to journalArticle

@article{8dbed2c19d1748f887037026a7308c70,
title = "How to Build a Complete Q-Matrix for a Cognitively Diagnostic Test",
abstract = "The Q-matrix of a cognitively diagnostic test is said to be complete if it guarantees the identifiability of all possible proficiency classes among examinees. An incomplete Q-matrix causes examinees to be assigned to proficiency classes to which they do not belong. Completeness of the Q-matrix is therefore a key requirement of any cognitively diagnostic test. The importance of the completeness property of the Q-matrix of a test as a fundamental condition to guarantee a reliable estimate of an examinee’s attribute profile has only recently been realized by researchers. In fact, inspection of extant assessments based on the cognitive diagnosis framework often revealed that, in hindsight, the Q-matrices used with these tests were not complete. Thus, the availability of rules for building a complete Q-matrix at the early stages of test development is perhaps at least as desirable as rules for identifying the completeness of a given Q-matrix. This article presents procedures for constructing Q-matrices that are complete. The famous Fraction-Subtraction test problems by K. K. Tatsuoka (1984) are used throughout for illustration.",
keywords = "Cognitive Diagnosis, Completeness, Diagnostic Classification Models (DCMs), General DCMs, Q-matrix",
author = "Koehn, {Hans Friedrich} and Chiu, {Chia Yi}",
year = "2018",
month = "7",
day = "1",
doi = "10.1007/s00357-018-9255-0",
language = "English (US)",
volume = "35",
pages = "273--299",
journal = "Journal of Classification",
issn = "0176-4268",
publisher = "Springer New York",
number = "2",

}

TY - JOUR

T1 - How to Build a Complete Q-Matrix for a Cognitively Diagnostic Test

AU - Koehn, Hans Friedrich

AU - Chiu, Chia Yi

PY - 2018/7/1

Y1 - 2018/7/1

N2 - The Q-matrix of a cognitively diagnostic test is said to be complete if it guarantees the identifiability of all possible proficiency classes among examinees. An incomplete Q-matrix causes examinees to be assigned to proficiency classes to which they do not belong. Completeness of the Q-matrix is therefore a key requirement of any cognitively diagnostic test. The importance of the completeness property of the Q-matrix of a test as a fundamental condition to guarantee a reliable estimate of an examinee’s attribute profile has only recently been realized by researchers. In fact, inspection of extant assessments based on the cognitive diagnosis framework often revealed that, in hindsight, the Q-matrices used with these tests were not complete. Thus, the availability of rules for building a complete Q-matrix at the early stages of test development is perhaps at least as desirable as rules for identifying the completeness of a given Q-matrix. This article presents procedures for constructing Q-matrices that are complete. The famous Fraction-Subtraction test problems by K. K. Tatsuoka (1984) are used throughout for illustration.

AB - The Q-matrix of a cognitively diagnostic test is said to be complete if it guarantees the identifiability of all possible proficiency classes among examinees. An incomplete Q-matrix causes examinees to be assigned to proficiency classes to which they do not belong. Completeness of the Q-matrix is therefore a key requirement of any cognitively diagnostic test. The importance of the completeness property of the Q-matrix of a test as a fundamental condition to guarantee a reliable estimate of an examinee’s attribute profile has only recently been realized by researchers. In fact, inspection of extant assessments based on the cognitive diagnosis framework often revealed that, in hindsight, the Q-matrices used with these tests were not complete. Thus, the availability of rules for building a complete Q-matrix at the early stages of test development is perhaps at least as desirable as rules for identifying the completeness of a given Q-matrix. This article presents procedures for constructing Q-matrices that are complete. The famous Fraction-Subtraction test problems by K. K. Tatsuoka (1984) are used throughout for illustration.

KW - Cognitive Diagnosis

KW - Completeness

KW - Diagnostic Classification Models (DCMs)

KW - General DCMs

KW - Q-matrix

UR - http://www.scopus.com/inward/record.url?scp=85052629366&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052629366&partnerID=8YFLogxK

U2 - 10.1007/s00357-018-9255-0

DO - 10.1007/s00357-018-9255-0

M3 - Article

AN - SCOPUS:85052629366

VL - 35

SP - 273

EP - 299

JO - Journal of Classification

JF - Journal of Classification

SN - 0176-4268

IS - 2

ER -