A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests

Hans Friedrich Koehn, Chia Yi Chiu

Research output: Contribution to journalArticle

Abstract

The Q-matrix of a cognitively diagnostic test is said to be complete if it allows for the identification of all possible proficiency classes among examinees. Completeness of the Q-matrix is therefore a key requirement for any cognitively diagnostic test. However, completeness of the Q-matrix is often difficult to establish, especially, for tests with a large number of items involving multiple attributes. As an additional complication, completeness is not an intrinsic property of the Q-matrix, but can only be assessed in reference to a specific cognitive diagnosis model (CDM) supposed to underly the data—that is, the Q-matrix of a given test can be complete for one model but incomplete for another. In this article, a method is presented for assessing whether a given Q-matrix is complete for a given CDM. The proposed procedure relies on the theoretical framework of general CDMs and is therefore legitimate for CDMs that can be reparameterized as a general CDM.

Original languageEnglish (US)
Pages (from-to)112-132
Number of pages21
JournalPsychometrika
Volume82
Issue number1
DOIs
StatePublished - Mar 1 2017

Fingerprint

Q-matrix
Diagnostic Tests
Routine Diagnostic Tests
Completeness
Complications
Model
Attribute
Requirements

Keywords

  • G-DINA
  • LCDM
  • Q-matrix
  • cognitive diagnosis
  • completeness
  • general cognitive diagnostic models

ASJC Scopus subject areas

  • Psychology(all)
  • Applied Mathematics

Cite this

A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests. / Koehn, Hans Friedrich; Chiu, Chia Yi.

In: Psychometrika, Vol. 82, No. 1, 01.03.2017, p. 112-132.

Research output: Contribution to journalArticle

@article{8e989636d26c47d385e40914263e6678,
title = "A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests",
abstract = "The Q-matrix of a cognitively diagnostic test is said to be complete if it allows for the identification of all possible proficiency classes among examinees. Completeness of the Q-matrix is therefore a key requirement for any cognitively diagnostic test. However, completeness of the Q-matrix is often difficult to establish, especially, for tests with a large number of items involving multiple attributes. As an additional complication, completeness is not an intrinsic property of the Q-matrix, but can only be assessed in reference to a specific cognitive diagnosis model (CDM) supposed to underly the data—that is, the Q-matrix of a given test can be complete for one model but incomplete for another. In this article, a method is presented for assessing whether a given Q-matrix is complete for a given CDM. The proposed procedure relies on the theoretical framework of general CDMs and is therefore legitimate for CDMs that can be reparameterized as a general CDM.",
keywords = "G-DINA, LCDM, Q-matrix, cognitive diagnosis, completeness, general cognitive diagnostic models",
author = "Koehn, {Hans Friedrich} and Chiu, {Chia Yi}",
year = "2017",
month = "3",
day = "1",
doi = "10.1007/s11336-016-9536-7",
language = "English (US)",
volume = "82",
pages = "112--132",
journal = "Psychometrika",
issn = "0033-3123",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests

AU - Koehn, Hans Friedrich

AU - Chiu, Chia Yi

PY - 2017/3/1

Y1 - 2017/3/1

N2 - The Q-matrix of a cognitively diagnostic test is said to be complete if it allows for the identification of all possible proficiency classes among examinees. Completeness of the Q-matrix is therefore a key requirement for any cognitively diagnostic test. However, completeness of the Q-matrix is often difficult to establish, especially, for tests with a large number of items involving multiple attributes. As an additional complication, completeness is not an intrinsic property of the Q-matrix, but can only be assessed in reference to a specific cognitive diagnosis model (CDM) supposed to underly the data—that is, the Q-matrix of a given test can be complete for one model but incomplete for another. In this article, a method is presented for assessing whether a given Q-matrix is complete for a given CDM. The proposed procedure relies on the theoretical framework of general CDMs and is therefore legitimate for CDMs that can be reparameterized as a general CDM.

AB - The Q-matrix of a cognitively diagnostic test is said to be complete if it allows for the identification of all possible proficiency classes among examinees. Completeness of the Q-matrix is therefore a key requirement for any cognitively diagnostic test. However, completeness of the Q-matrix is often difficult to establish, especially, for tests with a large number of items involving multiple attributes. As an additional complication, completeness is not an intrinsic property of the Q-matrix, but can only be assessed in reference to a specific cognitive diagnosis model (CDM) supposed to underly the data—that is, the Q-matrix of a given test can be complete for one model but incomplete for another. In this article, a method is presented for assessing whether a given Q-matrix is complete for a given CDM. The proposed procedure relies on the theoretical framework of general CDMs and is therefore legitimate for CDMs that can be reparameterized as a general CDM.

KW - G-DINA

KW - LCDM

KW - Q-matrix

KW - cognitive diagnosis

KW - completeness

KW - general cognitive diagnostic models

UR - http://www.scopus.com/inward/record.url?scp=84990848104&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990848104&partnerID=8YFLogxK

U2 - 10.1007/s11336-016-9536-7

DO - 10.1007/s11336-016-9536-7

M3 - Article

C2 - 27714544

AN - SCOPUS:84990848104

VL - 82

SP - 112

EP - 132

JO - Psychometrika

JF - Psychometrika

SN - 0033-3123

IS - 1

ER -