Identifiability Conditions in Cognitive Diagnosis: Implications for Q-Matrix Estimation Algorithms

The Q-matrix of a cognitive diagnosis (CD) assessment documents the item-attribute associations and is thus a key component of any CD test. However, the true Q-matrix underlying a CD assessment is never known; it must be estimated. In practice, this task is typically performed by content experts, which, however, can result in the misspecification of the Q-matrix, causing examinees to be misclassified. In response to these difficulties, algorithms have been developed for estimating the entire Q-matrix based on the item responses. Extant algorithms for estimating the Q-matrix under the conjunctive Deterministic Input Noisy “AND” Gate (DINA) model either impose the identifiability conditions from Chen et al. (J Amer Statist Assoc 110:850–866, 2015) or do not. The debate on which is “right” way to do is ongoing; especially, as these conditions are sufficient but not necessary, which means that viable alternative Q-matrix estimates may be ignored. The goal of this chapter was to compare the estimated Q-matrices obtained from three algorithms that do not impose the identifiability conditions on the Q-matrix estimator with the estimated Q-matrices obtained from two algorithms that do impose the identifiability conditions. Simulations were conducted using data conforming to the DINA model generated in using an identifiable “true” Q-matrix. The impact on Q-matrix estimation of three factors was controlled: the length of the test, the number of attributes, and the amount of error perturbation added to the data. The estimated Q-matrices were evaluated whether they met the identifiability conditions and in their capacity to enable the correct classification of examinees. The results show there is essentially no difference in the rates of correctly classified examinees between Q-matrix estimates obtained from algorithms imposing the identifiability conditions and those that do not.