Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles

Victoria L. Quirk, Justin L. Kern

Research output: Contribution to journalArticlepeer-review

Abstract

The measurement of psychological constructs is frequently based on self-report tests, which often have Likert-type items rated from “Strongly Disagree” to “Strongly Agree”. Recently, a family of item response theory (IRT) models called IRTree models have emerged that can parse out content traits (e.g., personality traits) from noise traits (e.g., response styles). In this study, we compare the selection validity and adverse impact consequences of noise traits on selection when scores are estimated using a generalized partial credit model (GPCM) or an IRTree model. First, we present a simulation which demonstrates that when noise traits do exist, the selection decisions made based on the IRTree model estimated scores have higher accuracy rates and have less instances of adverse impact based on extreme response style group membership when compared to the GPCM. Both models performed similarly when there was no influence of noise traits on the responses. Second, we present an application using data collected from the Open-Source Psychometrics Project Fisher Temperament Inventory dataset. We found that the IRTree model had a better fit, but a high agreement rate between the model decisions resulted in virtually identical impact ratios between the models. We offer considerations for applications of the IRTree model and future directions for research.
Original languageEnglish (US)
Article number216
JournalJournal of Intelligence
Volume11
Issue number11
DOIs
StatePublished - Nov 2023

Keywords

  • item response theory
  • adverse impact
  • classification accuracy
  • selection validity
  • Likert scales
  • extreme response styles
  • non-content variability
  • response process
  • IRTree models
  • item response trees

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Education
  • Developmental and Educational Psychology
  • Cognitive Neuroscience

Fingerprint

Dive into the research topics of 'Using IRTree Models to Promote Selection Validity in the Presence of Extreme Response Styles'. Together they form a unique fingerprint.

Cite this