Improving Our Understanding of Predictive Bias in Testing

Herman Aguinis, Steven A. Culpepper

Research output: Contribution to journalArticlepeer-review

Abstract

Predictive bias (i.e., differential prediction) means that regression equations predicting performance differ across groups based on protected status (e.g., ethnicity, sexual orientation, sexual identity, pregnancy, disability, and religion). Thus, making prescreening, admissions, and selection decisions when predictive bias exists violates principles of fairness based on equal treatment and opportunity. First, we conducted a two-part study showing that different types of predictive bias exist. Specifically, we conducted a Monte Carlo simulation showing that out-of-sample predictions provide a more precise understanding of the nature of predictive bias—whether it is based on intercept and/or slope differences across groups. Then, we conducted a college admissions study based on 29,734 Black and 304,372 White students, and 35,681 Latinx and 308,818 White students and provided evidence about the existence of both intercept- and slope-based predictive bias. Third, we discuss the nature and different types of predictive bias and offer analytical work to explain why each type exists, thereby providing insights into the causes of different types of predictive bias. We also map the statistical causes of predictive bias onto the existing literature on likely underlying psychological and contextual mechanisms. Overall, we hope our article will help reorient future predictive bias research from whether it exists to the why of different types of predictive bias.

Original languageEnglish (US)
Pages (from-to)402-414
Number of pages13
JournalJournal of Applied Psychology
Volume109
Issue number3
DOIs
StatePublished - Mar 2024

Keywords

  • affirmative action
  • and inclusion
  • diversity
  • equal opportunity
  • equity
  • fairness
  • test bias

ASJC Scopus subject areas

  • Applied Psychology

Fingerprint

Dive into the research topics of 'Improving Our Understanding of Predictive Bias in Testing'. Together they form a unique fingerprint.

Cite this