Hierarchical Dependencies in Classroom Settings Influence Algorithmic Bias Metrics

Clara Belitz, Hae Jin Lee, Nidhi Nasiar, Stephen E. Fancsali, Steve Ritter, Husni Almoubayyed, Ryan S. Baker, Jaclyn Ocumpaugh, Nigel Bosch

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Measuring algorithmic bias in machine learning has historically focused on statistical inequalities pertaining to specific groups. However, the most common metrics (i.e., those focused on individual- or group-conditioned error rates) are not currently well-suited to educational settings because they assume that each individual observation is independent from the others. This is not statistically appropriate when studying certain common educational outcomes, because such metrics cannot account for the relationship between students in classrooms or multiple observations per student across an academic year. In this paper, we present novel adaptations of algorithmic bias measurements for regression for both independent and nested data structures. Using hierarchical linear models, we rigorously measure algorithmic bias in a machine learning model of the relationship between student engagement in an intelligent tutoring system and year-end standardized test scores. We conclude that classroom-level influences had a small but significant effect on models. Examining significance with hierarchical linear models helps determine which inequalities in educational settings might be explained by small sample sizes rather than systematic differences.

Original languageEnglish (US)
Title of host publicationLAK 2024 Conference Proceedings - 14th International Conference on Learning Analytics and Knowledge
PublisherAssociation for Computing Machinery
Number of pages9
ISBN (Electronic)9798400716188
StatePublished - Mar 18 2024
Event14th International Conference on Learning Analytics and Knowledge, LAK 2024 - Kyoto, Japan
Duration: Mar 18 2024Mar 22 2024

Publication series

NameACM International Conference Proceeding Series


Conference14th International Conference on Learning Analytics and Knowledge, LAK 2024


  • Algorithmic bias
  • Intelligent tutoring systems
  • Interactive learning environments
  • Predictive analytics

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software


Dive into the research topics of 'Hierarchical Dependencies in Classroom Settings Influence Algorithmic Bias Metrics'. Together they form a unique fingerprint.

Cite this