TY - GEN
T1 - Learning Behaviors Mediate the Effect of AI-powered Support for Metacognitive Calibration on Learning Outcomes
AU - Lee, Hae Jin
AU - Stinar, Frank
AU - Zong, Ruohan
AU - Valdiviejas, Hannah
AU - Wang, Dong
AU - Bosch, Nigel
N1 - This material is based upon work supported by the National Science Foundation under Award No. IIS-2202481.
PY - 2025/4/26
Y1 - 2025/4/26
N2 - Students struggle with accurately assessing their own performance, especially given little training to do so. We propose an AI-powered training tool to help students improve "metacognitive calibration,"or the ability to accurately predict their own learning, potentially enhancing learning outcomes by enabling students' use of metacognition-informed learning behaviors. We present results from a randomized controlled trial (N = 133) assessing the effectiveness of the tool in a college-level computer-based learning environment. The AI-driven tool significantly improved learning gains compared to the control group by 8.9% (t = -2.384, p =.019), and this effect was significantly mediated by learning behaviors. Overconfident students who received the intervention showed significantly greater metacognitive calibration improvement than the control group by 4.1% (t = 2.001, p =.049). These insights highlight the value of AI-powered metacognitive calibration training and the importance of promoting specific metacognition-informed learning behaviors in computer-based learning.
AB - Students struggle with accurately assessing their own performance, especially given little training to do so. We propose an AI-powered training tool to help students improve "metacognitive calibration,"or the ability to accurately predict their own learning, potentially enhancing learning outcomes by enabling students' use of metacognition-informed learning behaviors. We present results from a randomized controlled trial (N = 133) assessing the effectiveness of the tool in a college-level computer-based learning environment. The AI-driven tool significantly improved learning gains compared to the control group by 8.9% (t = -2.384, p =.019), and this effect was significantly mediated by learning behaviors. Overconfident students who received the intervention showed significantly greater metacognitive calibration improvement than the control group by 4.1% (t = 2.001, p =.049). These insights highlight the value of AI-powered metacognitive calibration training and the importance of promoting specific metacognition-informed learning behaviors in computer-based learning.
KW - Computer-based Learning Environments
KW - Explainable AI
KW - Human-computer Interaction
KW - Metacognitive Calibration
KW - Self-regulated Learning
UR - https://www.scopus.com/pages/publications/105005738869
UR - https://www.scopus.com/pages/publications/105005738869#tab=citedBy
U2 - 10.1145/3706598.3713960
DO - 10.1145/3706598.3713960
M3 - Conference contribution
AN - SCOPUS:105005738869
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2025 - Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery
T2 - 2025 CHI Conference on Human Factors in Computing Systems, CHI 2025
Y2 - 26 April 2025 through 1 May 2025
ER -