TY - JOUR
T1 - Identifying supportive student factors for mindset interventions
T2 - A two-model machine learning approach
AU - Bosch, Nigel
N1 - Research reported in this manuscript was supported by the National Study of Learning Mindsets Early Career Fellowship with funding generously provided by the Bezos Family Foundation to Student Experience Research Network (formerly Mindset Scholars Network) and the University of Texas at Austin Population Research Center. The University of Texas at Austin receives core support from the National Institute of Child Health and Human Development under the award number 5R24 HD042849. This manuscript uses data from the National Study of Learning Mindsets (doi:10.3886/ICPSR37353.v1) (PI: D. Yeager; Co-Is: R. Crosnoe, C. Dweck, C. Muller, B. Schneider, & G. Walton), which was made possible through methods and data systems created by the Project for Education Research That Scales (PERTS), data collection carried out by ICF International, meetings hosted by Student Experience Research Network at the Center for Advanced Study in the Behavioral Sciences at Stanford University, assistance from C. Hulleman, R. Ferguson, M. Shankar, T. Brock, C. Romero, D. Paunesku, C. Macrander, T. Wilson, E. Konar, M. Weiss, E. Tipton, and A. Duckworth, and funding from the Raikes Foundation, the William T. Grant Foundation, the Spencer Foundation, the Bezos Family Foundation, the Character Lab, the Houston Endowment, the National Institutes of Health under award number R01HD084772-01, the National Science Foundation under grant number 1761179, Angela Duckworth (personal gift), and the President and Dean of Humanities and Social Sciences at Stanford University. The content is solely the responsibility of the author(s) and does not necessarily represent the official views of the Bezos Family Foundation, Student Experience Research Network, the University of Texas at Austin Population Research Center, the National Institutes of Health, the National Science Foundation, or other funders.
This manuscript uses data from the National Study of Learning Mindsets (doi:10.3886/ICPSR37353.v1) (PI: D. Yeager; Co-Is: R. Crosnoe, C. Dweck, C. Muller, B. Schneider, & G. Walton), which was made possible through methods and data systems created by the Project for Education Research That Scales (PERTS), data collection carried out by ICF International, meetings hosted by Student Experience Research Network at the Center for Advanced Study in the Behavioral Sciences at Stanford University, assistance from C. Hulleman, R. Ferguson, M. Shankar, T. Brock, C. Romero, D. Paunesku, C. Macrander, T. Wilson, E. Konar, M. Weiss, E. Tipton, and A. Duckworth, and funding from the Raikes Foundation, the William T. Grant Foundation, the Spencer Foundation , the Bezos Family Foundation, the Character Lab, the Houston Endowment , the National Institutes of Health under award number R01HD084772-01 , the National Science Foundation under grant number 1761179 , Angela Duckworth (personal gift), and the President and Dean of Humanities and Social Sciences at Stanford University .
Research reported in this manuscript was supported by the National Study of Learning Mindsets Early Career Fellowship with funding generously provided by the Bezos Family Foundation to Student Experience Research Network (formerly Mindset Scholars Network) and the University of Texas at Austin Population Research Center. The University of Texas at Austin receives core support from the National Institute of Child Health and Human Development under the award number 5R24 HD042849 .
PY - 2021/7
Y1 - 2021/7
N2 - Growth mindset interventions foster students’ beliefs that their abilities can grow through effort and appropriate strategies. However, not every student benefits from such interventions – yet research identifying which student factors support growth mindset interventions is sparse. In this study, we utilized machine learning methods to predict growth mindset effectiveness in a nationwide experiment in the U.S. with over 10,000 students. These methods enable analysis of arbitrarily-complex interactions between combinations of student-level predictor variables and intervention outcome, defined as the improvement in grade point average (GPA) during the transition to high school. We utilized two separate machine learning models: one to control for complex relationships between 51 student-level predictors and GPA, and one to predict the change in GPA due to the intervention. We analyzed the trained models to discover which features influenced model predictions most, finding that prior academic achievement, blocked navigations (attempting to navigate through the intervention software too quickly), self-reported reasons for learning, and race/ethnicity were the most important predictors in the model for predicting intervention effectiveness. As in previous research, we found that the intervention was most effective for students with prior low academic achievement. Unique to this study, we found that blocked navigations predicted an intervention effect as low as 0.185 GPA points (on a 0–4 scale) less than the mean. This was a notable negative prediction given that the mean intervention effect in our sample was just 0.026 GPA points, though few students (4.4%) experienced a substantial number of blocked navigation events. We also found that some minoritized students were predicted to benefit less (or even not at all) from the intervention. Our findings have implications for the design of computer-administered growth mindset interventions, especially in relation to students who experience procedural difficulties completing the intervention.
AB - Growth mindset interventions foster students’ beliefs that their abilities can grow through effort and appropriate strategies. However, not every student benefits from such interventions – yet research identifying which student factors support growth mindset interventions is sparse. In this study, we utilized machine learning methods to predict growth mindset effectiveness in a nationwide experiment in the U.S. with over 10,000 students. These methods enable analysis of arbitrarily-complex interactions between combinations of student-level predictor variables and intervention outcome, defined as the improvement in grade point average (GPA) during the transition to high school. We utilized two separate machine learning models: one to control for complex relationships between 51 student-level predictors and GPA, and one to predict the change in GPA due to the intervention. We analyzed the trained models to discover which features influenced model predictions most, finding that prior academic achievement, blocked navigations (attempting to navigate through the intervention software too quickly), self-reported reasons for learning, and race/ethnicity were the most important predictors in the model for predicting intervention effectiveness. As in previous research, we found that the intervention was most effective for students with prior low academic achievement. Unique to this study, we found that blocked navigations predicted an intervention effect as low as 0.185 GPA points (on a 0–4 scale) less than the mean. This was a notable negative prediction given that the mean intervention effect in our sample was just 0.026 GPA points, though few students (4.4%) experienced a substantial number of blocked navigation events. We also found that some minoritized students were predicted to benefit less (or even not at all) from the intervention. Our findings have implications for the design of computer-administered growth mindset interventions, especially in relation to students who experience procedural difficulties completing the intervention.
KW - 21st century abilities
KW - Data science applications in education
KW - Secondary education
UR - http://www.scopus.com/inward/record.url?scp=85102861149&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85102861149&partnerID=8YFLogxK
U2 - 10.1016/j.compedu.2021.104190
DO - 10.1016/j.compedu.2021.104190
M3 - Article
C2 - 35528023
AN - SCOPUS:85102861149
SN - 0360-1315
VL - 167
JO - Computers and Education
JF - Computers and Education
M1 - 104190
ER -