TY - CONF
T1 - Towards a model-free estimate of the limits to student modeling accuracy
AU - Chen, Binglin
AU - West, Matthew
AU - Zilles, Craig
N1 - Funding Information:
This work was partially supported by NSF DUE-1347722, NSF CMMI-1150490, and the College of Engineering at the University of Illinois at Urbana-Champaign under the Strategic Instructional Initiatives Program (SIIP). The authors would like to thank Luc Paquette for useful discussions.
Publisher Copyright:
© 2018 International Educational Data Mining Society. All rights reserved.
PY - 2018
Y1 - 2018
N2 - This paper attempts to quantify the accuracy limit of “next-item-correct” prediction by using numerical optimization to estimate the student’s probability of getting each question correct given a complete sequence of item responses. This optimization is performed without an explicit parameterized model of student behavior, but with the constraint that a student’s likelihood of getting a problem correct only increases or remains unchanged with additional practice (i.e., no forgetting). We present results for this method for the Assistments 2009–2010 data where it suggests that there is only modest opportunity for improvement beyond the state of the art predictors. Furthermore, we describe a framework for applying this method to datasets where problems can be tagged with multiple skills and problem difficulties. Lastly, we discuss the limitations of this method, specifically its inability to give tight bounds on short sequences.
AB - This paper attempts to quantify the accuracy limit of “next-item-correct” prediction by using numerical optimization to estimate the student’s probability of getting each question correct given a complete sequence of item responses. This optimization is performed without an explicit parameterized model of student behavior, but with the constraint that a student’s likelihood of getting a problem correct only increases or remains unchanged with additional practice (i.e., no forgetting). We present results for this method for the Assistments 2009–2010 data where it suggests that there is only modest opportunity for improvement beyond the state of the art predictors. Furthermore, we describe a framework for applying this method to datasets where problems can be tagged with multiple skills and problem difficulties. Lastly, we discuss the limitations of this method, specifically its inability to give tight bounds on short sequences.
UR - http://www.scopus.com/inward/record.url?scp=85084012991&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85084012991&partnerID=8YFLogxK
M3 - Paper
AN - SCOPUS:85084012991
T2 - 11th International Conference on Educational Data Mining, EDM 2018
Y2 - 15 July 2018 through 18 July 2018
ER -