Intrinsically Interpretable Artificial Neural Networks for Learner Modeling

Modern AI algorithms are so complex, that it is often impossible even for expert AI engineers to fully explain how they make decisions. Researchers in education are increasingly using such“black-box”algorithms for a wide variety of tasks. This lack of transparency has rightfully raised concerns over issues of fairness, accountability, and trust. Post-hoc explainability techniques exist that aim to address this issue. However, studies in both educational and non-educational contexts have highlighted fundamental problems with these approaches. In this proposed project, we take an alternative approach that aims to make complex AI learner models more intrinsically interpretable, while illustrating how such interpretability can be evaluated. We aim to (1) develop an interpretable neural network, comparing accuracy and issues relevant to interpretability approaches as a whole, (2) evaluate this model’s level of interpretability using a humangrounded evaluation approach, and (3) validate the model’s inner representations and explore some hypothetical advantages of interpretable models, including their use for knowledge discovery.