### Abstract

This paper attempts to quantify the accuracy limit of “next-item-correct” prediction by using numerical optimization to estimate the student’s probability of getting each question correct given a complete sequence of item responses. This optimization is performed without an explicit parameterized model of student behavior, but with the constraint that a student’s likelihood of getting a problem correct only increases or remains unchanged with additional practice (i.e., no forgetting). We present results for this method for the Assistments 2009–2010 data where it suggests that there is only modest opportunity for improvement beyond the state of the art predictors. Furthermore, we describe a framework for applying this method to datasets where problems can be tagged with multiple skills and problem difficulties. Lastly, we discuss the limitations of this method, specifically its inability to give tight bounds on short sequences.

Original language | English (US) |
---|---|

State | Published - Jan 1 2018 |

Event | 11th International Conference on Educational Data Mining, EDM 2018 - Buffalo, United States Duration: Jul 15 2018 → Jul 18 2018 |

### Conference

Conference | 11th International Conference on Educational Data Mining, EDM 2018 |
---|---|

Country | United States |

City | Buffalo |

Period | 7/15/18 → 7/18/18 |

### Fingerprint

### ASJC Scopus subject areas

- Computer Science Applications
- Information Systems

### Cite this

*Towards a model-free estimate of the limits to student modeling accuracy*. Paper presented at 11th International Conference on Educational Data Mining, EDM 2018, Buffalo, United States.

**Towards a model-free estimate of the limits to student modeling accuracy.** / Chen, Binglin; West, Matthew; Zilles, Craig.

Research output: Contribution to conference › Paper

}

TY - CONF

T1 - Towards a model-free estimate of the limits to student modeling accuracy

AU - Chen, Binglin

AU - West, Matthew

AU - Zilles, Craig

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This paper attempts to quantify the accuracy limit of “next-item-correct” prediction by using numerical optimization to estimate the student’s probability of getting each question correct given a complete sequence of item responses. This optimization is performed without an explicit parameterized model of student behavior, but with the constraint that a student’s likelihood of getting a problem correct only increases or remains unchanged with additional practice (i.e., no forgetting). We present results for this method for the Assistments 2009–2010 data where it suggests that there is only modest opportunity for improvement beyond the state of the art predictors. Furthermore, we describe a framework for applying this method to datasets where problems can be tagged with multiple skills and problem difficulties. Lastly, we discuss the limitations of this method, specifically its inability to give tight bounds on short sequences.

AB - This paper attempts to quantify the accuracy limit of “next-item-correct” prediction by using numerical optimization to estimate the student’s probability of getting each question correct given a complete sequence of item responses. This optimization is performed without an explicit parameterized model of student behavior, but with the constraint that a student’s likelihood of getting a problem correct only increases or remains unchanged with additional practice (i.e., no forgetting). We present results for this method for the Assistments 2009–2010 data where it suggests that there is only modest opportunity for improvement beyond the state of the art predictors. Furthermore, we describe a framework for applying this method to datasets where problems can be tagged with multiple skills and problem difficulties. Lastly, we discuss the limitations of this method, specifically its inability to give tight bounds on short sequences.

UR - http://www.scopus.com/inward/record.url?scp=85072292658&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072292658&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85072292658

ER -