Measuring revealed student scheduling preferences using constrained discrete choice models

Jacob Bailey, Matthew West, Craig Zilles

Research output: Contribution to journalConference article

Abstract

For constrained student resources with large student populations it is often necessary to implement some form of reservation or scheduling system. Examples of scheduled-access resources can include one-on-one tutoring, machine shops or labs, and computer-based testing facilities. For planning and resource scheduling purposes it is important to be able to forecast demand, and thus it is important to understand what drives student preferences for particular scheduling time slots. Measuring these preferences can be challenging, however, for at least the following three reasons. (1) Revealed preferences (what students actually choose) can differ significantly from stated preferences (what they say they will want at a future time), requiring the use of actual scheduling data to infer preferences or utilities. (2) The utility that students derive from particular choices is multifactorial, so that in a computer-based testing facility, for example, students may prefer to take their exam mid-afternoon, but they may also prefer to take it as close to the end of the exam period as possible, and it can be difficult to disentangle these factors. (3) Capacity constraints will frequently lead to many time slots being fully reserved, making it unclear which slots were actually preferred. This paper presents a general framework for measuring revealed student preferences from actual reservation or scheduling data. This framework is based on the theory of constrained discrete choice modeling, as used in economics for modeling consumer preferences. A multifactorial random utility model (RUM) is formulated for student scheduling preferences and the model is trained on scheduling data using maximum likelihood estimation (MLE) and cross-validated on multiple rounds of training/test data splits. Results are presented using scheduling data from a computer-based testing facility with approximately 50, 000 student reservations over three semesters (Spring 2015 to Spring 2016, inclusive). We show that this measurement methodology can accurately capture student preferences in real-world scheduling data and can successfully separate out time-in-week preferences from time-within-exam preferences. Errors are quantified using both log-likelihood with per-reservation data and root mean square error (RMSE) with data aggregated to the time slot level. We discuss both estimation and simulation algorithms for constrained discrete choice models and discuss how Monte Carlo simulation can be used to obtain uncertainty predictions for predicting expected usage.

Original languageEnglish (US)
JournalASEE Annual Conference and Exposition, Conference Proceedings
Volume2017-June
StatePublished - Jun 24 2017
Event124th ASEE Annual Conference and Exposition - Columbus, United States
Duration: Jun 25 2017Jun 28 2017

Fingerprint

Scheduling
Students
Testing
Machine shops
Maximum likelihood estimation
Mean square error
Planning
Economics

ASJC Scopus subject areas

  • Engineering(all)

Cite this

@article{2a7ad72942374f7ebc97ca23a3e3462c,
title = "Measuring revealed student scheduling preferences using constrained discrete choice models",
abstract = "For constrained student resources with large student populations it is often necessary to implement some form of reservation or scheduling system. Examples of scheduled-access resources can include one-on-one tutoring, machine shops or labs, and computer-based testing facilities. For planning and resource scheduling purposes it is important to be able to forecast demand, and thus it is important to understand what drives student preferences for particular scheduling time slots. Measuring these preferences can be challenging, however, for at least the following three reasons. (1) Revealed preferences (what students actually choose) can differ significantly from stated preferences (what they say they will want at a future time), requiring the use of actual scheduling data to infer preferences or utilities. (2) The utility that students derive from particular choices is multifactorial, so that in a computer-based testing facility, for example, students may prefer to take their exam mid-afternoon, but they may also prefer to take it as close to the end of the exam period as possible, and it can be difficult to disentangle these factors. (3) Capacity constraints will frequently lead to many time slots being fully reserved, making it unclear which slots were actually preferred. This paper presents a general framework for measuring revealed student preferences from actual reservation or scheduling data. This framework is based on the theory of constrained discrete choice modeling, as used in economics for modeling consumer preferences. A multifactorial random utility model (RUM) is formulated for student scheduling preferences and the model is trained on scheduling data using maximum likelihood estimation (MLE) and cross-validated on multiple rounds of training/test data splits. Results are presented using scheduling data from a computer-based testing facility with approximately 50, 000 student reservations over three semesters (Spring 2015 to Spring 2016, inclusive). We show that this measurement methodology can accurately capture student preferences in real-world scheduling data and can successfully separate out time-in-week preferences from time-within-exam preferences. Errors are quantified using both log-likelihood with per-reservation data and root mean square error (RMSE) with data aggregated to the time slot level. We discuss both estimation and simulation algorithms for constrained discrete choice models and discuss how Monte Carlo simulation can be used to obtain uncertainty predictions for predicting expected usage.",
author = "Jacob Bailey and Matthew West and Craig Zilles",
year = "2017",
month = "6",
day = "24",
language = "English (US)",
volume = "2017-June",
journal = "ASEE Annual Conference and Exposition, Conference Proceedings",
issn = "2153-5965",

}

TY - JOUR

T1 - Measuring revealed student scheduling preferences using constrained discrete choice models

AU - Bailey, Jacob

AU - West, Matthew

AU - Zilles, Craig

PY - 2017/6/24

Y1 - 2017/6/24

N2 - For constrained student resources with large student populations it is often necessary to implement some form of reservation or scheduling system. Examples of scheduled-access resources can include one-on-one tutoring, machine shops or labs, and computer-based testing facilities. For planning and resource scheduling purposes it is important to be able to forecast demand, and thus it is important to understand what drives student preferences for particular scheduling time slots. Measuring these preferences can be challenging, however, for at least the following three reasons. (1) Revealed preferences (what students actually choose) can differ significantly from stated preferences (what they say they will want at a future time), requiring the use of actual scheduling data to infer preferences or utilities. (2) The utility that students derive from particular choices is multifactorial, so that in a computer-based testing facility, for example, students may prefer to take their exam mid-afternoon, but they may also prefer to take it as close to the end of the exam period as possible, and it can be difficult to disentangle these factors. (3) Capacity constraints will frequently lead to many time slots being fully reserved, making it unclear which slots were actually preferred. This paper presents a general framework for measuring revealed student preferences from actual reservation or scheduling data. This framework is based on the theory of constrained discrete choice modeling, as used in economics for modeling consumer preferences. A multifactorial random utility model (RUM) is formulated for student scheduling preferences and the model is trained on scheduling data using maximum likelihood estimation (MLE) and cross-validated on multiple rounds of training/test data splits. Results are presented using scheduling data from a computer-based testing facility with approximately 50, 000 student reservations over three semesters (Spring 2015 to Spring 2016, inclusive). We show that this measurement methodology can accurately capture student preferences in real-world scheduling data and can successfully separate out time-in-week preferences from time-within-exam preferences. Errors are quantified using both log-likelihood with per-reservation data and root mean square error (RMSE) with data aggregated to the time slot level. We discuss both estimation and simulation algorithms for constrained discrete choice models and discuss how Monte Carlo simulation can be used to obtain uncertainty predictions for predicting expected usage.

AB - For constrained student resources with large student populations it is often necessary to implement some form of reservation or scheduling system. Examples of scheduled-access resources can include one-on-one tutoring, machine shops or labs, and computer-based testing facilities. For planning and resource scheduling purposes it is important to be able to forecast demand, and thus it is important to understand what drives student preferences for particular scheduling time slots. Measuring these preferences can be challenging, however, for at least the following three reasons. (1) Revealed preferences (what students actually choose) can differ significantly from stated preferences (what they say they will want at a future time), requiring the use of actual scheduling data to infer preferences or utilities. (2) The utility that students derive from particular choices is multifactorial, so that in a computer-based testing facility, for example, students may prefer to take their exam mid-afternoon, but they may also prefer to take it as close to the end of the exam period as possible, and it can be difficult to disentangle these factors. (3) Capacity constraints will frequently lead to many time slots being fully reserved, making it unclear which slots were actually preferred. This paper presents a general framework for measuring revealed student preferences from actual reservation or scheduling data. This framework is based on the theory of constrained discrete choice modeling, as used in economics for modeling consumer preferences. A multifactorial random utility model (RUM) is formulated for student scheduling preferences and the model is trained on scheduling data using maximum likelihood estimation (MLE) and cross-validated on multiple rounds of training/test data splits. Results are presented using scheduling data from a computer-based testing facility with approximately 50, 000 student reservations over three semesters (Spring 2015 to Spring 2016, inclusive). We show that this measurement methodology can accurately capture student preferences in real-world scheduling data and can successfully separate out time-in-week preferences from time-within-exam preferences. Errors are quantified using both log-likelihood with per-reservation data and root mean square error (RMSE) with data aggregated to the time slot level. We discuss both estimation and simulation algorithms for constrained discrete choice models and discuss how Monte Carlo simulation can be used to obtain uncertainty predictions for predicting expected usage.

UR - http://www.scopus.com/inward/record.url?scp=85030529510&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030529510&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85030529510

VL - 2017-June

JO - ASEE Annual Conference and Exposition, Conference Proceedings

JF - ASEE Annual Conference and Exposition, Conference Proceedings

SN - 2153-5965

ER -