Automatic assessment of complex assignments using topic models

Saar Kuzi, William Cope, Duncan Ferguson, Chase Geigle, Cheng Xiang Zhai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Automated assessment of complex assignments is crucial for scaling up learning of complex skills such as critical thinking. To address this challenge, one previous work has applied supervised machine learning to automate the assessment by learning from examples of graded assignments by humans. However, in the previous work, only simple lexical features, such as words or n-grams, have been used. In this paper, we propose to use topics as features for this task, which are more interpretable than those simple lexical features and can also address polysemy and synonymy of lexical semantics. The topics can be learned automatically from the student assignment data by using a probabilistic topic model. We propose and study multiple approaches to construct topical features and to combine topical features with simple lexical features. We evaluate the proposed methods using clinical case assignments performed by veterinary medicine students. The experimental results show that topical features are generally very effective and can substantially improve performance when added on top of the lexical features. However, their effectiveness is highly sensitive to how the topics are constructed and a combination of topics constructed using multiple views of the text data works the best. Our results also show that combining the prediction results of using different types of topical features and of topical and lexical features is more effective than pooling all features together to form a larger feature space.

Original languageEnglish (US)
Title of host publicationProceedings of the 6th 2019 ACM Conference on Learning at Scale, L@S 2019
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450368049
DOIs
StatePublished - Jun 24 2019
Event6th ACM Conference on Learning at Scale, L@S 2019 - Chicago, United States
Duration: Jun 24 2019Jun 25 2019

Publication series

NameProceedings of the 6th 2019 ACM Conference on Learning at Scale, L@S 2019

Conference

Conference6th ACM Conference on Learning at Scale, L@S 2019
CountryUnited States
CityChicago
Period6/24/196/25/19

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Education

Fingerprint Dive into the research topics of 'Automatic assessment of complex assignments using topic models'. Together they form a unique fingerprint.

Cite this