How Students Search Video Captions to Learn: An Analysis of Search Terms and Behavioral Timing Data

Zhilin Zhang, Bhavya Bhavya, Lawrence Angrave, Ruihua Sui, Rob Kooper, Chirantan Mahipal, Yun Huang

Research output: Contribution to journalConference articlepeer-review


Engineering students used ClassTranscribe, an accessible video player, in multiple engineering courses to view course videos and search for video content. The tool collected detailed timestamped student behavioral data from 1,894 students across 25 engineering courses that included what individual students searched for and when. A previous analysis, published in ASEE 2020 [1], found that using ClassTranscribe caption search significantly predicted improvement in final exam scores in a computer science course. In this paper we present how students used the search functionality based on a more detailed analysis of the log data. ClassTranscribe automatically created captions and transcripts for all lecture videos using an Azure speech-to-text system that was supplemented with crowd-sourced editing to fix captioning errors. The search functionality used the timestamped caption data to find specific video moments both within the current video or across the entire course. The number of search activities per person ranged from zero to 186 events. An in-depth analysis of the students (N=167) who performed 1,022 searches was conducted to gain insight into student search needs and behaviors. Based on the total number of searches performed, students were grouped into “Infrequent Searcher” (< 18 searches) and “Frequent Searcher” (18 to 110 searches) using clustering algorithms. The search queries used by each group were found to follow the Zipf's Law and were categorized into STEM-related terms, course logistics and others. Our study reports on students' search context, behaviors, strategies, and optimizations. Using Universal Design for Learning as a foundation, we discuss the implications for educators, designers, and developers who are interested in providing new learning pathways to support and enhance video-based learning environments.

Original languageEnglish (US)
JournalASEE Annual Conference and Exposition, Conference Proceedings
StatePublished - Jul 26 2021
Event2021 ASEE Virtual Annual Conference, ASEE 2021 - Virtual, Online
Duration: Jul 26 2021Jul 29 2021

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'How Students Search Video Captions to Learn: An Analysis of Search Terms and Behavioral Timing Data'. Together they form a unique fingerprint.

Cite this