TY - GEN
T1 - Who benefits? positive learner outcomes from behavioral analytics of online lecture video viewing using classtranscribe
AU - Angrave, Lawrence
AU - Zhang, Zhilin
AU - Henricks, Genevieve
AU - Mahipal, Chirantan
N1 - Funding Information:
We wish to thank the Illinois students who contributed to the ClassTranscribe project, members of the Illinois Computer Science Education group, and the Illinois iLearn Group who helped in the preparation of this paper. We also acknowledge the invaluable technical support from University of Illinois students, staff, and faculty, including Rob Kooper, Dave Mussulman and Scott Cimarusti, Tim Yang, and Jon Gunderson. The research reported here was supported by a Microsoft Corporation gift to the University of Illinois as part of the 2019 Lighthouse Accessibility Microsoft-Illinois partnership and the Institute of Education Sciences, U.S. Department of Education through Grant R305A180211 to the Board of Trustees of the University of Illinois. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.
PY - 2020/2/26
Y1 - 2020/2/26
N2 - Lecture material of a sophomore large-enrollment (N=271) system programming 15-week class was delivered solely online using a new video-based web platform. The platform provided accurate accessible transcriptions and captioning plus a custom text-searchable interface to rapidly find relevant video moments from the entire course. The system logged student searching and viewing behaviors as fine-grained web browser interaction events including fullscreen- switching, loss-of-focus, incremental searching events, and continued-video-watching events with the latter at 15-second granularity. Student learning behaviors and findings from three research questions are presented using individual-level performance and interaction data. Firstly, we report on learning outcomes from alternative learning paths that arise from the course's application of Universal Design for Learning principles. Secondly, final exam performance was equal or better to prior semesters that utilized traditional in-person live lectures. Thirdly, learning outcomes of low and high performing students were analyzed independently by grouping students into four quartiles based on their non-final-exam course performance of programming assignments and quizzes. We introduce and justify an empirically-defined qualification threshold for sufficient video minutes viewed for each group. In all quartiles, students who watched an above-threshold of video minutes improved their in-group final exam performance (ranging from +6% to +14%) with the largest gain for the lowest-performing quartile. The improvement was similar in magnitude for all groups when expressed as a fraction of unrewarded final exam points. Overall, the study presents and evaluates how learner use of online video using ClassTranscribe predicts course performance and positive learning outcomes.
AB - Lecture material of a sophomore large-enrollment (N=271) system programming 15-week class was delivered solely online using a new video-based web platform. The platform provided accurate accessible transcriptions and captioning plus a custom text-searchable interface to rapidly find relevant video moments from the entire course. The system logged student searching and viewing behaviors as fine-grained web browser interaction events including fullscreen- switching, loss-of-focus, incremental searching events, and continued-video-watching events with the latter at 15-second granularity. Student learning behaviors and findings from three research questions are presented using individual-level performance and interaction data. Firstly, we report on learning outcomes from alternative learning paths that arise from the course's application of Universal Design for Learning principles. Secondly, final exam performance was equal or better to prior semesters that utilized traditional in-person live lectures. Thirdly, learning outcomes of low and high performing students were analyzed independently by grouping students into four quartiles based on their non-final-exam course performance of programming assignments and quizzes. We introduce and justify an empirically-defined qualification threshold for sufficient video minutes viewed for each group. In all quartiles, students who watched an above-threshold of video minutes improved their in-group final exam performance (ranging from +6% to +14%) with the largest gain for the lowest-performing quartile. The improvement was similar in magnitude for all groups when expressed as a fraction of unrewarded final exam points. Overall, the study presents and evaluates how learner use of online video using ClassTranscribe predicts course performance and positive learning outcomes.
KW - Accessibility
KW - Behavioral-analytics
KW - Captions
KW - Learning
KW - Learning-analytics
KW - Student-behavior
KW - Transcription-search
KW - Video-search
UR - http://www.scopus.com/inward/record.url?scp=85081594631&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85081594631&partnerID=8YFLogxK
U2 - 10.1145/3328778.3366953
DO - 10.1145/3328778.3366953
M3 - Conference contribution
AN - SCOPUS:85081594631
T3 - Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE
SP - 1193
EP - 1199
BT - SIGCSE 2020 - Proceedings of the 51st ACM Technical Symposium on Computer Science Education
PB - Association for Computing Machinery
T2 - 51st ACM SIGCSE Technical Symposium on Computer Science Education, SIGCSE 2020
Y2 - 11 March 2020 through 14 March 2020
ER -