Evaluating the Low-Stakes Assessment Performance: Student-Perceived Accessibility, Belongingness, and Self-Efficacy in Connection to the Use of Digital Notes in Engineering and Computing Courses

Course content plays a critical role in student success. Among all college students, students with disabilities (SWD) face numerous additional challenges when digital content is inaccessible or difficult to use. Digital note-taking has been increasingly implemented and studied in higher education for its potential to further develop universal design for learning (UDL) techniques to benefit all students, especially SWD. This study combines low-stakes assessment results with surveys about accessibility, belongingness, self-efficacy and perceived learning in connection to the use of digital notes. The multi-discipline, multi-site, and multi-timepoint research design includes two junior-level undergraduate probability and statistics courses in the disciplines of Computer Science and Industrial Engineering at the University of Illinois Urbana Champaign (UIUC). The study investigated the effect of digital notes introduced at week 10, produced using ClassTranscribe developed at UIUC. The digital notes allow for transcripts, screenshots, and mathematical equations taken directly from video recordings. The digital notes were provided to the students of the two classes in both.pdf and.epub formats with included links to the video. Additionally, we considered Students With Accessibility Needs (SWAN) if they reported conditions that prevented them from attending class at some point while not having an officially recognized disability by the university. The students in the classes were surveyed three times: a baseline survey followed by two posttest surveys. The baseline survey was conducted early in the Fall 2022 semester by measuring the students' perceived accessibility of the course using the Web Content Accessibility Guidelines (WCAG)-based POUR model in the context of mathematically-rich engineering courses. The POUR model (perceivable, operable, understandable, and robust subscales) was found previously to be a significant predictor of perceived learning, while disability type was not a significant predictor of perceived learning. After the digital notes were introduced, two posttest surveys were conducted in the 12^th and 16^th week, respectively, to record changes in students' responses for the POUR scales, perceived learning, belongingness, self-efficacy, and their association with low-stakes assessment including homework assignments and quizzes. The results from the responses of 285 students (including 22 SWD) for the first survey found that students generally reported positive scores for the courses' accessibility in all four POUR subscales (scores average > 4.40 out of 5). SWAN and students without disabilities (SWOD) differed in perceivable subscales (p < 0.006), with SWAN finding course material less perceivable. As the course content's difficulty level increased, SWD differed from SWOD significantly regarding perceivable, understandable and robustness subscales at the second survey with lower scores for SWD (FDR p < 0.02). Even though the perceived accessibility scales for the classes decreased in the second survey, the scores for all scales were still relatively high (score average > 4.3 out of 5). We found all four subscales did not change significantly for digital note users while SWOD who didn't use digital notes reported significantly decreased understandable, robust and operable subscales (FDR p < 0.05). The change of perceivable subscale was positively correlated to the change of low stakes assessments in SWD (r = 0.427, p < 0.08). Analysis of the second (n = 255, 18 SWD) and third survey (n = 224, 16 SWD) also found that digital notes helped increase the belongingness of SWAN (p < 0.02) and increase the self-efficacy for SWAN (p < 0.003). In addition, SWAN who used digital notes reported higher perceived learning than those who didn't (p < 0.03). This study provides empirical evidence of the benefits of UDL based digital notes in promoting the belongingness, self-efficacy and perceived learning of SWAN. It also shows the evidence SWD benefited from digital notes in their low stakes assessments. Results further show the close relationship among belongingness, self-efficacy and perceived learning and yet distinctness of these learning outcome metrics. In addition, the perceived accessibility is confirmed to be uniquely useful for understanding the needs of SWD, and thus deserves more attention to help them succeed.