TY - GEN
T1 - Embedded-check a Code Quality Tool for Automatic Firmware Verification
AU - Ferrão, Rafael Corsi
AU - Montagner, Igor Dos Santos
AU - Silva, Mariana Teixeira
AU - Zilles, Craig
AU - Azevedo, Rodolfo
N1 - Publisher Copyright:
© 2024 ACM.
PY - 2024/7/3
Y1 - 2024/7/3
N2 - Developing embedded microcontroller code is a complex task, especially for undergrad students new to this area. These students often make high-level conceptual mistakes beyond the scope of commercial standards like MISRA-C. These conceptual errors need to be checked manually through code feedback, a process that is time-consuming, error-prone, and does not scale well with an increasing number of students and/or assignments. In this paper, we present an embedded-check an automated tool that can detect common and critical errors students make when learning to code firmware. A set of 13 rules (baremetal and FreeRTOS) was devised based on our experience from several years of teaching Embedded systems. To validate our tool, we compared its results with manual code review of N=99 projects from the last 3 course offerings. We furthered our analysis by running our tool on N=1132 coding lab submissions that did not receive manual feedback and were used as part of classroom activities. We found that the top-3 errors flagged in the projects were already present when students completed the lab activities. We found that (i) our tool also identified all issues discovered during manual code feedback, (ii) our tool detected issues in 86% of student submissions, whereas manual code feedback only flagged 28% of the submissions as problematic, and (iii) 94.3% of students made some code quality error on individual assignments. Within this results, we believe that our tool can have a significant impact when used both as an formative assessment tool to support learning and as a learning analytics tool to improve teaching.
AB - Developing embedded microcontroller code is a complex task, especially for undergrad students new to this area. These students often make high-level conceptual mistakes beyond the scope of commercial standards like MISRA-C. These conceptual errors need to be checked manually through code feedback, a process that is time-consuming, error-prone, and does not scale well with an increasing number of students and/or assignments. In this paper, we present an embedded-check an automated tool that can detect common and critical errors students make when learning to code firmware. A set of 13 rules (baremetal and FreeRTOS) was devised based on our experience from several years of teaching Embedded systems. To validate our tool, we compared its results with manual code review of N=99 projects from the last 3 course offerings. We furthered our analysis by running our tool on N=1132 coding lab submissions that did not receive manual feedback and were used as part of classroom activities. We found that the top-3 errors flagged in the projects were already present when students completed the lab activities. We found that (i) our tool also identified all issues discovered during manual code feedback, (ii) our tool detected issues in 86% of student submissions, whereas manual code feedback only flagged 28% of the submissions as problematic, and (iii) 94.3% of students made some code quality error on individual assignments. Within this results, we believe that our tool can have a significant impact when used both as an formative assessment tool to support learning and as a learning analytics tool to improve teaching.
KW - code quality tools
KW - embedded systems
KW - RTOS
KW - static analysis
UR - http://www.scopus.com/inward/record.url?scp=85198902510&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85198902510&partnerID=8YFLogxK
U2 - 10.1145/3649217.3653577
DO - 10.1145/3649217.3653577
M3 - Conference contribution
AN - SCOPUS:85198902510
T3 - Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE
SP - 66
EP - 72
BT - ITiCSE 2024 - Proceedings of the 2024 Conference Innovation and Technology in Computer Science Education
PB - Association for Computing Machinery
T2 - 29th Conference Innovation and Technology in Computer Science Education, ITiCSE 2024
Y2 - 8 July 2024 through 10 July 2024
ER -