Skip to main navigation Skip to search Skip to main content

ReDefining Code Comprehension: Function Naming as a Mechanism for Evaluating Code Comprehension

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

''Explain in Plain English'' (EiPE) questions are widely used to assess code comprehension skills but are challenging to grade automatically. Recent approaches like Code Generation Based Grading (CGBG) leverage large language models (LLMs) to generate code from student explanations and validate its equivalence to the original code using unit tests. However, this approach does not differentiate between high-level, purpose-focused responses and low-level, implementation-focused ones, limiting its effectiveness in assessing comprehension level. We propose a modified approach where students generate function names, emphasizing the function's purpose over implementation details. We evaluate this method in an introductory programming course and analyze it using Item Response Theory (IRT) to assess the difficulty and discrimination of function naming exercises as exam items and to compare their alignment with traditional EiPE grading standards. We also publish this work as an open source Python package for auto-grading EiPE questions, providing a scalable solution for adoption.

Original languageEnglish (US)
Title of host publicationITiCSE 2025 - Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education
PublisherAssociation for Computing Machinery
Pages44-50
Number of pages7
ISBN (Electronic)9798400715679
DOIs
StatePublished - Jun 27 2025
Event30th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE 2025 - Nijmegen, Netherlands
Duration: Jun 27 2025Jul 2 2025

Publication series

NameAnnual Conference on Innovation and Technology in Computer Science Education, ITiCSE
Volume1
ISSN (Print)1942-647X

Conference

Conference30th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE 2025
Country/TerritoryNetherlands
CityNijmegen
Period6/27/257/2/25

Keywords

  • code comprehension
  • eipe
  • eipl
  • explain in plain english
  • explain in plain language
  • function naming
  • gpt-4o
  • large language model
  • llm
  • solo taxonomy

ASJC Scopus subject areas

  • Management of Technology and Innovation
  • Education

Fingerprint

Dive into the research topics of 'ReDefining Code Comprehension: Function Naming as a Mechanism for Evaluating Code Comprehension'. Together they form a unique fingerprint.

Cite this