Utilizing programming traces to explore and model the dimensions of novices' code-writing skill

Yingbin Zhang, Luc Paquette, Juan D. Pinto, Aysa Xuemo Fan

Research output: Contribution to journalArticlepeer-review


Studies have found that most novice programmers have low proficiency in writing code. However, it is unclear what subskills compose code writing and which subskills novice programmers struggle with. This study utilizes programming traces to identify latent subskills that constitute code writing so that teachers can offer specific instruction on the weak subskills. Data were collected from an undergraduate course teaching introductory computer science in Java. Six hundred and fourteen students made submissions to homework programming questions in a web-based learning system. Based on the submission traces, we computed 11 features related to correctness and time students spent on their submissions. We conducted an exploratory factor analysis on two-thirds of students selected randomly and identified four factors. The first factor, code style proficiency, was mainly related to code style errors. The second, syntactic proficiency, concerned compiler errors. The third is semantic proficiency, which concerns runtime and logic errors. The fourth, syntactic debugging proficiency, concerned the success rate and time required for fixing compiler and code style errors. A confirmatory factor analysis conducted on the remaining one-third of the data supported the four-factor structure. The factor model showed measurement invariance between the data set where the model was developed and two new datasets, one from the same sample but collected at a different time point and another from a different sample and context (onsite course vs. online course). The factors were related to prior programming abilities, programming language familiarity, and future exam performance. These associations provided validity evidence for the factor model.

Original languageEnglish (US)
Pages (from-to)1041-1058
Number of pages18
JournalComputer Applications in Engineering Education
Issue number4
StatePublished - Jul 2023


  • CS1
  • code writing
  • factor analysis
  • novice programmer
  • programming trace analysis

ASJC Scopus subject areas

  • General Computer Science
  • Education
  • General Engineering


Dive into the research topics of 'Utilizing programming traces to explore and model the dimensions of novices' code-writing skill'. Together they form a unique fingerprint.

Cite this