An investigation of the relationship between argument structure and essay quality in assessed writing

This study investigates the characteristics of argumentation in essay performances on a second language (L2) writing test. Inspired by Toulmin's ([1958] 2003) model of argumentation and research on evaluating the soundness of arguments (Means & Voss, 1996; Schwarz, Neuman, Gil, & Ilya, 2003), we examined argument structure and quality in 150 argumentative essays of different levels on an integrated English-as-a-second-language (ESL) writing placement test at a US university. The essays were coded using Toulmin's elements, scored on a Likert scale in terms of argument relevance and acceptability, and compared via multivariate analysis of variance. Results show that both argument structure and quality are associated with differences in writing proficiency. However, in terms of argument structure, the presence of individual Toulmin's elements alone does not predict proficiency level. Instead, it is the structural complexity, i.e., co-occurrence of multiple structural elements, that reflects the proficiency differences. The findings of this study help disentangle the nature of argumentation across proficiency levels, offering both insights over L2 writing development and validity evidence for L2 writing assessments. Additionally, this study has methodological implications for L2 writing assessment research, suggesting the added value of incorporating fine-grained measures of argumentation in scale development and validation.