Establishing analytic score profiles for large-scale L2 writing assessment: The case of the CET-4 writing test

Shaoyan Zou, Xun Yan, Jason Fan

Research output: Contribution to journalArticlepeer-review

Abstract

This study addresses a critical need in large-scale L2 writing assessment by emphasizing the significance of tailoring assessments to specific teaching and learning contexts. Focusing on the CET-4 writing test in China, the research unfolded in two phases. In Phase I, an empirically-developed analytic rating scale designed for the CET-4 writing test was rigorously validated. Twenty-one raters used this scale to rate 30 essays, and Many-Facets Rasch Model (MFRM) analysis was performed on the rating data. The outcomes demonstrate the scale's robustness in effectively differentiating examinees’ writing performance, ensuring consistency among raters, and mitigating rater variation at both individual and group level. Phase II extends the research scope by applying the validated scale to score 142 CET-4 writing scripts. Utilizing Hierarchical and K-Means cluster analyses, this phase unveils three distinct score profiles. These findings are significant for both the CET-4 writing test and other L2 large-scale writing assessment. Theoretically, this study introduces a perspective that aims to enhance our understanding of learners’ performance in large-scale L2 writing assessment. Methodologically, this study presents a framework that integrates the validation of the rating scale with the identification of distinct score clusters, thus aiming to provide a more detailed solution for tailoring assessments to specific learning contexts.

Original languageEnglish (US)
Article number100826
JournalAssessing Writing
Volume60
DOIs
StatePublished - Apr 2024

Keywords

  • Analytic rating scale
  • Cluster analysis
  • Large-scale L2 writing assessment
  • Localization
  • MFRM analysis
  • Score profile

ASJC Scopus subject areas

  • Language and Linguistics
  • Education
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'Establishing analytic score profiles for large-scale L2 writing assessment: The case of the CET-4 writing test'. Together they form a unique fingerprint.

Cite this