Examining the comparability between paper- and computer-based versions of an integrated writing placement test

The English Placement Test (EPT) is a process-oriented integrated writing placement test for newly-admitted international students at the University of Illinois at Urbana-Champaign. In order to meet student demand, since 2012 the EPT has been administered in both paper-pencil (on-campus) and computer-delivered (online) versions. Both versions feature a two-draft essay writing process and have identical testing procedures except that the online EPT does not have a peer review session, which is built into the on-campus test. This study examined the comparability of the on-campus and online versions, focusing on essay quality and examinee preference among 26 examinees who took both versions within a week, in counterbalanced order. Essay quality was measured in terms of linguistic (complexity, accuracy, fluency) and rhetorical features (integration of sources, progression of ideas, argument effectiveness). No meaningful differences in essay quality were observed between the two versions, although online essays were slightly longer. Post-test questionnaire responses revealed that a majority of test-takers preferred the online version for its convenience. We discussed the advantages and disadvantages of including peer review in writing placement tests, and we concluded by providing recommendations for evaluating comparability as a part of standard quality control practice in local tests.