Validity and topic generality of a writing performance test

Hee Kyung Lee, Carolyn Jane Anderson

Research output: Contribution to journalArticle

Abstract

The goal of the current study was to examine the validity and topic generality of a writing performance test designed to place international students into appropriate ESL courses at a large mid-western university. Because for each test administration the test randomly rotates three academic topics integrated with listening and reading sources, it is necessary to investigate the extent to which the three topics are compatible in terms of difficulty and generality across a diverse group of examinees. ESL Placement Test (EPT) scores from more than 1,000 examinees were modeled using multinomial logistic regression. Possible explanatory variables were identified as the assigned writing topic, students' majors, and their scores on the Test of English as a Foreign Language (TOEFL). Results indicate that after controlling for general English proficiency as measured by the TOEFL, students' majors were not related to their writing performance; however, the different topics did affect performance. In light of test validity, the demonstrated topic effect argues against the comparability of the three topics. Nevertheless, the absence of effect from the interaction of essay topic and writers' majors supports the generality of each topic for examinees from a wide range of disciplinary areas. Study limitations and future research suggestions are discussed.

Original languageEnglish (US)
Pages (from-to)307-330
Number of pages24
JournalLanguage Testing
Volume24
Issue number3
DOIs
StatePublished - Jul 1 2007

Fingerprint

performance
foreign language
student
Generality
Performance Test
logistics
writer
regression
university
interaction
English as a Foreign Language
Group
Interaction
Test Scores
English Proficiency
Test Validity
Placement Test
International Students
Logistic Regression
Writer

ASJC Scopus subject areas

  • Language and Linguistics
  • Social Sciences (miscellaneous)
  • Linguistics and Language

Cite this

Validity and topic generality of a writing performance test. / Lee, Hee Kyung; Anderson, Carolyn Jane.

In: Language Testing, Vol. 24, No. 3, 01.07.2007, p. 307-330.

Research output: Contribution to journalArticle

@article{9b40f85463084822b1f98d83f9e8a431,
title = "Validity and topic generality of a writing performance test",
abstract = "The goal of the current study was to examine the validity and topic generality of a writing performance test designed to place international students into appropriate ESL courses at a large mid-western university. Because for each test administration the test randomly rotates three academic topics integrated with listening and reading sources, it is necessary to investigate the extent to which the three topics are compatible in terms of difficulty and generality across a diverse group of examinees. ESL Placement Test (EPT) scores from more than 1,000 examinees were modeled using multinomial logistic regression. Possible explanatory variables were identified as the assigned writing topic, students' majors, and their scores on the Test of English as a Foreign Language (TOEFL). Results indicate that after controlling for general English proficiency as measured by the TOEFL, students' majors were not related to their writing performance; however, the different topics did affect performance. In light of test validity, the demonstrated topic effect argues against the comparability of the three topics. Nevertheless, the absence of effect from the interaction of essay topic and writers' majors supports the generality of each topic for examinees from a wide range of disciplinary areas. Study limitations and future research suggestions are discussed.",
author = "Lee, {Hee Kyung} and Anderson, {Carolyn Jane}",
year = "2007",
month = "7",
day = "1",
doi = "10.1177/0265532207077200",
language = "English (US)",
volume = "24",
pages = "307--330",
journal = "Language Testing",
issn = "0265-5322",
publisher = "SAGE Publications Ltd",
number = "3",

}

TY - JOUR

T1 - Validity and topic generality of a writing performance test

AU - Lee, Hee Kyung

AU - Anderson, Carolyn Jane

PY - 2007/7/1

Y1 - 2007/7/1

N2 - The goal of the current study was to examine the validity and topic generality of a writing performance test designed to place international students into appropriate ESL courses at a large mid-western university. Because for each test administration the test randomly rotates three academic topics integrated with listening and reading sources, it is necessary to investigate the extent to which the three topics are compatible in terms of difficulty and generality across a diverse group of examinees. ESL Placement Test (EPT) scores from more than 1,000 examinees were modeled using multinomial logistic regression. Possible explanatory variables were identified as the assigned writing topic, students' majors, and their scores on the Test of English as a Foreign Language (TOEFL). Results indicate that after controlling for general English proficiency as measured by the TOEFL, students' majors were not related to their writing performance; however, the different topics did affect performance. In light of test validity, the demonstrated topic effect argues against the comparability of the three topics. Nevertheless, the absence of effect from the interaction of essay topic and writers' majors supports the generality of each topic for examinees from a wide range of disciplinary areas. Study limitations and future research suggestions are discussed.

AB - The goal of the current study was to examine the validity and topic generality of a writing performance test designed to place international students into appropriate ESL courses at a large mid-western university. Because for each test administration the test randomly rotates three academic topics integrated with listening and reading sources, it is necessary to investigate the extent to which the three topics are compatible in terms of difficulty and generality across a diverse group of examinees. ESL Placement Test (EPT) scores from more than 1,000 examinees were modeled using multinomial logistic regression. Possible explanatory variables were identified as the assigned writing topic, students' majors, and their scores on the Test of English as a Foreign Language (TOEFL). Results indicate that after controlling for general English proficiency as measured by the TOEFL, students' majors were not related to their writing performance; however, the different topics did affect performance. In light of test validity, the demonstrated topic effect argues against the comparability of the three topics. Nevertheless, the absence of effect from the interaction of essay topic and writers' majors supports the generality of each topic for examinees from a wide range of disciplinary areas. Study limitations and future research suggestions are discussed.

UR - http://www.scopus.com/inward/record.url?scp=34447275914&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34447275914&partnerID=8YFLogxK

U2 - 10.1177/0265532207077200

DO - 10.1177/0265532207077200

M3 - Article

AN - SCOPUS:34447275914

VL - 24

SP - 307

EP - 330

JO - Language Testing

JF - Language Testing

SN - 0265-5322

IS - 3

ER -