Validating a Holistic Rubric for Scoring Short Answer Reading Questions

Sara T. Cushing; Rurik Tywoniw

doi:10.1007/978-981-15-8952-2_8

Validating a Holistic Rubric for Scoring Short Answer Reading Questions

Sara T. Cushing, Rurik Tywoniw

Linguistics

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

This study presents an investigation of the validity of a practical scoring procedure for short answer (sentence length) reading comprehension questions, using a holistic rubric for sets of responses rather than scoring individual responses, on a university-based English proficiency test. Thirty-three previously scored response sets were rescored by the researchers at the item level, with ratings for completion, fidelity to the source text, and overlap with source text language. Results confirm that holistic section-based scores are predicted by fidelity ratings only, providing evidence to support the scoring inference of the validity argument.

Original language	English (US)
Title of host publication	Another Generation of Fundamental Considerations in Language Assessment
Subtitle of host publication	A Festschrift in Honor of Lyle F. Bachman
Editors	Gary J. Ockey, Brent A. Green
Publisher	Springer
Chapter	8
Pages	113-128
ISBN (Electronic)	9789811589522
ISBN (Print)	9789811589515
DOIs	https://doi.org/10.1007/978-981-15-8952-2_8
State	Published - Nov 24 2020

Online availability

10.1007/978-981-15-8952-2_8

Library availability

Discover UIUC Full Text

Cite this

@inbook{fcb7691afa8a408686042ef08f62dadc,

title = "Validating a Holistic Rubric for Scoring Short Answer Reading Questions",

abstract = "This study presents an investigation of the validity of a practical scoring procedure for short answer (sentence length) reading comprehension questions, using a holistic rubric for sets of responses rather than scoring individual responses, on a university-based English proficiency test. Thirty-three previously scored response sets were rescored by the researchers at the item level, with ratings for completion, fidelity to the source text, and overlap with source text language. Results confirm that holistic section-based scores are predicted by fidelity ratings only, providing evidence to support the scoring inference of the validity argument.",

author = "Cushing, {Sara T.} and Rurik Tywoniw",

year = "2020",

month = nov,

day = "24",

doi = "10.1007/978-981-15-8952-2_8",

language = "English (US)",

isbn = "9789811589515",

pages = "113--128",

editor = "Ockey, {Gary J.} and Green, {Brent A.}",

booktitle = "Another Generation of Fundamental Considerations in Language Assessment",

publisher = "Springer",

address = "Germany",

}

TY - CHAP

T1 - Validating a Holistic Rubric for Scoring Short Answer Reading Questions

AU - Cushing, Sara T.

AU - Tywoniw, Rurik

PY - 2020/11/24

Y1 - 2020/11/24

N2 - This study presents an investigation of the validity of a practical scoring procedure for short answer (sentence length) reading comprehension questions, using a holistic rubric for sets of responses rather than scoring individual responses, on a university-based English proficiency test. Thirty-three previously scored response sets were rescored by the researchers at the item level, with ratings for completion, fidelity to the source text, and overlap with source text language. Results confirm that holistic section-based scores are predicted by fidelity ratings only, providing evidence to support the scoring inference of the validity argument.

AB - This study presents an investigation of the validity of a practical scoring procedure for short answer (sentence length) reading comprehension questions, using a holistic rubric for sets of responses rather than scoring individual responses, on a university-based English proficiency test. Thirty-three previously scored response sets were rescored by the researchers at the item level, with ratings for completion, fidelity to the source text, and overlap with source text language. Results confirm that holistic section-based scores are predicted by fidelity ratings only, providing evidence to support the scoring inference of the validity argument.

U2 - 10.1007/978-981-15-8952-2_8

DO - 10.1007/978-981-15-8952-2_8

M3 - Chapter

SN - 9789811589515

SP - 113

EP - 128

BT - Another Generation of Fundamental Considerations in Language Assessment

A2 - Ockey, Gary J.

A2 - Green, Brent A.

PB - Springer

ER -

Validating a Holistic Rubric for Scoring Short Answer Reading Questions

Abstract

Online availability

Library availability

Fingerprint

Cite this