Reliability of scores on the summative performance assessments

Yanyun Yang, Albert Oosterhof, Yan Xia

Research output: Contribution to journalArticlepeer-review


The authors address the reliability of scores obtained on the summative performance assessments during the pilot year of our research. Contrary to classical test theory, we discussed the advantages of using generalizability theory for estimating reliability of scores for summative performance assessments. Generalizability theory was used as the framework because of the flexibility this approach provides for examining sources of inconsistency within a complex assessment. Two major sources of inconsistency on scores considered in this study were raters and agencies (teachers' rating vs. researchers' rating). Overall, results showed that the inconsistency in scores attributable to raters and agencies was relatively small. Suggestions regarding improvement of consistency in the subsequent years of our research were provided.

Original languageEnglish (US)
Pages (from-to)465-479
Number of pages15
JournalJournal of Educational Research
Issue number6
StatePublished - Nov 2 2015
Externally publishedYes


  • classical test theory
  • generalizability theory
  • reliability
  • summative performance assessment

ASJC Scopus subject areas

  • Education


Dive into the research topics of 'Reliability of scores on the summative performance assessments'. Together they form a unique fingerprint.

Cite this