EVALUATING THE EFFECTIVENESS OF A FULL-POPULATION ESTIMATION METHOD

Henry Braun, Jinming Zhang, Sailesh Vezzu

Research output: Contribution to journalArticlepeer-review

Abstract



At present, although the percentages of students with disabilities (SDs) and/or students who are English language learners (ELL) excluded from a NAEP administration are reported, no statistical adjustment is made for these excluded students in the calculation of NAEP results. However, the exclusion rates for both SD and ELL students vary substantially across jurisdictions at a given administration, and, in some cases, have changed substantially over time within a jurisdiction. Consequently, comparisons of performance based on reported NAEP scores may indeed be biased by differential exclusion and identification practices.

Using only NAEP data, this report investigates plausible explanations for the observed heterogeneity among jurisdictions in exclusion rates. It also examines the operating characteristics of a particular class of methods that carry out statistical adjustments to NAEP's reported scores to address the possible bias due to differential exclusion rates. The final results of such adjustments are termed full‐population estimates (FPEs). The conclusions are that there is both a strong likelihood of bias and that neither the current NAEP procedure nor the FPE methodologies constitutes an ideal solution. The former because it assumes that all excluded students could not meaningfully participate in NAEP, and the latter because they implicitly assume that all students could obtain a proper NAEP score.
Original languageEnglish (US)
Pages (from-to)i-58
JournalETS Research Report Series
Volume2008
Issue number1
DOIs
StatePublished - Jun 1 2008
Externally publishedYes

Keywords

  • excluded students
  • Full-population estimates (FPEs)
  • indirect standardization
  • NAEP

Fingerprint Dive into the research topics of 'EVALUATING THE EFFECTIVENESS OF A FULL-POPULATION ESTIMATION METHOD'. Together they form a unique fingerprint.

Cite this