How do missing data bias estimates of within-group agreement? Sensitivity of SD WG, CVWG, rWG(J), r WG(J)* , and ICC to systematic nonresponse

Daniel A. Newman, Hock Peng Sin

Research output: Contribution to journalArticlepeer-review


In multilevel theory testing, estimation of group-level properties (i.e., consensus and diversity) is often complicated by missing data. Researchers are left to draw inferences about group constructs (e.g., organizational climate and climate strength) from the responses of only a subset of group members. This study analyzes the biasing impact of random and non-random missingness patterns on within-group agreement and reliability (standard deviation, coefficient of variation, rWG(J), r*WG(J), AD M, aWG , and intraclass correlation) across a range of response rates, numbers of items, and systematic missing data mechanisms. Results demonstrate biases up to 20% over- or underestimation for common response rates found in organizational research. Correction formulae are presented, which enable assessment of the sensitivity of multilevel results to survey nonresponse.

Original languageEnglish (US)
Pages (from-to)113-147
Number of pages35
JournalOrganizational Research Methods
Issue number1
StatePublished - Jan 2009
Externally publishedYes


  • Aggregation
  • Group agreement
  • Missing data
  • Multilevel analysis

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Strategy and Management
  • Management of Technology and Innovation


Dive into the research topics of 'How do missing data bias estimates of within-group agreement? Sensitivity of SD <sub>WG</sub>, CV<sub>WG</sub>, r<sub>WG(J)</sub>, r <sub>WG(J)</sub>* , and ICC to systematic nonresponse'. Together they form a unique fingerprint.

Cite this