Within-Group Agreement (r_WG): Two Theoretical Parameters and their Estimators

When measuring group-level psychological properties (e.g., organizational climate, leadership, team motivation), researchers typically aggregate individual perceptions to represent the group. L. R. James provided the groundbreaking insight that, in order to justify aggregating individual perceptions to represent a group-level property, one must first establish that there exist shared perceptions—or shared psychological meaning—within the group. Here we label and describe two distinct theoretical parameters that can both be used to define within-group agreement: (a) ψ_WG (i.e., a parameter that defines within-group agreement as Individual True-Score Consensus), which arises from the theoretical work of L. R. James and colleagues in the 1970s, and (b) ρ_WG (i.e., a parameter that treats within-group agreement as a Group True-Score Reliability Analog), which forms the theoretical basis for the r_WG(J) index. We extend the work of L. R. James by offering a systematic comparison of different estimators of the two within-group agreement parameters (ψ_WG and ρ_WG). Recommendations are provided for estimating within-group agreement, to continue the legacy of justified measurement of group-level psychological properties.