Never Say "always"? Extreme Item Wording Effects on Scalar Invariance and Item Response Curves

When writing items for survey measures, common advice dictates that one should avoid using extreme words like "always." However, the systematic study of extreme wording effects is rare. The current study applies confirmatory factor analysis (CFA) and item response theory (IRT) methods to assess the effects of extreme item wording (i.e., the word "always") on item-level, response option-level, and scale- (or test) level invariance. The authors hypothesized that including the word "always" in an item stem would affect responses such that individuals would be less likely to strongly agree with these items. To test this hypothesis, six items with extreme wording from the Wong and Law Emotional Intelligence Scale (WLEIS) were compared with more moderately worded versions of the same items. Although an effect was found for item wording, the magnitude of nonequivalence was small and is unlikely to have a strong influence on scale-level measurement outcomes. Implications for evaluating survey psychometric properties are discussed.