Never Say "always"? Extreme Item Wording Effects on Scalar Invariance and Item Response Curves

Christopher D. Nye, Daniel A Newman, Dana L. Joseph

Research output: Contribution to journalArticle

Abstract

When writing items for survey measures, common advice dictates that one should avoid using extreme words like "always." However, the systematic study of extreme wording effects is rare. The current study applies confirmatory factor analysis (CFA) and item response theory (IRT) methods to assess the effects of extreme item wording (i.e., the word "always") on item-level, response option-level, and scale- (or test) level invariance. The authors hypothesized that including the word "always" in an item stem would affect responses such that individuals would be less likely to strongly agree with these items. To test this hypothesis, six items with extreme wording from the Wong and Law Emotional Intelligence Scale (WLEIS) were compared with more moderately worded versions of the same items. Although an effect was found for item wording, the magnitude of nonequivalence was small and is unlikely to have a strong influence on scale-level measurement outcomes. Implications for evaluating survey psychometric properties are discussed.

Original languageEnglish (US)
Pages (from-to)806-830
Number of pages25
JournalOrganizational Research Methods
Volume13
Issue number4
DOIs
StatePublished - Sep 16 2010

    Fingerprint

Keywords

  • factor analysis
  • invariance testing
  • item response theory
  • measurement models
  • quantitative research

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Strategy and Management
  • Management of Technology and Innovation

Cite this