How Big Are My Effects? Examining the Magnitude of Effect Sizes in Studies of Measurement Equivalence

Christopher D. Nye, Jacob Bradburn, Jeffrey Olenick, Christopher Bialko, Fritz Drasgow

Research output: Contribution to journalArticlepeer-review

Abstract

Recently, an effect size measure, known as dMACS, was developed for confirmatory factor analytic (CFA) studies of measurement equivalence. Although this index has several advantages over traditional methods of identifying nonequivalence, the scale and interpretation of this effect size are still unclear. As a result, the interpretation of the effect size is left to the subjective judgment of the researcher. To remedy this issue for other effect sizes, some have proposed guidelines for evaluating the magnitude of an effect based on the distribution of effect sizes in the literature. The goal of the current research was to develop similar guidelines for effect sizes of measurement nonequivalence and build on this work by also examining the practical importance of nonequivalence. Based on a review of past research, we conducted two simulation studies to generate distributions of effects sizes. Assuming the ideal scenario of invariant referent items, the results of these simulations were then used to develop empirical guidelines for interpreting nonequivalence and its effects on observed outcomes.

Original languageEnglish (US)
Pages (from-to)678-709
Number of pages32
JournalOrganizational Research Methods
Volume22
Issue number3
DOIs
StatePublished - Jul 1 2019

Keywords

  • factor analysis
  • invariance testing
  • measurement models
  • quantitative research
  • structural equation modeling
  • survey research

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Strategy and Management
  • Management of Technology and Innovation

Fingerprint Dive into the research topics of 'How Big Are My Effects? Examining the Magnitude of Effect Sizes in Studies of Measurement Equivalence'. Together they form a unique fingerprint.

Cite this