Theoretical, Statistical, and Substantive Issues in the Assessment of Construct Dimensionality: Accounting for the Item Response Process

Can one accurately infer the dimensionality of constructs such as emotions (i.e., happy-sad), work-family spillover (i.e., positive-negative), or job performance (i.e., organizational citizenship behaviors and counterproductive work behaviors) with commonly used methods? In this article, the authors show how the misapplication of commonly used methods (e.g., factor analysis [FA]) to data originating from an ideal point response process (i.e., self-reported typical behaviors: attitudes, personality, emotions, or interests) can lead to incorrect theoretical and statistical inferences. The authors demonstrate that principal components analysis (PCA) produces an additional spurious dimension despite Likert scaling procedures (i.e., reverse scoring and excluding items with low item-total correlations to improve scale reliability). This incorrectly leads to a conclusion against bipolarity. The authors illustrate the substantive implications for organizational research with emotions data showing that the misapplication of FA could underlie the longstanding debate on the bipolarity of affect. To circumvent this potential problem, the authors propose analytic steps to determine if the recovered constructs are spurious. Additionally, the authors lay out specific issues that need to be considered when evaluating the bipolarity of self-reported typical behavior constructs such as work-family spillover and job performance.