Using analysis of covariance (ANCOVA) with fallible covariates

Steven Andrew Culpepper, Herman Aguinis

Research output: Contribution to journalArticlepeer-review

Abstract

Analysis of covariance (ANCOVA) is used widely in psychological research implementing nonexperimental designs. However, when covariates are fallible (i.e., measured with error), which is the norm, researchers must choose from among 3 inadequate courses of action: (a) know that the assumption that covariates are perfectly reliable is violated but use ANCOVA anyway (and, most likely, report misleading results); (b) attempt to employ 1 of several measurement error models with the understanding that no research has examined their relative performance and with the added practical difficulty that several of these models are not available in commonly used statistical software; or (c) not use ANCOVA at all. First, we discuss analytic evidence to explain why using ANCOVA with fallible covariates produces bias and a systematic inflation of Type I error rates that may lead to the incorrect conclusion that treatment effects exist. Second, to provide a solution for this problem, we conduct 2 Monte Carlo studies to compare 4 existing approaches for adjusting treatment effects in the presence of covariate measurement error: errors-in-variables (EIV; Warren, White, & Fuller, 1974), Lord's (1960) method, Raaijmakers and Pieters's (1987) method (R&P), and structural equation modeling methods proposed by Sörbom (1978) and Hayduk (1996). Results show that EIV models are superior in terms of parameter accuracy, statistical power, and keeping Type I error close to the nominal value. Finally, we offer a program written in R that performs all needed computations for implementing EIV models so that ANCOVA can be used to obtain accurate results even when covariates are measured with error.

Original languageEnglish (US)
Pages (from-to)166-178
Number of pages13
JournalPsychological Methods
Volume16
Issue number2
DOIs
StatePublished - Jun 2011
Externally publishedYes

Keywords

  • Analysis of covariance
  • Measurement error
  • Research design
  • Structural equation modeling

ASJC Scopus subject areas

  • Psychology (miscellaneous)
  • History and Philosophy of Science

Fingerprint Dive into the research topics of 'Using analysis of covariance (ANCOVA) with fallible covariates'. Together they form a unique fingerprint.

Cite this