RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods

Yan Xia, Yanyun Yang

Research output: Contribution to journalArticle

Abstract

In structural equation modeling, application of the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis index (TLI) highly relies on the conventional cutoff values developed under normal-theory maximum likelihood (ML) with continuous data. For ordered categorical data, unweighted least squares (ULS) and diagonally weighted least squares (DWLS) based on polychoric correlation matrices have been recommended in previous studies. Although no clear suggestions exist regarding the application of these fit indices when analyzing ordered categorical variables, practitioners are still tempted to adopt the conventional cutoff rules. The purpose of our research was to answer the question: Given a population polychoric correlation matrix and a hypothesized model, if ML results in a specific RMSEA value (e.g.,.08), what is the RMSEA value when ULS or DWLS is applied? CFI and TLI were investigated in the same fashion. Both simulated and empirical polychoric correlation matrices with various degrees of model misspecification were employed to address the above question. The results showed that DWLS and ULS lead to smaller RMSEA and larger CFI and TLI values than does ML for all manipulated conditions, regardless of whether or not the indices are scaled. Applying the conventional cutoffs to DWLS and ULS, therefore, has a pronounced tendency not to discover model–data misfit. Discussions regarding the use of RMSEA, CFI, and TLI for ordered categorical data are given.

Original languageEnglish (US)
Pages (from-to)409-428
Number of pages20
JournalBehavior Research Methods
Volume51
Issue number1
DOIs
StatePublished - Feb 15 2019
Externally publishedYes

Fingerprint

Least-Squares Analysis
Categorical
Structural Equation Modeling
Approximation
Research
Population

Keywords

  • Diagonally weighted least squares
  • Fit index
  • Maximum likelihood
  • Ordered categorical data
  • Structural equation modeling
  • Unweighted least squares

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Psychology (miscellaneous)
  • Psychology(all)

Cite this

RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data : The story they tell depends on the estimation methods. / Xia, Yan; Yang, Yanyun.

In: Behavior Research Methods, Vol. 51, No. 1, 15.02.2019, p. 409-428.

Research output: Contribution to journalArticle

@article{d66870dff0b549aa98e6ecf8cbe6cfe9,
title = "RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods",
abstract = "In structural equation modeling, application of the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis index (TLI) highly relies on the conventional cutoff values developed under normal-theory maximum likelihood (ML) with continuous data. For ordered categorical data, unweighted least squares (ULS) and diagonally weighted least squares (DWLS) based on polychoric correlation matrices have been recommended in previous studies. Although no clear suggestions exist regarding the application of these fit indices when analyzing ordered categorical variables, practitioners are still tempted to adopt the conventional cutoff rules. The purpose of our research was to answer the question: Given a population polychoric correlation matrix and a hypothesized model, if ML results in a specific RMSEA value (e.g.,.08), what is the RMSEA value when ULS or DWLS is applied? CFI and TLI were investigated in the same fashion. Both simulated and empirical polychoric correlation matrices with various degrees of model misspecification were employed to address the above question. The results showed that DWLS and ULS lead to smaller RMSEA and larger CFI and TLI values than does ML for all manipulated conditions, regardless of whether or not the indices are scaled. Applying the conventional cutoffs to DWLS and ULS, therefore, has a pronounced tendency not to discover model–data misfit. Discussions regarding the use of RMSEA, CFI, and TLI for ordered categorical data are given.",
keywords = "Diagonally weighted least squares, Fit index, Maximum likelihood, Ordered categorical data, Structural equation modeling, Unweighted least squares",
author = "Yan Xia and Yanyun Yang",
year = "2019",
month = "2",
day = "15",
doi = "10.3758/s13428-018-1055-2",
language = "English (US)",
volume = "51",
pages = "409--428",
journal = "Behavior Research Methods",
issn = "1554-351X",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data

T2 - The story they tell depends on the estimation methods

AU - Xia, Yan

AU - Yang, Yanyun

PY - 2019/2/15

Y1 - 2019/2/15

N2 - In structural equation modeling, application of the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis index (TLI) highly relies on the conventional cutoff values developed under normal-theory maximum likelihood (ML) with continuous data. For ordered categorical data, unweighted least squares (ULS) and diagonally weighted least squares (DWLS) based on polychoric correlation matrices have been recommended in previous studies. Although no clear suggestions exist regarding the application of these fit indices when analyzing ordered categorical variables, practitioners are still tempted to adopt the conventional cutoff rules. The purpose of our research was to answer the question: Given a population polychoric correlation matrix and a hypothesized model, if ML results in a specific RMSEA value (e.g.,.08), what is the RMSEA value when ULS or DWLS is applied? CFI and TLI were investigated in the same fashion. Both simulated and empirical polychoric correlation matrices with various degrees of model misspecification were employed to address the above question. The results showed that DWLS and ULS lead to smaller RMSEA and larger CFI and TLI values than does ML for all manipulated conditions, regardless of whether or not the indices are scaled. Applying the conventional cutoffs to DWLS and ULS, therefore, has a pronounced tendency not to discover model–data misfit. Discussions regarding the use of RMSEA, CFI, and TLI for ordered categorical data are given.

AB - In structural equation modeling, application of the root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis index (TLI) highly relies on the conventional cutoff values developed under normal-theory maximum likelihood (ML) with continuous data. For ordered categorical data, unweighted least squares (ULS) and diagonally weighted least squares (DWLS) based on polychoric correlation matrices have been recommended in previous studies. Although no clear suggestions exist regarding the application of these fit indices when analyzing ordered categorical variables, practitioners are still tempted to adopt the conventional cutoff rules. The purpose of our research was to answer the question: Given a population polychoric correlation matrix and a hypothesized model, if ML results in a specific RMSEA value (e.g.,.08), what is the RMSEA value when ULS or DWLS is applied? CFI and TLI were investigated in the same fashion. Both simulated and empirical polychoric correlation matrices with various degrees of model misspecification were employed to address the above question. The results showed that DWLS and ULS lead to smaller RMSEA and larger CFI and TLI values than does ML for all manipulated conditions, regardless of whether or not the indices are scaled. Applying the conventional cutoffs to DWLS and ULS, therefore, has a pronounced tendency not to discover model–data misfit. Discussions regarding the use of RMSEA, CFI, and TLI for ordered categorical data are given.

KW - Diagonally weighted least squares

KW - Fit index

KW - Maximum likelihood

KW - Ordered categorical data

KW - Structural equation modeling

KW - Unweighted least squares

UR - http://www.scopus.com/inward/record.url?scp=85047997723&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047997723&partnerID=8YFLogxK

U2 - 10.3758/s13428-018-1055-2

DO - 10.3758/s13428-018-1055-2

M3 - Article

C2 - 29869222

AN - SCOPUS:85047997723

VL - 51

SP - 409

EP - 428

JO - Behavior Research Methods

JF - Behavior Research Methods

SN - 1554-351X

IS - 1

ER -