Persistent confusion in nutrition and obesity research about the validity of classic nonparametric tests in the presence of heteroscedasticity: Evidence of the problem and valid alternatives

Cynthia M. Kroeger, Keisuke Ejima, Bridget A. Hannon, Tanya M. Halliday, Bryan McComb, Margarita Teran-Garcia, John A. Dawson, David B. King, Andrew W. Brown, David B. Allison

Research output: Contribution to journalReview articlepeer-review

Abstract

The use of classic nonparametric tests (cNPTs), such as the Kruskal-Wallis and Mann-Whitney U tests, in the presence of unequal variance for between-group comparisons of means and medians may lead to marked increases in the rate of falsely rejecting null hypotheses and decreases in statistical power. Yet, this practice remains prevalent in the scientific literature, including nutrition and obesity literature. Some nutrition and obesity studies use a cNPT in the presence of unequal variance (i.e., heteroscedasticity), sometimes because of the mistaken rationale that the test corrects for heteroscedasticity. Herein, we discuss misconceptions of using cNPTs in the presence of heteroscedasticity. We then discuss assumptions, purposes, and limitations of 3 common tests used to test for mean differences between multiple groups, including 2 parametric tests: Fisher's ANOVA and Welch's ANOVA; and 1 cNPT: The Kruskal-Wallis test. To document the impact of heteroscedasticity on the validity of these tests under conditions similar to those used in nutrition and obesity research, we conducted simple simulations and assessed type I error rates (i.e., false positives, defined as incorrectly rejecting the null hypothesis). We demonstrate that type I error rates for Fisher's ANOVA, which does not account for heteroscedasticity, and Kruskal-Wallis, which tests for differences in distributions rather than means, deviated from the expected significance level. Greater deviation from the expected type I error rate was observed as the heterogeneity increased, especially in the presence of an imbalanced sample size. We provide brief tutorial guidance for authors, editors, and reviewers to identify appropriate statistical tests when test assumptions are violated, with a particular focus on cNPTs.

Original languageEnglish (US)
Pages (from-to)517-524
Number of pages8
JournalAmerican Journal of Clinical Nutrition
Volume113
Issue number3
DOIs
StatePublished - Mar 1 2021

Keywords

  • association
  • causation
  • heteroscedasticity
  • nonparametric tests
  • nutrition
  • obesity
  • research rigor
  • statistical methods

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Nutrition and Dietetics

Fingerprint Dive into the research topics of 'Persistent confusion in nutrition and obesity research about the validity of classic nonparametric tests in the presence of heteroscedasticity: Evidence of the problem and valid alternatives'. Together they form a unique fingerprint.

Cite this