Abstract

The reproducibility of statistical findings has become a concern not only for statisticians, but for all researchers engaged in empirical discovery. Section 2 of this article identifies key reasons statistical findings may not replicate, including power and sampling issues; misapplication of statistical tests; the instability of findings under reasonable perturbations of data or models; lack of access to methods, data, or equipment; and cultural barriers such as researcher incentives and rewards. Section 3 discusses five proposed remedies for these replication failures: improved prepublication and postpublication validation of findings; the complete disclosure of research steps; assessment of the stability of statistical findings; providing access to digital research objects, in particular data and software; and ensuring these objects are legally reusable.

Original languageEnglish (US)
Pages (from-to)1-19
Number of pages19
JournalAnnual Review of Statistics and Its Application
Volume2
DOIs
StatePublished - Apr 10 2015

Fingerprint

Disclosure
Reproducibility
Statistical test
Incentives
Reward
Replication
Perturbation
Software
Object
Model
Remedies
Statistical tests
Sampling

Keywords

  • Code sharing
  • Data sharing
  • Open code
  • Open data
  • Open licensing
  • Open science
  • Replication
  • Reproducible research
  • Statistical reproducibility

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Reproducing statistical results. / Stodden, Victoria.

In: Annual Review of Statistics and Its Application, Vol. 2, 10.04.2015, p. 1-19.

Research output: Contribution to journalArticle

@article{e96c7599657745e2a0f6f4c2337b3883,
title = "Reproducing statistical results",
abstract = "The reproducibility of statistical findings has become a concern not only for statisticians, but for all researchers engaged in empirical discovery. Section 2 of this article identifies key reasons statistical findings may not replicate, including power and sampling issues; misapplication of statistical tests; the instability of findings under reasonable perturbations of data or models; lack of access to methods, data, or equipment; and cultural barriers such as researcher incentives and rewards. Section 3 discusses five proposed remedies for these replication failures: improved prepublication and postpublication validation of findings; the complete disclosure of research steps; assessment of the stability of statistical findings; providing access to digital research objects, in particular data and software; and ensuring these objects are legally reusable.",
keywords = "Code sharing, Data sharing, Open code, Open data, Open licensing, Open science, Replication, Reproducible research, Statistical reproducibility",
author = "Victoria Stodden",
year = "2015",
month = "4",
day = "10",
doi = "10.1146/annurev-statistics-010814-020127",
language = "English (US)",
volume = "2",
pages = "1--19",
journal = "Annual Review of Statistics and Its Application",
issn = "2326-831X",
publisher = "Annual Reviews Inc.",

}

TY - JOUR

T1 - Reproducing statistical results

AU - Stodden, Victoria

PY - 2015/4/10

Y1 - 2015/4/10

N2 - The reproducibility of statistical findings has become a concern not only for statisticians, but for all researchers engaged in empirical discovery. Section 2 of this article identifies key reasons statistical findings may not replicate, including power and sampling issues; misapplication of statistical tests; the instability of findings under reasonable perturbations of data or models; lack of access to methods, data, or equipment; and cultural barriers such as researcher incentives and rewards. Section 3 discusses five proposed remedies for these replication failures: improved prepublication and postpublication validation of findings; the complete disclosure of research steps; assessment of the stability of statistical findings; providing access to digital research objects, in particular data and software; and ensuring these objects are legally reusable.

AB - The reproducibility of statistical findings has become a concern not only for statisticians, but for all researchers engaged in empirical discovery. Section 2 of this article identifies key reasons statistical findings may not replicate, including power and sampling issues; misapplication of statistical tests; the instability of findings under reasonable perturbations of data or models; lack of access to methods, data, or equipment; and cultural barriers such as researcher incentives and rewards. Section 3 discusses five proposed remedies for these replication failures: improved prepublication and postpublication validation of findings; the complete disclosure of research steps; assessment of the stability of statistical findings; providing access to digital research objects, in particular data and software; and ensuring these objects are legally reusable.

KW - Code sharing

KW - Data sharing

KW - Open code

KW - Open data

KW - Open licensing

KW - Open science

KW - Replication

KW - Reproducible research

KW - Statistical reproducibility

UR - http://www.scopus.com/inward/record.url?scp=84928107878&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84928107878&partnerID=8YFLogxK

U2 - 10.1146/annurev-statistics-010814-020127

DO - 10.1146/annurev-statistics-010814-020127

M3 - Article

AN - SCOPUS:84928107878

VL - 2

SP - 1

EP - 19

JO - Annual Review of Statistics and Its Application

JF - Annual Review of Statistics and Its Application

SN - 2326-831X

ER -