Reproducibility and replicability of rodent phenotyping in preclinical studies

Neri Kafkafi, Joseph Agassi, Elissa J. Chesler, John C. Crabbe, Wim E. Crusio, David Eilam, Robert Gerlai, Ilan Golani, Alex Gomez-Marin, Ruth Heller, Fuad Iraqi, Iman Jaljuli, Natasha A. Karp, Hugh Morgan, George Nicholson, Donald W. Pfaff, S. Helene Richter, Philip B. Stark, Oliver Stiedl, Victoria StoddenLisa M. Tarantino, Valter Tucci, William Valdar, Robert W. Williams, Hanno Würbel, Yoav Benjamini

Research output: Contribution to journalReview article

Abstract

The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

Original languageEnglish (US)
Pages (from-to)218-232
Number of pages15
JournalNeuroscience and Biobehavioral Reviews
Volume87
DOIs
StatePublished - Apr 2018

Fingerprint

Information Dissemination
Rodentia
Genotype
Research Personnel
Health
Metadata

Keywords

  • Data sharing
  • False discoveries
  • GxE interaction
  • Heterogenization
  • Replicability
  • Reproducibility
  • Validity

ASJC Scopus subject areas

  • Neuropsychology and Physiological Psychology
  • Cognitive Neuroscience
  • Behavioral Neuroscience

Cite this

Kafkafi, N., Agassi, J., Chesler, E. J., Crabbe, J. C., Crusio, W. E., Eilam, D., ... Benjamini, Y. (2018). Reproducibility and replicability of rodent phenotyping in preclinical studies. Neuroscience and Biobehavioral Reviews, 87, 218-232. https://doi.org/10.1016/j.neubiorev.2018.01.003

Reproducibility and replicability of rodent phenotyping in preclinical studies. / Kafkafi, Neri; Agassi, Joseph; Chesler, Elissa J.; Crabbe, John C.; Crusio, Wim E.; Eilam, David; Gerlai, Robert; Golani, Ilan; Gomez-Marin, Alex; Heller, Ruth; Iraqi, Fuad; Jaljuli, Iman; Karp, Natasha A.; Morgan, Hugh; Nicholson, George; Pfaff, Donald W.; Richter, S. Helene; Stark, Philip B.; Stiedl, Oliver; Stodden, Victoria; Tarantino, Lisa M.; Tucci, Valter; Valdar, William; Williams, Robert W.; Würbel, Hanno; Benjamini, Yoav.

In: Neuroscience and Biobehavioral Reviews, Vol. 87, 04.2018, p. 218-232.

Research output: Contribution to journalReview article

Kafkafi, N, Agassi, J, Chesler, EJ, Crabbe, JC, Crusio, WE, Eilam, D, Gerlai, R, Golani, I, Gomez-Marin, A, Heller, R, Iraqi, F, Jaljuli, I, Karp, NA, Morgan, H, Nicholson, G, Pfaff, DW, Richter, SH, Stark, PB, Stiedl, O, Stodden, V, Tarantino, LM, Tucci, V, Valdar, W, Williams, RW, Würbel, H & Benjamini, Y 2018, 'Reproducibility and replicability of rodent phenotyping in preclinical studies', Neuroscience and Biobehavioral Reviews, vol. 87, pp. 218-232. https://doi.org/10.1016/j.neubiorev.2018.01.003
Kafkafi, Neri ; Agassi, Joseph ; Chesler, Elissa J. ; Crabbe, John C. ; Crusio, Wim E. ; Eilam, David ; Gerlai, Robert ; Golani, Ilan ; Gomez-Marin, Alex ; Heller, Ruth ; Iraqi, Fuad ; Jaljuli, Iman ; Karp, Natasha A. ; Morgan, Hugh ; Nicholson, George ; Pfaff, Donald W. ; Richter, S. Helene ; Stark, Philip B. ; Stiedl, Oliver ; Stodden, Victoria ; Tarantino, Lisa M. ; Tucci, Valter ; Valdar, William ; Williams, Robert W. ; Würbel, Hanno ; Benjamini, Yoav. / Reproducibility and replicability of rodent phenotyping in preclinical studies. In: Neuroscience and Biobehavioral Reviews. 2018 ; Vol. 87. pp. 218-232.
@article{a8cd97ee2da04d1ab69b436610795d83,
title = "Reproducibility and replicability of rodent phenotyping in preclinical studies",
abstract = "The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.",
keywords = "Data sharing, False discoveries, GxE interaction, Heterogenization, Replicability, Reproducibility, Validity",
author = "Neri Kafkafi and Joseph Agassi and Chesler, {Elissa J.} and Crabbe, {John C.} and Crusio, {Wim E.} and David Eilam and Robert Gerlai and Ilan Golani and Alex Gomez-Marin and Ruth Heller and Fuad Iraqi and Iman Jaljuli and Karp, {Natasha A.} and Hugh Morgan and George Nicholson and Pfaff, {Donald W.} and Richter, {S. Helene} and Stark, {Philip B.} and Oliver Stiedl and Victoria Stodden and Tarantino, {Lisa M.} and Valter Tucci and William Valdar and Williams, {Robert W.} and Hanno W{\"u}rbel and Yoav Benjamini",
year = "2018",
month = "4",
doi = "10.1016/j.neubiorev.2018.01.003",
language = "English (US)",
volume = "87",
pages = "218--232",
journal = "Neuroscience and Biobehavioral Reviews",
issn = "0149-7634",
publisher = "Elsevier Limited",

}

TY - JOUR

T1 - Reproducibility and replicability of rodent phenotyping in preclinical studies

AU - Kafkafi, Neri

AU - Agassi, Joseph

AU - Chesler, Elissa J.

AU - Crabbe, John C.

AU - Crusio, Wim E.

AU - Eilam, David

AU - Gerlai, Robert

AU - Golani, Ilan

AU - Gomez-Marin, Alex

AU - Heller, Ruth

AU - Iraqi, Fuad

AU - Jaljuli, Iman

AU - Karp, Natasha A.

AU - Morgan, Hugh

AU - Nicholson, George

AU - Pfaff, Donald W.

AU - Richter, S. Helene

AU - Stark, Philip B.

AU - Stiedl, Oliver

AU - Stodden, Victoria

AU - Tarantino, Lisa M.

AU - Tucci, Valter

AU - Valdar, William

AU - Williams, Robert W.

AU - Würbel, Hanno

AU - Benjamini, Yoav

PY - 2018/4

Y1 - 2018/4

N2 - The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

AB - The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

KW - Data sharing

KW - False discoveries

KW - GxE interaction

KW - Heterogenization

KW - Replicability

KW - Reproducibility

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=85042385750&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042385750&partnerID=8YFLogxK

U2 - 10.1016/j.neubiorev.2018.01.003

DO - 10.1016/j.neubiorev.2018.01.003

M3 - Review article

C2 - 29357292

AN - SCOPUS:85042385750

VL - 87

SP - 218

EP - 232

JO - Neuroscience and Biobehavioral Reviews

JF - Neuroscience and Biobehavioral Reviews

SN - 0149-7634

ER -