A multi-disciplinary perspective on emergent and future innovations in peer review

Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Katz, Bastian Greshake Tzovaras, Josmel Pacheco-MendozaNazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O'Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb

Research output: Contribution to journalReview article

Abstract

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

Original languageEnglish (US)
Article number1151
JournalF1000Research
Volume6
DOIs
StatePublished - Jan 1 2017

Fingerprint

Peer Review
Innovation
Communication
Research
Motivation
Research Peer Review
Communication systems
Certification
Quality Control
Quality control
Technology

Keywords

  • Incentives
  • Open Peer Review
  • Open Science
  • Quality Control
  • Scholarly Publishing
  • Social Media
  • Web 2.0

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Immunology and Microbiology(all)
  • Pharmacology, Toxicology and Pharmaceutics(all)

Cite this

Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., ... Colomb, J. (2017). A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Research, 6, [1151]. https://doi.org/10.12688/f1000research.12037.3

A multi-disciplinary perspective on emergent and future innovations in peer review. / Tennant, Jonathan P.; Dugan, Jonathan M.; Graziotin, Daniel; Jacques, Damien C.; Waldner, François; Mietchen, Daniel; Elkhatib, Yehia; B. Collister, Lauren; Pikas, Christina K.; Crick, Tom; Masuzzo, Paola; Caravaggi, Anthony; Berg, Devin R.; Niemeyer, Kyle E.; Ross-Hellauer, Tony; Mannheimer, Sara; Rigling, Lillian; Katz, Daniel S.; Greshake Tzovaras, Bastian; Pacheco-Mendoza, Josmel; Fatima, Nazeefa; Poblet, Marta; Isaakidis, Marios; Irawan, Dasapta Erwin; Renaut, Sébastien; Madan, Christopher R.; Matthias, Lisa; Nørgaard Kjær, Jesper; O'Donnell, Daniel Paul; Neylon, Cameron; Kearns, Sarah; Selvaraju, Manojkumar; Colomb, Julien.

In: F1000Research, Vol. 6, 1151, 01.01.2017.

Research output: Contribution to journalReview article

Tennant, JP, Dugan, JM, Graziotin, D, Jacques, DC, Waldner, F, Mietchen, D, Elkhatib, Y, B. Collister, L, Pikas, CK, Crick, T, Masuzzo, P, Caravaggi, A, Berg, DR, Niemeyer, KE, Ross-Hellauer, T, Mannheimer, S, Rigling, L, Katz, DS, Greshake Tzovaras, B, Pacheco-Mendoza, J, Fatima, N, Poblet, M, Isaakidis, M, Irawan, DE, Renaut, S, Madan, CR, Matthias, L, Nørgaard Kjær, J, O'Donnell, DP, Neylon, C, Kearns, S, Selvaraju, M & Colomb, J 2017, 'A multi-disciplinary perspective on emergent and future innovations in peer review', F1000Research, vol. 6, 1151. https://doi.org/10.12688/f1000research.12037.3
Tennant JP, Dugan JM, Graziotin D, Jacques DC, Waldner F, Mietchen D et al. A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Research. 2017 Jan 1;6. 1151. https://doi.org/10.12688/f1000research.12037.3
Tennant, Jonathan P. ; Dugan, Jonathan M. ; Graziotin, Daniel ; Jacques, Damien C. ; Waldner, François ; Mietchen, Daniel ; Elkhatib, Yehia ; B. Collister, Lauren ; Pikas, Christina K. ; Crick, Tom ; Masuzzo, Paola ; Caravaggi, Anthony ; Berg, Devin R. ; Niemeyer, Kyle E. ; Ross-Hellauer, Tony ; Mannheimer, Sara ; Rigling, Lillian ; Katz, Daniel S. ; Greshake Tzovaras, Bastian ; Pacheco-Mendoza, Josmel ; Fatima, Nazeefa ; Poblet, Marta ; Isaakidis, Marios ; Irawan, Dasapta Erwin ; Renaut, Sébastien ; Madan, Christopher R. ; Matthias, Lisa ; Nørgaard Kjær, Jesper ; O'Donnell, Daniel Paul ; Neylon, Cameron ; Kearns, Sarah ; Selvaraju, Manojkumar ; Colomb, Julien. / A multi-disciplinary perspective on emergent and future innovations in peer review. In: F1000Research. 2017 ; Vol. 6.
@article{1f84564e6bf24ec48301d3c72f3d4d46,
title = "A multi-disciplinary perspective on emergent and future innovations in peer review",
abstract = "Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.",
keywords = "Incentives, Open Peer Review, Open Science, Quality Control, Scholarly Publishing, Social Media, Web 2.0",
author = "Tennant, {Jonathan P.} and Dugan, {Jonathan M.} and Daniel Graziotin and Jacques, {Damien C.} and Fran{\cc}ois Waldner and Daniel Mietchen and Yehia Elkhatib and {B. Collister}, Lauren and Pikas, {Christina K.} and Tom Crick and Paola Masuzzo and Anthony Caravaggi and Berg, {Devin R.} and Niemeyer, {Kyle E.} and Tony Ross-Hellauer and Sara Mannheimer and Lillian Rigling and Katz, {Daniel S.} and {Greshake Tzovaras}, Bastian and Josmel Pacheco-Mendoza and Nazeefa Fatima and Marta Poblet and Marios Isaakidis and Irawan, {Dasapta Erwin} and S{\'e}bastien Renaut and Madan, {Christopher R.} and Lisa Matthias and {N{\o}rgaard Kj{\ae}r}, Jesper and O'Donnell, {Daniel Paul} and Cameron Neylon and Sarah Kearns and Manojkumar Selvaraju and Julien Colomb",
year = "2017",
month = "1",
day = "1",
doi = "10.12688/f1000research.12037.3",
language = "English (US)",
volume = "6",
journal = "F1000Research",
issn = "2046-1402",
publisher = "F1000 Research Ltd.",

}

TY - JOUR

T1 - A multi-disciplinary perspective on emergent and future innovations in peer review

AU - Tennant, Jonathan P.

AU - Dugan, Jonathan M.

AU - Graziotin, Daniel

AU - Jacques, Damien C.

AU - Waldner, François

AU - Mietchen, Daniel

AU - Elkhatib, Yehia

AU - B. Collister, Lauren

AU - Pikas, Christina K.

AU - Crick, Tom

AU - Masuzzo, Paola

AU - Caravaggi, Anthony

AU - Berg, Devin R.

AU - Niemeyer, Kyle E.

AU - Ross-Hellauer, Tony

AU - Mannheimer, Sara

AU - Rigling, Lillian

AU - Katz, Daniel S.

AU - Greshake Tzovaras, Bastian

AU - Pacheco-Mendoza, Josmel

AU - Fatima, Nazeefa

AU - Poblet, Marta

AU - Isaakidis, Marios

AU - Irawan, Dasapta Erwin

AU - Renaut, Sébastien

AU - Madan, Christopher R.

AU - Matthias, Lisa

AU - Nørgaard Kjær, Jesper

AU - O'Donnell, Daniel Paul

AU - Neylon, Cameron

AU - Kearns, Sarah

AU - Selvaraju, Manojkumar

AU - Colomb, Julien

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

AB - Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

KW - Incentives

KW - Open Peer Review

KW - Open Science

KW - Quality Control

KW - Scholarly Publishing

KW - Social Media

KW - Web 2.0

UR - http://www.scopus.com/inward/record.url?scp=85028984570&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028984570&partnerID=8YFLogxK

U2 - 10.12688/f1000research.12037.3

DO - 10.12688/f1000research.12037.3

M3 - Review article

C2 - 29188015

AN - SCOPUS:85028984570

VL - 6

JO - F1000Research

JF - F1000Research

SN - 2046-1402

M1 - 1151

ER -