Work in Progress: Reviewing engineering education scholarship in a mentored reviewer program: Trends based on geographic region

Lisa Benson, Gary Lichtenstein, Evan Ko, Rebecca Bates, Kelsey Watts, Karin Jensen

Research output: Chapter in Book/Report/Conference proceedingConference contribution


As a field, engineering education research (EER) draws from multiple disciplines and geographic regions. This presents challenges in the process of peer reviewing EER manuscripts, as peer reviewers typically do not receive formal training in critiquing research publications. We investigated how EER scholars develop an understanding of how to conduct peer reviews of manuscripts related to engineering education. We draw from theories of cognition and schema development to address the following question: What aspects of EER manuscripts are reviewers paying attention to, and how do these aspects differ for reviewers from different geographic regions?The context for this study was a mentored peer reviewer program, which pairs less experienced reviewers (mentees) with experienced reviewers (mentors) to review a set of manuscripts. Participants (n=27: nine mentors and 18 mentees) from six countries and varied levels of expertise were organized into triads (one mentor and two mentees). Each triad collaboratively evaluated each manuscript, crafted feedback to authors and formulated recommendations to journal editors. Data collected from participants included Structured Peer Reviews (SPR) that prompted participants to comment on the main strengths and weaknesses of a short EER manuscript and formulate a recommendation to the editor. All participants completed SPRs for one manuscript prior to the program (Pre-SPR) and a second manuscript at the end of the program (Post-SPR). SPRs were coded and categorized into six themes: context, methods, results, discussion, mechanics, and EER relevance. Themes were categorized into codes related to positive or negative assessments.The preliminary analysis presented in this paper focuses on the extent to which reviewers converged in the breadth of what they commented on in the pre- and post-SPR forms. Results were compared based on where participants earned their doctorate. Differences between participants from different geographic regions emerged in the Pre-SPR data but were minimal in the post-SPR. This research provides a foundation for considering peer review as a form of professional development as findings suggest that training affects reviewers' understanding of EER scholarship.

Original languageEnglish (US)
Title of host publication2022 IEEE Frontiers in Education Conference, FIE 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665462440
StatePublished - 2022
Externally publishedYes
Event2022 IEEE Frontiers in Education Conference, FIE 2022 - Uppsala, Sweden
Duration: Oct 8 2022Oct 11 2022

Publication series

NameProceedings - Frontiers in Education Conference, FIE
ISSN (Print)1539-4565


Conference2022 IEEE Frontiers in Education Conference, FIE 2022


  • academic publishing
  • engineering education research
  • mental models
  • peer review

ASJC Scopus subject areas

  • Software
  • Education
  • Computer Science Applications


Dive into the research topics of 'Work in Progress: Reviewing engineering education scholarship in a mentored reviewer program: Trends based on geographic region'. Together they form a unique fingerprint.

Cite this