AI Benchmarking for Science: Efforts from the MLCommons Science Working Group

Jeyan Thiyagalingam, Gregor von Laszewski, Junqi Yin, Murali Emani, Juri Papay, Gregg Barrett, Piotr Luszczek, Aristeidis Tsaris, Christine Kirkpatrick, Feiyi Wang, Tom Gibbs, Venkatram Vishwanath, Mallikarjun Shankar, Geoffrey Fox, Tony Hey

Research output: Chapter in Book/Report/Conference proceedingConference contribution


With machine learning (ML) becoming a transformative tool for science, the scientific community needs a clear catalogue of ML techniques, and their relative benefits on various scientific problems, if they were to make significant advances in science using AI. Although this comes under the purview of benchmarking, conventional benchmarking initiatives are focused on performance, and as such, science, often becomes a secondary criteria. In this paper, we describe a community effort from a working group, namely, MLCommons Science Working Group, in developing science-specific AI benchmarking for the international scientific community. Since the inception of the working group in 2020, the group has worked very collaboratively with a number of national laboratories, academic institutions and industries, across the world, and has developed four science-specific AI benchmarks. We will describe the overall process, the resulting benchmarks along with some initial results. We foresee that this initiative is likely to be very transformative for the AI for Science, and for performance-focused communities.

Original languageEnglish (US)
Title of host publicationHigh Performance Computing. ISC High Performance 2022 International Workshops - Revised Selected Papers
EditorsHartwig Anzt, Amanda Bienz, Piotr Luszczek, Marc Baboulin
Number of pages18
ISBN (Print)9783031232190
StatePublished - 2022
Externally publishedYes
Event37th International Conference on High Performance Computing , ISC High Performance 2022 - Hamburg, Germany
Duration: May 29 2022Jun 2 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13387 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference37th International Conference on High Performance Computing , ISC High Performance 2022


  • AI for Science
  • Benchmarks
  • Machine learning
  • Science

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'AI Benchmarking for Science: Efforts from the MLCommons Science Working Group'. Together they form a unique fingerprint.

Cite this