Optimizing I/O performance of HPC applications with autotuning

Babak Behzad, Surendra Byna, Prabhat, Marc Snir

Research output: Contribution to journalArticlepeer-review


Parallel Input output is an essential component of modern high-performance computing (HPC). Obtaining good I/O performance for a broad range of applications on diverse HPC platforms is a major challenge, in part, because of complex inter dependencies between I/O middleware and hardware. The parallel file system and I/O middleware layers all offer optimization parameters that can, in theory, result in better I/O performance. Unfortunately, the right combination of parameters is highly dependent on the application, HPC platform, problem size, and concurrency. Scientific application developers do not have the time or expertise to take on the substantial burden of identifying good parameters for each problem configuration. They resort to using system defaults, a choice that frequently results in poor I/O performance. We expect this problem to be compounded on exascale-class machines, which will likely have a deeper software stack with hierarchically arranged hardware resources. We present as a solution to this problem an autotuning system for optimizing I/O performance, I/O performance modeling, I/O tuning, and I/O patterns. We demonstrate the value of this framework across several HPC platforms and applications at scale.

Original languageEnglish (US)
Article number15
JournalACM Transactions on Parallel Computing
Issue number4
StatePublished - Mar 2019


  • Autotuning
  • HPC
  • I/O
  • Parallel file systems
  • Performance optimization
  • Storage

ASJC Scopus subject areas

  • Software
  • Modeling and Simulation
  • Hardware and Architecture
  • Computer Science Applications
  • Computational Theory and Mathematics


Dive into the research topics of 'Optimizing I/O performance of HPC applications with autotuning'. Together they form a unique fingerprint.

Cite this