A multiplatform study of i/o behavior on petascale supercomputers

Huong Luu, Marianne Winslett, William Gropp, Robert Ross, Philip Carns, Kevin Harms, Mr Prabhat, Suren Byna, Yushu Yao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We examine the I/O behavior of thousands of supercomputing applications "in the wild," by analyzing the Darshan logs of over a million jobs representing a combined total of six years of I/O behavior across three leading high-performance computing platforms. We mined these logs to analyze the I/O behavior of applications across all their runs on a platform; the evolution of an application's I/O behavior across time, and across platforms; and the I/O behavior of a platform's entire workload. Our analysis techniques can help developers and platform owners improve I/O performance and I/O system utilization, by quickly identifying underperforming applications and offering early intervention to save system resources. We summarize our observations regarding how jobs perform I/O and the throughput they attain in practice.

Original languageEnglish (US)
Title of host publicationHPDC 2015 - Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing
PublisherAssociation for Computing Machinery, Inc
Pages33-44
Number of pages12
ISBN (Electronic)9781450335508
DOIs
StatePublished - Jun 15 2015
Event24th ACM Symposium on High-Performance Parallel and Distributed Computing, HPDC 2015 - Portland, United States
Duration: Jun 15 2015Jun 19 2015

Publication series

NameHPDC 2015 - Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing

Other

Other24th ACM Symposium on High-Performance Parallel and Distributed Computing, HPDC 2015
CountryUnited States
CityPortland
Period6/15/156/19/15

Keywords

  • HPC
  • Input/Output
  • Parallel I/O
  • Performance Analysis

ASJC Scopus subject areas

  • Computer Science Applications
  • Computational Theory and Mathematics
  • Software

Fingerprint Dive into the research topics of 'A multiplatform study of i/o behavior on petascale supercomputers'. Together they form a unique fingerprint.

  • Cite this

    Luu, H., Winslett, M., Gropp, W., Ross, R., Carns, P., Harms, K., Prabhat, M., Byna, S., & Yao, Y. (2015). A multiplatform study of i/o behavior on petascale supercomputers. In HPDC 2015 - Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing (pp. 33-44). (HPDC 2015 - Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing). Association for Computing Machinery, Inc. https://doi.org/10.1145/2749246.2749269