Pattern-driven parallel I/O tuning

Babak Behzad, Surendra Byna, Prabhat, Marc Snir

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The contemporary parallel I/O software stack is complex due to a large number of configurations for tuning I/O performance. Without a proper configuration, I/O becomes a performance bottleneck. As high performance computing (HPC) is moving towards exascale, poor I/O performance has a significant impact on the runtime of large-scale simulations producing massive amounts of data. In this paper, we focus on developing a framework for tuning parallel I/O configurations automatically. This auto-tuning framework first traces high-level I/O accesses and analyzes data write patterns. Based on these patterns and historically avail- Able tuning parameters for similar patterns, the framework selects best performing configurations at runtime. If previous history for a pattern is unavailable, the framework initiates model-based training to acquire efficient set of tuning parameters. Our framework includes a runtime system to apply the selected configurations using dynamic linking, without the need for changing application source code. In this paper, we describe this framework and evaluate it using multiple I/O kernels extracted from real applications and demonstrate substantial I/O performance improvement.

Original languageEnglish (US)
Title of host publicationProceedings of PDSW 2015
Subtitle of host publication10th Parallel Data Storage Workshop - Held in conjunction with SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherAssociation for Computing Machinery, Inc
Pages43-48
Number of pages6
ISBN (Electronic)9781450340083
DOIs
StatePublished - Nov 15 2015
Event10th Parallel Data Storage Workshop, PDSW 2015 - Austin, United States
Duration: Nov 16 2015 → …

Publication series

NameProceedings of PDSW 2015: 10th Parallel Data Storage Workshop - Held in conjunction with SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis

Other

Other10th Parallel Data Storage Workshop, PDSW 2015
CountryUnited States
CityAustin
Period11/16/15 → …

ASJC Scopus subject areas

  • Information Systems

Fingerprint Dive into the research topics of 'Pattern-driven parallel I/O tuning'. Together they form a unique fingerprint.

  • Cite this

    Behzad, B., Byna, S., Prabhat, & Snir, M. (2015). Pattern-driven parallel I/O tuning. In Proceedings of PDSW 2015: 10th Parallel Data Storage Workshop - Held in conjunction with SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 43-48). (Proceedings of PDSW 2015: 10th Parallel Data Storage Workshop - Held in conjunction with SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis). Association for Computing Machinery, Inc. https://doi.org/10.1145/2834976.2834977