Parallel I/O prefetching using MPI file caching and I/O signatures

Surendra Byna, Yong Chen, Xian He Sun, Rajee Thakur, William Gropp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Parallel I/O prefetching is considered to be effective in improving I/O performance. However, the effectiveness depends on determining patterns among future I/O accesses swiftly and fetching data in time, which is difficult to achieve in general. In this study, we propose an I/O signature-based prefetching strategy. The idea is to use a predetermined I/O signature of an application to guide prefetching. To put this idea to work, we first derived a classification of patterns and introduced a simple and effective signature notation to represent patterns. We then developed a toolkit to trace and generate I/O signatures automatically. Finally, we designed and implemented a thread-based client-side collective prefetching cache layer for MPI-IO library to support prefetching. A prefetching thread reads I/O signatures of an application and adjusts them by observing I/O accesses at runtime. Experimental results show that the proposed prefetching method improves I/O performance significantly for applications with complex patterns.

Original languageEnglish (US)
Title of host publication2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008
DOIs
StatePublished - Dec 1 2008
Event2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008 - Austin, TX, United States
Duration: Nov 15 2008Nov 21 2008

Publication series

Name2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008

Other

Other2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008
CountryUnited States
CityAustin, TX
Period11/15/0811/21/08

Keywords

  • I/O signatures
  • MPI-IO
  • Parallel I/O
  • Prefetching

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Cite this

Byna, S., Chen, Y., Sun, X. H., Thakur, R., & Gropp, W. (2008). Parallel I/O prefetching using MPI file caching and I/O signatures. In 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008 [5213604] (2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008). https://doi.org/10.1109/SC.2008.5213604