A case for using MPI's derived datatypes to improve I/O performance

Rajeev Thakur, William Gropp, Ewing Lusk

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

MPI-IO, the I/O part of the MPI-2 standard, is a promising new interface for parallel I/O. A key feature of MPI-IO is that it allows users to access several noncontiguous pieces of data from a file with a single I/O function call by defining file views with derived datatypes. We explain how critical this feature is for high performance, why users must create and use derived datatypes whenever possible, and how it enables implementations to perform optimizations. In particular, we describe two optimizations our MPI-IO implementation, ROMIO, performs: data sieving and collective I/O. We demonstrate the performance and portability of the approach with performance results on five different parallel machines: HP Exemplar, IBM SP, Intel Paragon, NEC SX-4, and SGI Origin2000.

Original languageEnglish (US)
Title of host publicationSC 1998 - Proceedings of the ACM/IEEE Conference on Supercomputing
PublisherAssociation for Computing Machinery
ISBN (Electronic)081868707X
DOIs
StatePublished - 1998
Event1998 ACM/IEEE Conference on Supercomputing, SC 1998 - Orlando, United States
Duration: Nov 7 1998Nov 13 1998

Publication series

NameProceedings of the International Conference on Supercomputing
Volume1998-November

Conference

Conference1998 ACM/IEEE Conference on Supercomputing, SC 1998
Country/TerritoryUnited States
CityOrlando
Period11/7/9811/13/98

Keywords

  • MPI-IO
  • Parallel I/O

ASJC Scopus subject areas

  • Computer Science(all)

Cite this