High performance MPI-2 one-sided communication over InfiniBand

Weihang Jiang, Jiuxing Liu, Hyun Wook Jin, Dhabaleswar K. Panda, William D Gropp, Rajeev Thakur

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many existing MPI-2 one-sided communication implementations are built on top of MPI send/receive operations. Although this approach can achieve good portability, it suffers from high communication overhead and dependency on remote process for communication progress. To address these problems, we propose a high performance MPI-2 one-sided communication design over the InfiniBand Architecture. In our design, MPI-2 one-sided communication operations such as MPI_Put, MPI_Get and MPI Accumulate are directly mapped to InfiniBand Remote Direct Memory Access (RDMA) operations. Our design has been implemented based on MPICH2 over InfiniBand. We present detailed design issues for this approach and perform a set of micro-benchmarks to characterize different aspects of its performance. Our performance evaluation shows that compared with the design based on MPI send/receive, our design can improve throughput up to 77%, and reduce lantency and synchronization overhead up to 19% and 13%, respectively. Under certain process skew, the bad impact can be significantly reduced by new design, from 41% to nearly 0%. It also can achieve better overlap of communication and computation.

Original languageEnglish (US)
Title of host publication2004 IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2004
Pages531-538
Number of pages8
StatePublished - Sep 29 2004
Externally publishedYes
Event2004 IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2004 - Chicago, IL, United States
Duration: Apr 19 2004Apr 22 2004

Publication series

Name2004 IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2004

Other

Other2004 IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2004
CountryUnited States
CityChicago, IL
Period4/19/044/22/04

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'High performance MPI-2 one-sided communication over InfiniBand'. Together they form a unique fingerprint.

Cite this