A case study of communication optimizations on 3D mesh interconnects

Abhinav Bhatelé, Eric Bohm, Laxmikant V. Kalé

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Optimal network performance is critical to efficient parallel scaling for communication-bound applications on large machines. With wormhole routing, no-load latencies do not increase significantly with number of hops traveled. Yet, we, and others have recently shown that in presence of contention, message latencies can grow substantially large. Hence task mapping strategies should take the topology of the machine into account on large machines. In this paper, we present topology aware mapping as a technique to optimize communication on 3-dimensional mesh interconnects and hence improve performance. Our methodology is facilitated by the idea of object-based decomposition used in Charm++ which separates the processes of decomposition from mapping of computation to processors and allows a more flexible mapping based on communication patterns between objects. Exploiting this and the topology of the allocated job partition, we present mapping strategies for a production code, OpenAtom to improve overall performance and scaling. OpenAtom presents complex communication scenarios of interaction involving multiple groups of objects and makes the mapping task a challenge. Results are presented for OpenAtom on up to 16,384 processors of Blue Gene/L, 8,192 processors of Blue Gene/P and 2,048 processors of Cray XT3.

Original languageEnglish (US)
Title of host publicationEuro-Par 2009 Parallel Processing - 15th International Euro-Par Conference, Proceedings
Pages1015-1028
Number of pages14
DOIs
StatePublished - Nov 9 2009
EventEuro-Par 2009 Parallel Processing - 15th International Euro-Par Conference, Proceedings - Delft, Netherlands
Duration: Aug 25 2009Aug 28 2009

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5704 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuro-Par 2009 Parallel Processing - 15th International Euro-Par Conference, Proceedings
CountryNetherlands
CityDelft
Period8/25/098/28/09

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'A case study of communication optimizations on 3D mesh interconnects'. Together they form a unique fingerprint.

  • Cite this

    Bhatelé, A., Bohm, E., & Kalé, L. V. (2009). A case study of communication optimizations on 3D mesh interconnects. In Euro-Par 2009 Parallel Processing - 15th International Euro-Par Conference, Proceedings (pp. 1015-1028). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5704 LNCS). https://doi.org/10.1007/978-3-642-03869-3_94