Parallelizing MPI using tasks for hybrid programming models

Surabhi Jain, Gengbin Zheng, Maria Garzaran, James H. Cownie, Taru Doodi, Terry L. Wilmarth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many-core architectures such as the Intel® Xeon PhiTM provide dozens of cores and hundreds of hardware threads. For these machines, a basic MPI implementation is inefficient, as it does not take advantage of the shared data across the ranks on the same node. A hybrid-programming model called MPI+X, where MPI is used between the nodes and a threading model X (Pthreads or OpenMP∗among others) can better utilize the compute and memory resources. In this paper, we investigate how the MPI library can run in parallel without competing with the threads of X that are running the application, and in particular without oversubscribing the available hardware resources. We assess whether MPI can use tasks that can be executed by idle threads in X. To prototype such a system, we have chosen X to be OpenMP. Our experimental results show that parallelizing MPI calls using OpenMP tasks can provide significant speedups.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE 32nd International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1303-1312
Number of pages10
ISBN (Print)9781538655559
DOIs
StatePublished - Aug 3 2018
Externally publishedYes
Event32nd IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018 - Vancouver, Canada
Duration: May 21 2018May 25 2018

Publication series

NameProceedings - 2018 IEEE 32nd International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018

Other

Other32nd IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2018
CountryCanada
CityVancouver
Period5/21/185/25/18

Keywords

  • Hybrid programming
  • MPI
  • OpenMP
  • OpenMP task
  • Thread oversubscription

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems and Management

Fingerprint Dive into the research topics of 'Parallelizing MPI using tasks for hybrid programming models'. Together they form a unique fingerprint.

Cite this