Porting ordinary applications to Blue Gene/Q supercomputers

Ketan Maheshwari, Justin M. Wozniak, Timothy G. Armstrong, Daniel S. Katz, T. Andrew Binkowski, Xiaoliang Zhong, Olle Heinonen, Dmitry Karpeyev, Michael Wilde

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Efficiently porting ordinary applications to Blue Gene/Q supercomputers is a significant challenge. Codes are often originally developed without considering advanced architectures and related tool chains. Science needs frequently lead users to want to run large numbers of relatively small jobs (often called many-task computing, an ensemble, or a workflow), which can conflict with supercomputer configurations. In this paper, we discuss techniques developed to execute ordinary applications over leadership class supercomputers. We use the high-performance Swift parallel scripting framework and build two workflow execution techniques - sub-jobs and main-wrap. The sub-jobs technique, built on top of the IBM Blue Gene/Q resource manager Cobalt's sub-block jobs, lets users submit multiple, independent, repeated smaller jobs within a single larger resource block. The main-wrap technique is a scheme that enables C/C++ programs to be defined as functions that are wrapped by a high-performance Swift wrapper and that are invoked as a Swift script. We discuss the needs, benefits, technicalities, and current limitations of these techniques. We further discuss the real-world science enabled by these techniques and the results obtained.

Original languageEnglish (US)
Title of host publicationProceedings - 11th IEEE International Conference on eScience, eScience 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages420-428
Number of pages9
ISBN (Electronic)9781467393256
DOIs
StatePublished - Oct 22 2015
Externally publishedYes
Event11th IEEE International Conference on eScience, eScience 2015 - Munich, Germany
Duration: Aug 31 2015Sep 4 2015

Publication series

NameProceedings - 11th IEEE International Conference on eScience, eScience 2015

Other

Other11th IEEE International Conference on eScience, eScience 2015
CountryGermany
CityMunich
Period8/31/159/4/15

Keywords

  • BG/Q
  • Supercomputers
  • Swift

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Management Science and Operations Research

Fingerprint Dive into the research topics of 'Porting ordinary applications to Blue Gene/Q supercomputers'. Together they form a unique fingerprint.

Cite this