Container solutions for HPC systems: A case study of using shifter on blue waters

Maxim Belkin, Hon Wai Leong, Roland Haas, Eliu A. Huerta, Mark Neubauer, Galen Wesley Arnold, David Lesny

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Software container solutions have revolutionized application development approaches by enabling lightweight platform abstractions within the so-called “containers.” Several solutions are being actively developed in attempts to bring the benefits of containers to high-performance computing systems with their stringent security demands on the one hand and fundamental resource sharing requirements on the other. In this paper, we discuss the benefits and short-comings of such solutions when deployed on real HPC systems and applied to production scientific applications. We highlight use cases that are either enabled by or significantly benefit from such solutions. We discuss the efforts by HPC system administrators and support staff to support users of these type of workloads on HPC systems not initially designed with these workloads in mind focusing on NCSA's Blue Waters system.

Original languageEnglish (US)
Title of host publicationPractice and Experience in Advanced Research Computing 2018
Subtitle of host publicationSeamless Creativity, PEARC 2018
PublisherAssociation for Computing Machinery
ISBN (Print)9781450364461
DOIs
StatePublished - Jul 22 2018
Event2018 Practice and Experience in Advanced Research Computing Conference: Seamless Creativity, PEARC 2018 - Pittsburgh, United States
Duration: Jul 22 2017Jul 26 2017

Publication series

NameACM International Conference Proceeding Series

Other

Other2018 Practice and Experience in Advanced Research Computing Conference: Seamless Creativity, PEARC 2018
CountryUnited States
CityPittsburgh
Period7/22/177/26/17

Keywords

  • Data Science
  • Petascale
  • Reproducibility

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Container solutions for HPC systems: A case study of using shifter on blue waters'. Together they form a unique fingerprint.

  • Cite this

    Belkin, M., Leong, H. W., Haas, R., Huerta, E. A., Neubauer, M., Arnold, G. W., & Lesny, D. (2018). Container solutions for HPC systems: A case study of using shifter on blue waters. In Practice and Experience in Advanced Research Computing 2018: Seamless Creativity, PEARC 2018 [a43] (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3219104.3219145