HAOCl: Harnessing Large-scale Heterogeneous Processors Made Easy

Yao Chen, Xin Long, Jiong He, Yuhang Chen, Hongshi Tan, Zhenxiang Zhang, Marianne Winslett, Deming Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

—The pervasive adoption of Deep Learning (DL) and Graph Processing (GP) makes it a de facto requirement to build large-scale clusters of heterogeneous accelerators including GPUs and FPGAs. The OpenCL programming framework can be used on the individual nodes of such clusters but is not intended for deployment in a distributed manner. Fortunately, the original OpenCL semantics naturally fit into the programming environment of heterogeneous clusters. In this paper, we propose a Äeterogeneity-aware OpenCL-like (HaoCL) programming framework to facilitate the programming of a wide range of scientific applications including DL and GP workloads on large-scale heterogeneous clusters. With HaoCL, existing applications can be directly deployed on heterogeneous clusters without any modifications to the original OpenCL source code and without awareness of the underlying hardware topologies and configurations. Our experiments show that HaoCL imposes a negligible overhead in a distributed environment, and provides near-liner speedups on standard benchmarks when computation or data size exceeds the capacity of a single node. The system design and the evaluations are presented in this demo paper.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE 40th International Conference on Distributed Computing Systems, ICDCS 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1231-1234
Number of pages4
ISBN (Electronic)9781728170022
DOIs
StatePublished - Nov 2020
Event40th IEEE International Conference on Distributed Computing Systems, ICDCS 2020 - Singapore, Singapore
Duration: Nov 29 2020Dec 1 2020

Publication series

NameProceedings - International Conference on Distributed Computing Systems
Volume2020-November

Conference

Conference40th IEEE International Conference on Distributed Computing Systems, ICDCS 2020
CountrySingapore
CitySingapore
Period11/29/2012/1/20

Keywords

  • -heterogeneous cluster
  • Deep learning
  • Distributed computing
  • Machine learning
  • OpenCL

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'HAOCl: Harnessing Large-scale Heterogeneous Processors Made Easy'. Together they form a unique fingerprint.

Cite this