DL: A data layout transformation system for heterogeneous computing

I. Jui Sung, Geng Daniel Liu, Wen Mei W. Hwu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

For many-core architectures like the GPUs, efficient off-chip memory access is crucial to high performance; the applications are often limited by off-chip memory bandwidth. Transforming data layout is an effective way to reshape the access patterns to improve off-chip memory access behavior, but several challenges had limited the use of automated data layout transformation systems on GPUs, namely how to efficiently handle arrays of aggregates, and transparently marshal data between layouts required by different performance sensitive kernels and legacy host code. While GPUs have higher memory bandwidth and are natural candidates for marshaling data between layouts, the relatively constrained GPU memory capacity, compared to that of the CPU, implies that not only the temporal cost of marshaling but also the spatial overhead must be considered for any practical layout transformation systems. This paper presents DL, a practical GPU data layout transformation system that addresses these problems: first, a novel approach to laying out array of aggregate types across GPU and CPU architectures is proposed to further improve memory parallelism and kernel performance beyond what is achieved by human programmers using discrete arrays today. Our proposed new layout can be derived in situ from the traditional Array of Structure, Structure of Arrays, and adjacent Discrete Arrays layouts used by programmers. Second, DL has a run-time library implemented in OpenCL that transparently and efficiently converts, or marshals, data to accommodate application components that have different data layout requirements. We present insights that lead to the design of this highly efficient run-time marshaling library. In particular, the in situ transformation implemented in the library is comparable or faster than optimized traditional out-of-place transformations while avoiding doubling the GPU DRAM usage. Third, we show experimental results that the new layout approach leads to substantial performance improvement at the applications level even when all marshaling cost is taken into account.

Original languageEnglish (US)
Title of host publication2012 Innovative Parallel Computing, InPar 2012
DOIs
StatePublished - Dec 12 2012
Externally publishedYes
Event2012 Innovative Parallel Computing, InPar 2012 - San Jose, CA, United States
Duration: May 13 2012May 14 2012

Publication series

Name2012 Innovative Parallel Computing, InPar 2012

Other

Other2012 Innovative Parallel Computing, InPar 2012
CountryUnited States
CitySan Jose, CA
Period5/13/125/14/12

    Fingerprint

Keywords

  • Data Layout
  • GPU
  • Marshaling

ASJC Scopus subject areas

  • Software

Cite this

Sung, I. J., Liu, G. D., & Hwu, W. M. W. (2012). DL: A data layout transformation system for heterogeneous computing. In 2012 Innovative Parallel Computing, InPar 2012 [6339606] (2012 Innovative Parallel Computing, InPar 2012). https://doi.org/10.1109/InPar.2012.6339606