Abstract
We present a hybrid GPU implementation and performance analysis of Nekbone, which represents one of the core kernels of the incompressible Navier–Stokes solver Nek5000. The implementation is based on OpenACC and CUDA Fortran for local parallelization of the compute-intensive matrix–matrix multiplication part, which significantly minimizes the modification of the existing CPU code while extending the simulation capability of the code to GPU architectures. Our discussion includes the GPU results of OpenACC interoperating with CUDA Fortran and the gather–scatter operations with GPUDirect communication. We demonstrate performance of up to 552 Tflops on 16, 384 GPUs of the OLCF Cray XK7 Titan.
Original language | English (US) |
---|---|
Pages (from-to) | 4160-4180 |
Number of pages | 21 |
Journal | Journal of Supercomputing |
Volume | 72 |
Issue number | 11 |
DOIs | |
State | Published - Nov 1 2016 |
Keywords
- CUDA Fortran
- GPUDirect
- Gather–scatter communication
- Nekbone/Nek5000
- OpenACC
- Spectral element discretization
ASJC Scopus subject areas
- Software
- Theoretical Computer Science
- Information Systems
- Hardware and Architecture