OpenACC acceleration of the Nek5000 spectral element code

Stefano Markidis, Jing Gong, Michael Schliephake, Erwin Laure, Alistair Hart, David Henty, Katherine Heisey, Paul Fischer

Research output: Contribution to journalArticlepeer-review


We present a case study of porting NekBone, a skeleton version of the Nek5000 code, to a parallel GPU-accelerated system. Nek5000 is a computational fluid dynamics code based on the spectral element method used for the simulation of incompressible flow. The original NekBone Fortran source code has been used as the base and enhanced by OpenACC directives. The profiling of NekBone provided an assessment of the suitability of the code for GPU systems, and indicated possible kernel optimizations. To port NekBone to GPU systems required little effort and a small number of additional lines of code (approximately one OpenACC directive per 1000 lines of code). The naïve implementation using OpenACC leads to little performance improvement: on a single node, from 16 Gflops obtained with the version without OpenACC, we reached 20 Gflops with the naïve OpenACC implementation. An optimized NekBone version leads to a 43 Gflop performance on a single node. In addition, we ported and optimized NekBone to parallel GPU systems, reaching a parallel efficiency of 79.9% on 1024 GPUs of the Titan XK7 supercomputer at the Oak Ridge National Laboratory.

Original languageEnglish (US)
Pages (from-to)311-319
Number of pages9
JournalInternational Journal of High Performance Computing Applications
Issue number3
StatePublished - Aug 28 2015
Externally publishedYes


  • Nek5000
  • OpenACC
  • optimization of NekBone with OpenACC
  • porting NekBone to GPU

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture


Dive into the research topics of 'OpenACC acceleration of the Nek5000 spectral element code'. Together they form a unique fingerprint.

Cite this