Nek5000 with OpenACC

Jing Gong, Stefano Markidis, Michael Schliephake, Erwin Laure, Dan Henningson, Philipp Schlatter, Adam Peplinski, Alistair Hart, Jens Doleschal, David Henty, Paul Fischer

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Nek5000 is a computational fluid dynamics code based on the spectral element method used for the simulation of incompressible flows. We follow up on an earlier study which ported the simplified version of Nek5000 to a GPU-accelerated system by presenting the hybrid CPU/GPU implementation of the full Nek5000 code using OpenACC. The matrix-matrix multiplication, the Nek5000 gather-scatter operator and a preconditioned Conjugate Gradient solver have implemented using OpenACC for multi-GPU systems.We report an speed-up of 1.3 on single node of a Cray XK6 when using OpenACC directives in Nek5000. On 512 nodes of the Titan supercomputer, the speed-up can be approached to 1.4. A performance analysis of the Nek5000 code using Score-P and Vampir performance monitoring tools shows that overlapping of GPU kernels with host-accelerator memory transfers would considerably increase the performance of the OpenACC version of Nek5000 code.

Original languageEnglish (US)
Title of host publicationSolving Software Challenges for Exascale - International Conference on Exascale Applications and Software, EASC 2014, Revised Selected Papers
EditorsStefano Markidis, Erwin Laure
Number of pages12
ISBN (Electronic)9783319159751
StatePublished - Jan 1 2015
Externally publishedYes
EventInternational Conference on Exascale Applications and Software, EASC 2014 - Stockholm, Sweden
Duration: Apr 2 2014Apr 3 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


OtherInternational Conference on Exascale Applications and Software, EASC 2014


  • GPU programming
  • Nek5000
  • OpenACC
  • Spectral element method

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Nek5000 with OpenACC'. Together they form a unique fingerprint.

Cite this