GAMER-2: A GPU-accelerated adaptive mesh refinement code - Accuracy, performance, and scalability

Hsi Yu Schive, John A. ZuHone, Nathan J. Goldbaum, Matthew J. Turk, Massimo Gaspari, Chin Yu Cheng

Research output: Contribution to journalArticlepeer-review


We present GAMER-2, a GPU-accelerated adaptive mesh refinement (AMR) code for astrophysics. It provides a rich set of features, including adaptive time-stepping, several hydrodynamic schemes, magnetohydrodynamics, self-gravity, particles, star formation, chemistry, and radiative processes with GRACKLE, data analysis with YT, and memory pool for efficient object allocation. GAMER-2 is fully bitwise reproducible. For the performance optimization, it adopts hybrid OpenMP/MPI/GPU parallelization and utilizes overlapping CPU computation, GPU computation, and CPU-GPU communication. Load balancing is achieved using a Hilbert space-filling curve on a level-by-level basis without the need to duplicate the entire AMR hierarchy on each MPI process. To provide convincing demonstrations of the accuracy and performance of GAMER-2, we directly compare with ENZO on isolated disc galaxy simulations and with FLASH on galaxy cluster merger simulations. We show that the physical results obtained by different codes are in very good agreement, and GAMER-2 outperforms ENZO and FLASH by nearly one and two orders of magnitude, respectively, on the Blue Waters supercomputers using 1-256 nodes. More importantly, GAMER-2 exhibits similar or even better parallel scalability compared to the other two codes. We also demonstrate good weak and strong scaling using up to 4096 GPUs and 65 536 CPU cores, and achieve a uniform resolution as high as 10 2403 cells. Furthermore, GAMER-2 can be adopted as an AMR + GPUs framework and has been extensively used for the wave dark matter simulations. GAMER-2 is open source (available at and new contributions are welcome.

Original languageEnglish (US)
Pages (from-to)4815-4840
Number of pages26
JournalMonthly Notices of the Royal Astronomical Society
Issue number4
StatePublished - Dec 1 2018


  • Methods: numerical

ASJC Scopus subject areas

  • Astronomy and Astrophysics
  • Space and Planetary Science


Dive into the research topics of 'GAMER-2: A GPU-accelerated adaptive mesh refinement code - Accuracy, performance, and scalability'. Together they form a unique fingerprint.

Cite this