Optimization and architecture effects on GPU computing workload performance

John A. Stratton, Nasser Anssari, Christopher Rodrigues, I. Jui Sung, Nady Obeid, Liwen Chang, Geng Daniel Liu, Wen Mei Hwu

Research output: Chapter in Book/Report/Conference proceedingConference contribution


It is unquestionable that successive hardware generations have significantly improved GPU computing workload performance over the last several years. Moore's law and DRAM scaling have respectively increased single-chip peak instruction throughput by 3X and off-chip bandwidth by 2.2X from NVIDIA's GeForce 8800 GTX in November 2006 to its GeForce GTX 580 in November 2010. However, raw capability numbers typically underestimate the improvements in real application performance over the same time period, due to significant architectural feature improvements. To demonstrate the effects of architecture features and optimizations over time, we conducted experiments on a set of benchmarks from diverse application domains for multiple GPU architecture generations to understand how much performance has truly been improving for those workloads. First, we demonstrate that certain architectural features make a huge difference in the performance of unoptimized code, such as the inclusion of a general cache which can improve performance by 2-4x in some situations. Second, we describe what optimization patterns have been most essential and widely applicable for improving performance for GPU computing workloads across all architecture generations. Some important optimization patterns included data layout transformation, converting scatter accesses to gather accesses, GPU workload regularization, and granularity coarsening, each of which improved performance on some benchmark by over 20%, sometimes by a factor of more than 5x. While hardware improvements to baseline unoptimized code can reduce the speedup magnitude, these patterns remain important for even the most recent GPUs. Finally, we identify which added architectural features created significant new optimization opportunities, such as increased register file capacity or reduced bandwidth penalties for misaligned accesses, which increase performance by 2x or more in the optimized versions of relevant benchmarks.

Original languageEnglish (US)
Title of host publication2012 Innovative Parallel Computing, InPar 2012
StatePublished - 2012
Event2012 Innovative Parallel Computing, InPar 2012 - San Jose, CA, United States
Duration: May 13 2012May 14 2012

Publication series

Name2012 Innovative Parallel Computing, InPar 2012


Other2012 Innovative Parallel Computing, InPar 2012
Country/TerritoryUnited States
CitySan Jose, CA


  • CUDA
  • GPU
  • Optimization

ASJC Scopus subject areas

  • Software


Dive into the research topics of 'Optimization and architecture effects on GPU computing workload performance'. Together they form a unique fingerprint.

Cite this