Adaptive Cache Management for Energy-Efficient GPU Computing

Xuhao Chen, Li Wen Chang, Christopher I. Rodrigues, Jie Lv, Zhiying Wang, Wen Mei Hwu

Research output: Chapter in Book/Report/Conference proceedingConference contribution


With the SIMT execution model, GPUs can hidememory latency through massive multithreading for many applications that have regular memory access patterns. To support applications with irregular memory access patterns, cache hierarchies have been introduced to GPU architectures to capture temporal and spatial locality and mitigate the effect of irregular accesses. However, GPU caches exhibit poor efficiency due to the mismatch of the throughput-oriented execution model and its cache hierarchy design, which limits system performance and energy-efficiency. The massive amount of memory requests generated by GPU scause cache contention and resource congestion. Existing CPUcache management policies that are designed for multicoresystems, can be suboptimal when directly applied to GPUcaches. We propose a specialized cache management policy for GPGPUs. The cache hierarchy is protected from contention by the bypass policy based on reuse distance. Contention and resource congestion are detected at runtime. To avoid oversaturatingon-chip resources, the bypass policy is coordinated with warp throttling to dynamically control the active number of warps. We also propose a simple predictor to dynamically estimate the optimal number of active warps that can take full advantage of the cache space and on-chip resources. Experimental results show that cache efficiency is significantly improved and on-chip resources are better utilized for cache sensitive benchmarks. This results in a harmonic mean IPCimprovement of 74% and 17% (maximum 661% and 44% IPCimprovement), compared to the baseline GPU architecture and optimal static warp throttling, respectively.

Original languageEnglish (US)
Title of host publicationProceedings - 47th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2014
PublisherIEEE Computer Society
Number of pages13
ISBN (Electronic)9781479969982
StatePublished - Jan 15 2015
Event47th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2014 - Cambridge, United Kingdom
Duration: Dec 13 2014Dec 17 2014

Publication series

NameProceedings of the Annual International Symposium on Microarchitecture, MICRO
ISSN (Print)1072-4451


Other47th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2014
Country/TerritoryUnited Kingdom


  • bypass
  • cache management
  • warp throttling

ASJC Scopus subject areas

  • Hardware and Architecture


Dive into the research topics of 'Adaptive Cache Management for Energy-Efficient GPU Computing'. Together they form a unique fingerprint.

Cite this