Zorua: A holistic approach to resource virtualization in GPUs

Nandita Vijaykumar, Kevin Hsieh, Gennady Pekhimenw, Samira Khan, Ashish Shrestha, Saugata Ghose, Adwait Jog, Phillip B. Gibbons, Onur Mutlu

Research output: Chapter in Book/Report/Conference proceedingConference contribution


This paper introduces a new resource virtualization framework, Zorua, that decouples the programmer-specified resource usage of a GPU application from the actual allocation in the on-chip hardware resources. Zorua enables this decoupling by virtualizing each resource transparently to the programmer. The virtualization provided by Zorua builds on two key concepts-dynamic allocation of the on-chip resources and their oversubscription using a swap space in memory. Zorua provides a holistic GPU resource virtualization strategy, designed to (i) adaptively control the extent of oversubscription, and (ii) coordinate the dynamic management of multiple on-chip resources (i.e., registers, scratchpad memory, and thread slots), to maximize the effectiveness of virtualization. Zorua employs a hardware-software code-sign, comprising the compiler, a runtime system and hardware-based virtualization support. The runtime system leverages information from the compiler regarding resource requirements of each program phase to (i) dynamically allocate/deallocate the different resources in the physically available on-chip resources or their swap space, and (ii) manage the tradeoffbetween higher thread-level parallelism due to virtualization versus the latency and capacity overheads of swap space usage. We demonstrate that by providing the illusion of more resources than physically available via controlled and coordinated virtualization, Zorua offers several important benefits: (i) Programming Ease. Zorua eases the burden on the programmer to provide code that is tuned to efficiently utilize the physically available on-chip resources. (ii) Portability. Zorua alleviates the necessity of re-Tuning an application's resource usage when porting the application across GPU generations. (iii) Performance. By dynamically allocating resources and carefully oversubscribing them when necessary, Zorua improves or retains the performance of applications that are already highly tuned to best utilize the hardware resources. The holistic virtualization provided by Zorua can also enable other uses, including fine-grained resource sharing among multiple kernels and low-latency preemption of GPU programs.

Original languageEnglish (US)
Title of host publicationMICRO 2016 - 49th Annual IEEE/ACM International Symposium on Microarchitecture
PublisherIEEE Computer Society
ISBN (Electronic)9781509035083
StatePublished - Dec 14 2016
Externally publishedYes
Event49th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2016 - Taipei, Taiwan, Province of China
Duration: Oct 15 2016Oct 19 2016

Publication series

NameProceedings of the Annual International Symposium on Microarchitecture, MICRO
ISSN (Print)1072-4451


Other49th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2016
Country/TerritoryTaiwan, Province of China

ASJC Scopus subject areas

  • Hardware and Architecture


Dive into the research topics of 'Zorua: A holistic approach to resource virtualization in GPUs'. Together they form a unique fingerprint.

Cite this