Don't forget the I/O when allocating your LLC

Yifan Yuan, Mohammad Alian, Yipeng Wang, Ren Wang, Ilia Kurakin, Charlie Tai, Nam Sung Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In modern server CPUs, last-level cache (LLC) is a critical hardware resource that exerts significant influence on the performance of the workloads, and how to manage LLC is a key to the performance isolation and QoS in the cloud with multi-tenancy. In this paper, we argue that in addition to CPU cores, high-speed I/O is also important for LLC management. This is because of an Intel architectural innovation - Data Direct I/O (DDIO) - that directly injects the inbound I/O traffic to (part of) the LLC instead of the main memory. We summarize two problems caused by DDIO and show that (1) the default DDIO configuration may not always achieve optimal performance, (2) DDIO can decrease the performance of non-I/O workloads that share LLC with it by as high as 32%.We then present, the first LLC management mechanism that treats the I/O as the first-class citizen. Iat monitors and analyzes the performance of the core/LLC/DDIO using CPU's hardware performance counters and adaptively adjusts the number of LLC ways for DDIO or the tenants that demand more LLC capacity. In addition, Iat dynamically chooses the tenants that share its LLC resource with DDIO to minimize the performance interference by both the tenants and the I/O. Our experiments with multiple microbenchmarks and real-world applications demonstrate that with minimal overhead, Iat can effectively and stably reduce the performance degradation caused by DDIO.

Original languageEnglish (US)
Title of host publicationProceedings - 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture, ISCA 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages14
ISBN (Electronic)9781665433334
StatePublished - Jun 2021
Event48th ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2021 - Virtual, Online, Spain
Duration: Jun 14 2021Jun 19 2021

Publication series

NameProceedings - International Symposium on Computer Architecture
ISSN (Print)1063-6897


Conference48th ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2021
CityVirtual, Online


  • Cache partitioning
  • DDIO
  • Performance isolation

ASJC Scopus subject areas

  • Hardware and Architecture


Dive into the research topics of 'Don't forget the I/O when allocating your LLC'. Together they form a unique fingerprint.

Cite this