TY - GEN
T1 - Don't forget the I/O when allocating your LLC
AU - Yuan, Yifan
AU - Alian, Mohammad
AU - Wang, Yipeng
AU - Wang, Ren
AU - Kurakin, Ilia
AU - Tai, Charlie
AU - Kim, Nam Sung
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/6
Y1 - 2021/6
N2 - In modern server CPUs, last-level cache (LLC) is a critical hardware resource that exerts significant influence on the performance of the workloads, and how to manage LLC is a key to the performance isolation and QoS in the cloud with multi-tenancy. In this paper, we argue that in addition to CPU cores, high-speed I/O is also important for LLC management. This is because of an Intel architectural innovation - Data Direct I/O (DDIO) - that directly injects the inbound I/O traffic to (part of) the LLC instead of the main memory. We summarize two problems caused by DDIO and show that (1) the default DDIO configuration may not always achieve optimal performance, (2) DDIO can decrease the performance of non-I/O workloads that share LLC with it by as high as 32%.We then present, the first LLC management mechanism that treats the I/O as the first-class citizen. Iat monitors and analyzes the performance of the core/LLC/DDIO using CPU's hardware performance counters and adaptively adjusts the number of LLC ways for DDIO or the tenants that demand more LLC capacity. In addition, Iat dynamically chooses the tenants that share its LLC resource with DDIO to minimize the performance interference by both the tenants and the I/O. Our experiments with multiple microbenchmarks and real-world applications demonstrate that with minimal overhead, Iat can effectively and stably reduce the performance degradation caused by DDIO.
AB - In modern server CPUs, last-level cache (LLC) is a critical hardware resource that exerts significant influence on the performance of the workloads, and how to manage LLC is a key to the performance isolation and QoS in the cloud with multi-tenancy. In this paper, we argue that in addition to CPU cores, high-speed I/O is also important for LLC management. This is because of an Intel architectural innovation - Data Direct I/O (DDIO) - that directly injects the inbound I/O traffic to (part of) the LLC instead of the main memory. We summarize two problems caused by DDIO and show that (1) the default DDIO configuration may not always achieve optimal performance, (2) DDIO can decrease the performance of non-I/O workloads that share LLC with it by as high as 32%.We then present, the first LLC management mechanism that treats the I/O as the first-class citizen. Iat monitors and analyzes the performance of the core/LLC/DDIO using CPU's hardware performance counters and adaptively adjusts the number of LLC ways for DDIO or the tenants that demand more LLC capacity. In addition, Iat dynamically chooses the tenants that share its LLC resource with DDIO to minimize the performance interference by both the tenants and the I/O. Our experiments with multiple microbenchmarks and real-world applications demonstrate that with minimal overhead, Iat can effectively and stably reduce the performance degradation caused by DDIO.
KW - Cache partitioning
KW - DDIO
KW - Performance isolation
UR - http://www.scopus.com/inward/record.url?scp=85114697261&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85114697261&partnerID=8YFLogxK
U2 - 10.1109/ISCA52012.2021.00018
DO - 10.1109/ISCA52012.2021.00018
M3 - Conference contribution
AN - SCOPUS:85114697261
T3 - Proceedings - International Symposium on Computer Architecture
SP - 112
EP - 125
BT - Proceedings - 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture, ISCA 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 48th ACM/IEEE Annual International Symposium on Computer Architecture, ISCA 2021
Y2 - 14 June 2021 through 19 June 2021
ER -