μManycore: A Cloud-Native CPU for Tail at Scale

Jovan Stojkovic, Muhammad Shahbaz, Josep Torrellas, Chunao Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Microservices are emerging as a popular cloud-computing paradigm. Microservice environments execute typically-short service requests that interact with one another via remote procedure calls (often across machines), and are subject to stringent tail-latency constraints. In contrast, current processors are designed for traditional monolithic applications. They support global hardware cache coherence, provide large caches, incorporate microarchitecture for long-running, predictable applications (such as advanced prefetching), and are optimized to minimize average latency rather than tail latency. To address this imbalance, this paper proposes μManycore, an architecture optimized for cloud-native microservice environments. Based on a characterization of microservice applications, μManycore is designed to minimize unnecessary microarchitecture and mitigate overheads to reduce tail latency. Indeed, rather than supporting manycore-wide hardware cache coherence, μManycore has multiple small hardware cache-coherent domains, called Villages. Clusters of villages are interconnected with an on-package leaf-spine network, which has many redundant, low-hop-count paths between clusters. To minimize latency overheads, μManycore schedules and queues service requests in hardware, and includes hardware support to save and restore process state when doing a context-switch. Our simulation-based results show that μManycore delivers high performance. A cluster of 10 servers with a 1024-core μManycore in each server delivers 3.7× lower average latency, 15.5× higher throughput, and, importantly, 10.4× lower tail latency than a cluster with iso-power conventional server-class multicores. Similar good results are attained compared to a cluster with power-hungry iso-area conventional server-class multicores.

Original languageEnglish (US)
Title of host publicationISCA 2023 - Proceedings of the 2023 50th Annual International Symposium on Computer Architecture
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages459-473
Number of pages15
ISBN (Electronic)9798400700958
DOIs
StatePublished - Jun 17 2023
Event50th Annual International Symposium on Computer Architecture, ISCA 2023 - Orlando, United States
Duration: Jun 17 2023Jun 21 2023

Publication series

NameProceedings - International Symposium on Computer Architecture
ISSN (Print)1063-6897

Conference

Conference50th Annual International Symposium on Computer Architecture, ISCA 2023
Country/TerritoryUnited States
CityOrlando
Period6/17/236/21/23

Keywords

  • Microservices
  • cloud computing
  • manycore architecture

ASJC Scopus subject areas

  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'μManycore: A Cloud-Native CPU for Tail at Scale'. Together they form a unique fingerprint.

Cite this