Understanding host interconnect congestion

Saksham Agarwal, Rachit Agarwal, Behnam Montazeri, Masoud Moshref, Khaled Elmeleegy, Luigi Rizzo, Marc Asher De Kruijf, Gautam Kumar, Sylvia Ratnasamy, David Culler, Amin Vahdat

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present evidence and characterization of host congestion in production clusters: adoption of high-bandwidth access links leading to emergence of bottlenecks within the host interconnect (NIC-to-CPU data path). We demonstrate that contention on existing IO memory management units and/or the memory subsystem can significantly reduce the available NIC-to-CPU bandwidth, resulting in hundreds of microseconds of queueing delays and eventual packet drops at hosts (even when running a state-of-the-art congestion control protocol that accounts for CPU-induced host congestion). We also discuss implications of host interconnect congestion to design of future host architecture, network stacks and network protocols.

Original languageEnglish (US)
Title of host publicationHotNets 2022 - Proceedings of the 2022 21st ACM Workshop on Hot Topics in Networks
PublisherAssociation for Computing Machinery
Pages198-204
Number of pages7
ISBN (Electronic)9781450398992
DOIs
StatePublished - Nov 14 2022
Externally publishedYes
Event21st ACM Workshop on Hot Topics in Networks, HotNets 2022 - Austin, United States
Duration: Nov 14 2022Nov 15 2022

Publication series

NameHotNets 2022 - Proceedings of the 2022 21st ACM Workshop on Hot Topics in Networks

Conference

Conference21st ACM Workshop on Hot Topics in Networks, HotNets 2022
Country/TerritoryUnited States
CityAustin
Period11/14/2211/15/22

Keywords

  • congestion control
  • datacenter transport
  • network hardware

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Understanding host interconnect congestion'. Together they form a unique fingerprint.

Cite this