CodedBulk: Inter-datacenter bulk transfers using network coding

Shih Hao Tseng, Saksham Agarwal, Rachit Agarwal, Hitesh Ballani, Ao Tang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents CodedBulk, a system for high-throughput inter-datacenter bulk transfers. At its core, CodedBulk uses network coding, a technique from the coding theory community, that guarantees optimal throughput for individual bulk transfers. Prior attempts to using network coding in wired networks have faced several pragmatic and fundamental barriers. CodedBulk resolves these barriers by exploiting the unique properties of inter-datacenter networks, and by using a custom-designed hop-by-hop flow control mechanism that enables efficient realization of network coding atop existing transport protocols. An end-to-end CodedBulk implementation running on a geo-distributed inter-datacenter network improves bulk transfer throughput by 1.2-2.5× compared to state-of-the-art mechanisms that do not use network coding.

Original languageEnglish (US)
Title of host publicationProceedings of the 18th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2021
PublisherUSENIX Association
Pages15-28
Number of pages14
ISBN (Electronic)9781939133212
StatePublished - 2021
Externally publishedYes
Event18th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2021 - Virtual, Online
Duration: Apr 12 2021Apr 14 2021

Conference

Conference18th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2021
CityVirtual, Online
Period4/12/214/14/21

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Control and Systems Engineering

Fingerprint

Dive into the research topics of 'CodedBulk: Inter-datacenter bulk transfers using network coding'. Together they form a unique fingerprint.

Cite this