Network coding for distributed storage systems

Alexandras G. Dimakis, P. Brighten Godfrey, Martin J. Wainwright, Kannan Ramchandran

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Peer-to-peer distributed storage systems provide reliable access to data through redundancy spread over nodes across the Internet. A key goal is to minimize the amount of bandwidth used to maintain that redundancy. Storing a file using an erasure code, in fragments spread across nodes, promises to require less redundancy and hence less maintenance bandwidth than simple replication to provide the same level of reliability. However, since fragments must be periodically replaced as nodes fail, a key question is how to generate a new fragment in a distributed way while transferring as little data as possible across the network. In this paper, we introduce a general technique to analyze storage architectures that combine any form of coding and replication, as well as presenting two new schemes for maintaining redundancy using erasure codes. First, we show how to optimally generate MDS fragments directly from existing fragments in the system. Second, we introduce a new scheme called Regenerating Codes which use slightly larger fragments than MDS but have lower overall bandwidth use. We also show through simulation that in realistic environments, Regenerating Codes can reduce maintenance bandwidth use by 25% or more compared with the best previous design - a hybrid of replication and erasure codes - while simplifying system architecture.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE INFOCOM 2007
Subtitle of host publication26th IEEE International Conference on Computer Communications
Pages2000-2008
Number of pages9
DOIs
StatePublished - 2007
Externally publishedYes
EventIEEE INFOCOM 2007: 26th IEEE International Conference on Computer Communications - Anchorage, AK, United States
Duration: May 6 2007May 12 2007

Publication series

NameProceedings - IEEE INFOCOM
ISSN (Print)0743-166X

Other

OtherIEEE INFOCOM 2007: 26th IEEE International Conference on Computer Communications
CountryUnited States
CityAnchorage, AK
Period5/6/075/12/07

ASJC Scopus subject areas

  • Computer Science(all)
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Network coding for distributed storage systems'. Together they form a unique fingerprint.

Cite this