Budget-constrained bulk data transfer via internet and shipping networks

Brian Cho, Indranil Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Cloud collaborators wish to combine large amounts of data, in the order of TBs, from multiple distributed locations to a single datacenter. Such groups are faced with the challenge of reducing the latency of the transfer, without incurring excessive dollar costs. Our Pandora system is an autonomic system that creates data transfer plans that can satisfy latency and cost needs, by considering transferring the data through both Internet and disk shipments. Solving the planning problem is a critical step towards a truly autonomic bulk data transfer service. In this paper, we develop techniques to create an optimal transfer plan that minimizes transfer latency subject to a budget constraint. To systematically explore the solution space, we develop efficient binary search methods that find the optimal shipment transfer plan. Our experimental evaluation, driven by Internet bandwidth traces and actual shipment costs queried from FedEx web services, shows that these techniques work well on diverse, realistic networks.

Original languageEnglish (US)
Title of host publicationProceedings of the 8th ACM International Conference on Autonomic Computing, ICAC 2011 and Co-located Workshops
Pages71-80
Number of pages10
DOIs
StatePublished - Jul 15 2011
Event8th ACM International Conference on Autonomic Computing, ICAC 2011 and Co-located Workshops - Karlsruhe, Germany
Duration: Jun 14 2011Jun 18 2011

Publication series

NameProceedings of the 8th ACM International Conference on Autonomic Computing, ICAC 2011 and Co-located Workshops

Other

Other8th ACM International Conference on Autonomic Computing, ICAC 2011 and Co-located Workshops
CountryGermany
CityKarlsruhe
Period6/14/116/18/11

Keywords

  • cloud computing
  • data-intensive computing
  • wide-area data transfer

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Budget-constrained bulk data transfer via internet and shipping networks'. Together they form a unique fingerprint.

  • Cite this

    Cho, B., & Gupta, I. (2011). Budget-constrained bulk data transfer via internet and shipping networks. In Proceedings of the 8th ACM International Conference on Autonomic Computing, ICAC 2011 and Co-located Workshops (pp. 71-80). (Proceedings of the 8th ACM International Conference on Autonomic Computing, ICAC 2011 and Co-located Workshops). https://doi.org/10.1145/1998582.1998595