Cross-layer scheduling in cloud systems

Hilfi Alkaff, Indranil Gupta, Luke M. Leslie

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Today, cloud computing engines such as streamprocessing Storm and batch-processing Hadoop are being increasingly run atop software-defined networks (SDNs). In such cloud stacks, the scheduler of the application engine (which allocates tasks to servers) remains decoupled from the SDN scheduler (which allocates network routes). We propose 1 a new approach that performs cross-layer scheduling between the application layer and the networking layer. This coordinated scheduling orchestrates the placement of application tasks (e.g., Hadoop maps and reduces, or Storm bolts) in tandem with the selection of network routes that arise from these tasks. We present results from both cluster deployment and simulation, and using two representative network topologies: Fat-tree and Jellyfish. Our results show that cross-layer scheduling can improve throughput of Hadoop and Storm by between 26% to 34% in a 30-host cluster, and it scales well.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages236-245
Number of pages10
ISBN (Electronic)9781479982189
DOIs
StatePublished - Jan 1 2015
Event2015 IEEE International Conference on Cloud Engineering, IC2E 2015 - Tempe, United States
Duration: Mar 9 2015Mar 12 2015

Publication series

NameProceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015

Other

Other2015 IEEE International Conference on Cloud Engineering, IC2E 2015
CountryUnited States
CityTempe
Period3/9/153/12/15

Keywords

  • Cloud computing
  • Cross-layer
  • Hadoop
  • SDN
  • Storm

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Computer Networks and Communications
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Cross-layer scheduling in cloud systems'. Together they form a unique fingerprint.

  • Cite this

    Alkaff, H., Gupta, I., & Leslie, L. M. (2015). Cross-layer scheduling in cloud systems. In Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015 (pp. 236-245). [7092925] (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IC2E.2015.36