Cross-layer scheduling in cloud systems

Hilfi Alkaff, Indranil Gupta, Luke M. Leslie

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Today, cloud computing engines such as streamprocessing Storm and batch-processing Hadoop are being increasingly run atop software-defined networks (SDNs). In such cloud stacks, the scheduler of the application engine (which allocates tasks to servers) remains decoupled from the SDN scheduler (which allocates network routes). We propose 1 a new approach that performs cross-layer scheduling between the application layer and the networking layer. This coordinated scheduling orchestrates the placement of application tasks (e.g., Hadoop maps and reduces, or Storm bolts) in tandem with the selection of network routes that arise from these tasks. We present results from both cluster deployment and simulation, and using two representative network topologies: Fat-tree and Jellyfish. Our results show that cross-layer scheduling can improve throughput of Hadoop and Storm by between 26% to 34% in a 30-host cluster, and it scales well.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages236-245
Number of pages10
ISBN (Electronic)9781479982189
DOIs
StatePublished - Jan 1 2015
Event2015 IEEE International Conference on Cloud Engineering, IC2E 2015 - Tempe, United States
Duration: Mar 9 2015Mar 12 2015

Publication series

NameProceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015

Other

Other2015 IEEE International Conference on Cloud Engineering, IC2E 2015
CountryUnited States
CityTempe
Period3/9/153/12/15

Fingerprint

Scheduling
Engines
Bolts
Cloud computing
Oils and fats
Servers
Throughput
Topology

Keywords

  • Cloud computing
  • Cross-layer
  • Hadoop
  • SDN
  • Storm

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Computer Networks and Communications
  • Hardware and Architecture

Cite this

Alkaff, H., Gupta, I., & Leslie, L. M. (2015). Cross-layer scheduling in cloud systems. In Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015 (pp. 236-245). [7092925] (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IC2E.2015.36

Cross-layer scheduling in cloud systems. / Alkaff, Hilfi; Gupta, Indranil; Leslie, Luke M.

Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 236-245 7092925 (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Alkaff, H, Gupta, I & Leslie, LM 2015, Cross-layer scheduling in cloud systems. in Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015., 7092925, Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015, Institute of Electrical and Electronics Engineers Inc., pp. 236-245, 2015 IEEE International Conference on Cloud Engineering, IC2E 2015, Tempe, United States, 3/9/15. https://doi.org/10.1109/IC2E.2015.36
Alkaff H, Gupta I, Leslie LM. Cross-layer scheduling in cloud systems. In Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 236-245. 7092925. (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015). https://doi.org/10.1109/IC2E.2015.36
Alkaff, Hilfi ; Gupta, Indranil ; Leslie, Luke M. / Cross-layer scheduling in cloud systems. Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 236-245 (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015).
@inproceedings{f440cbc1a8d74a1f862efa517a8f77c9,
title = "Cross-layer scheduling in cloud systems",
abstract = "Today, cloud computing engines such as streamprocessing Storm and batch-processing Hadoop are being increasingly run atop software-defined networks (SDNs). In such cloud stacks, the scheduler of the application engine (which allocates tasks to servers) remains decoupled from the SDN scheduler (which allocates network routes). We propose 1 a new approach that performs cross-layer scheduling between the application layer and the networking layer. This coordinated scheduling orchestrates the placement of application tasks (e.g., Hadoop maps and reduces, or Storm bolts) in tandem with the selection of network routes that arise from these tasks. We present results from both cluster deployment and simulation, and using two representative network topologies: Fat-tree and Jellyfish. Our results show that cross-layer scheduling can improve throughput of Hadoop and Storm by between 26{\%} to 34{\%} in a 30-host cluster, and it scales well.",
keywords = "Cloud computing, Cross-layer, Hadoop, SDN, Storm",
author = "Hilfi Alkaff and Indranil Gupta and Leslie, {Luke M.}",
year = "2015",
month = "1",
day = "1",
doi = "10.1109/IC2E.2015.36",
language = "English (US)",
series = "Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "236--245",
booktitle = "Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015",
address = "United States",

}

TY - GEN

T1 - Cross-layer scheduling in cloud systems

AU - Alkaff, Hilfi

AU - Gupta, Indranil

AU - Leslie, Luke M.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Today, cloud computing engines such as streamprocessing Storm and batch-processing Hadoop are being increasingly run atop software-defined networks (SDNs). In such cloud stacks, the scheduler of the application engine (which allocates tasks to servers) remains decoupled from the SDN scheduler (which allocates network routes). We propose 1 a new approach that performs cross-layer scheduling between the application layer and the networking layer. This coordinated scheduling orchestrates the placement of application tasks (e.g., Hadoop maps and reduces, or Storm bolts) in tandem with the selection of network routes that arise from these tasks. We present results from both cluster deployment and simulation, and using two representative network topologies: Fat-tree and Jellyfish. Our results show that cross-layer scheduling can improve throughput of Hadoop and Storm by between 26% to 34% in a 30-host cluster, and it scales well.

AB - Today, cloud computing engines such as streamprocessing Storm and batch-processing Hadoop are being increasingly run atop software-defined networks (SDNs). In such cloud stacks, the scheduler of the application engine (which allocates tasks to servers) remains decoupled from the SDN scheduler (which allocates network routes). We propose 1 a new approach that performs cross-layer scheduling between the application layer and the networking layer. This coordinated scheduling orchestrates the placement of application tasks (e.g., Hadoop maps and reduces, or Storm bolts) in tandem with the selection of network routes that arise from these tasks. We present results from both cluster deployment and simulation, and using two representative network topologies: Fat-tree and Jellyfish. Our results show that cross-layer scheduling can improve throughput of Hadoop and Storm by between 26% to 34% in a 30-host cluster, and it scales well.

KW - Cloud computing

KW - Cross-layer

KW - Hadoop

KW - SDN

KW - Storm

UR - http://www.scopus.com/inward/record.url?scp=84944345986&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944345986&partnerID=8YFLogxK

U2 - 10.1109/IC2E.2015.36

DO - 10.1109/IC2E.2015.36

M3 - Conference contribution

AN - SCOPUS:84944345986

T3 - Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015

SP - 236

EP - 245

BT - Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -