Scale up vs. scale out in cloud storage and graph processing systems

Wenting Wang, Le Xu, Indranil Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deployers of cloud storage and iterative processing systems typically have to deal with either dollar budget constraints or throughput requirements. This paper examines the question of whether such cloud storage and iterative processing systems are more cost-efficient when scheduled on a COTS (scale out) cluster or a single beefy (scale up) machine. We experimentally evaluate two systems: 1) a distributed key-value store (Cassandra), and 2) a distributed graph processing system (GraphLab). Our studies reveal scenarios where each option is preferable over the other. We provide recommendations for deployers of such systems to decide between scale up vs. scale out, as a function of their dollar or throughput constraints. Our results indicate that there is a need for adaptive scheduling in heterogeneous clusters containing scale up and scale out nodes.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages428-433
Number of pages6
ISBN (Electronic)9781479982189
DOIs
StatePublished - Jan 1 2015
Event2015 IEEE International Conference on Cloud Engineering, IC2E 2015 - Tempe, United States
Duration: Mar 9 2015Mar 12 2015

Publication series

NameProceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015

Other

Other2015 IEEE International Conference on Cloud Engineering, IC2E 2015
CountryUnited States
CityTempe
Period3/9/153/12/15

Fingerprint

Processing
Throughput
Scheduling
Costs

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Computer Networks and Communications
  • Hardware and Architecture

Cite this

Wang, W., Xu, L., & Gupta, I. (2015). Scale up vs. scale out in cloud storage and graph processing systems. In Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015 (pp. 428-433). [7092956] (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IC2E.2015.55

Scale up vs. scale out in cloud storage and graph processing systems. / Wang, Wenting; Xu, Le; Gupta, Indranil.

Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 428-433 7092956 (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, W, Xu, L & Gupta, I 2015, Scale up vs. scale out in cloud storage and graph processing systems. in Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015., 7092956, Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015, Institute of Electrical and Electronics Engineers Inc., pp. 428-433, 2015 IEEE International Conference on Cloud Engineering, IC2E 2015, Tempe, United States, 3/9/15. https://doi.org/10.1109/IC2E.2015.55
Wang W, Xu L, Gupta I. Scale up vs. scale out in cloud storage and graph processing systems. In Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 428-433. 7092956. (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015). https://doi.org/10.1109/IC2E.2015.55
Wang, Wenting ; Xu, Le ; Gupta, Indranil. / Scale up vs. scale out in cloud storage and graph processing systems. Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 428-433 (Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015).
@inproceedings{c1ce291cd7d148a5998b8bdd089351b0,
title = "Scale up vs. scale out in cloud storage and graph processing systems",
abstract = "Deployers of cloud storage and iterative processing systems typically have to deal with either dollar budget constraints or throughput requirements. This paper examines the question of whether such cloud storage and iterative processing systems are more cost-efficient when scheduled on a COTS (scale out) cluster or a single beefy (scale up) machine. We experimentally evaluate two systems: 1) a distributed key-value store (Cassandra), and 2) a distributed graph processing system (GraphLab). Our studies reveal scenarios where each option is preferable over the other. We provide recommendations for deployers of such systems to decide between scale up vs. scale out, as a function of their dollar or throughput constraints. Our results indicate that there is a need for adaptive scheduling in heterogeneous clusters containing scale up and scale out nodes.",
author = "Wenting Wang and Le Xu and Indranil Gupta",
year = "2015",
month = "1",
day = "1",
doi = "10.1109/IC2E.2015.55",
language = "English (US)",
series = "Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "428--433",
booktitle = "Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015",
address = "United States",

}

TY - GEN

T1 - Scale up vs. scale out in cloud storage and graph processing systems

AU - Wang, Wenting

AU - Xu, Le

AU - Gupta, Indranil

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Deployers of cloud storage and iterative processing systems typically have to deal with either dollar budget constraints or throughput requirements. This paper examines the question of whether such cloud storage and iterative processing systems are more cost-efficient when scheduled on a COTS (scale out) cluster or a single beefy (scale up) machine. We experimentally evaluate two systems: 1) a distributed key-value store (Cassandra), and 2) a distributed graph processing system (GraphLab). Our studies reveal scenarios where each option is preferable over the other. We provide recommendations for deployers of such systems to decide between scale up vs. scale out, as a function of their dollar or throughput constraints. Our results indicate that there is a need for adaptive scheduling in heterogeneous clusters containing scale up and scale out nodes.

AB - Deployers of cloud storage and iterative processing systems typically have to deal with either dollar budget constraints or throughput requirements. This paper examines the question of whether such cloud storage and iterative processing systems are more cost-efficient when scheduled on a COTS (scale out) cluster or a single beefy (scale up) machine. We experimentally evaluate two systems: 1) a distributed key-value store (Cassandra), and 2) a distributed graph processing system (GraphLab). Our studies reveal scenarios where each option is preferable over the other. We provide recommendations for deployers of such systems to decide between scale up vs. scale out, as a function of their dollar or throughput constraints. Our results indicate that there is a need for adaptive scheduling in heterogeneous clusters containing scale up and scale out nodes.

UR - http://www.scopus.com/inward/record.url?scp=84944324357&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944324357&partnerID=8YFLogxK

U2 - 10.1109/IC2E.2015.55

DO - 10.1109/IC2E.2015.55

M3 - Conference contribution

AN - SCOPUS:84944324357

T3 - Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015

SP - 428

EP - 433

BT - Proceedings - 2015 IEEE International Conference on Cloud Engineering, IC2E 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -