Theius: A streaming visualization suite for hadoop clusters

Jon Tedesco, Roman Dudko, Abhishek Sharma, Reza Farivar, Roy Campbell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As cloud computing clusters continue to grow, maintaining the health of these clusters becomes increasingly challenging. Recent work has studied how we can efficiently monitor the status of machines in these clusters and how we can detect problems or predict them before they occur, yet little work has focused on addressing the bottleneck between when these failures occur and when they are fixed: system administrators. As monitoring and failure detection systems mature, we are able to extract tremendous amounts of information about the status of the system in real time. However, this amount of data is difficult to understand for human beings, especially those inexperienced with the particular cluster. In this paper, we introduce a web-based visualization suite called Theius to allow system administrators to quickly understand the state of the cloud system as a whole. We outline the key features of this visualization tool, and show that it is more intuitive and easy to use than Ganglia, a state-of-the-art visualization tool for clusters. Likewise, we demonstrate that our tool can scale, presenting a use case with our visualization showing a 5000 node cluster. Although our tool is implemented for Hadoop clusters, our contribution is general to any cloud computing system.

Original languageEnglish (US)
Title of host publicationProceedings of the IEEE International Conference on Cloud Engineering, IC2E 2013
Pages177-182
Number of pages6
DOIs
StatePublished - Aug 12 2013
Event1st IEEE International Conference on Cloud Engineering, IC2E 2013 - San Francisco, CA, United States
Duration: Mar 25 2013Mar 28 2013

Publication series

NameProceedings of the IEEE International Conference on Cloud Engineering, IC2E 2013

Other

Other1st IEEE International Conference on Cloud Engineering, IC2E 2013
CountryUnited States
CitySan Francisco, CA
Period3/25/133/28/13

    Fingerprint

Keywords

  • Cloud computing
  • Cluster computing
  • Failure detection
  • Failure prediction
  • Hadoop
  • Monitoring
  • Visualization

ASJC Scopus subject areas

  • Software

Cite this

Tedesco, J., Dudko, R., Sharma, A., Farivar, R., & Campbell, R. (2013). Theius: A streaming visualization suite for hadoop clusters. In Proceedings of the IEEE International Conference on Cloud Engineering, IC2E 2013 (pp. 177-182). [6529282] (Proceedings of the IEEE International Conference on Cloud Engineering, IC2E 2013). https://doi.org/10.1109/IC2E.2013.36