MONAD: Self-Adaptive Micro-Service Infrastructure for Heterogeneous Scientific Workflows

Phuong Nguyen, Klara Nahrstedt

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Scientific workflows have become a popular computational model in a variety of application domains, such as astronomy, material science, physics, and biology. As scientific applications are moving to the cloud to take advantage of the elasticity and service level agreement of resources, there has been a number of recent research efforts on cloud-based workflow systems that support various types of performance guarantees under resource cost constraints. However, most of the related work often requires advanced knowledge about workflow structures to perform scheduling and resource optimization. In addition, existing workflow systems usually employ a monolithic approach in workflow implementation and execution, which makes them inefficient in dealing with heterogeneous types of workflows. In this paper, we present MONAD, a self-adaptive micro-service infrastructure for heterogeneous scientific workflows. Specifically, our micro-service architecture helps improve the flexibility of workflow composition and execution, and enables fine-grained scheduling at task level, considering task sharing across different workflows. In addition, we employ a feedback control approach with artificial neural network-based system identification to provide resource adaptation without any advanced knowledge of workflow structures. Our evaluation on multiple realistic heterogeneous workflows demonstrates that our system is robust and efficient in dealing with dynamic scientific workloads.

Original languageEnglish (US)
Title of host publicationProceedings - 2017 IEEE International Conference on Autonomic Computing, ICAC 2017
EditorsXiaorui Wang, Hui Lei, Christopher Stewart
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages187-196
Number of pages10
ISBN (Electronic)9781538617618
DOIs
StatePublished - Aug 8 2017
Event14th IEEE International Conference on Autonomic Computing, ICAC 2017 - Columbus, United States
Duration: Jul 17 2017Jul 21 2017

Publication series

NameProceedings - 2017 IEEE International Conference on Autonomic Computing, ICAC 2017

Other

Other14th IEEE International Conference on Autonomic Computing, ICAC 2017
CountryUnited States
CityColumbus
Period7/17/177/21/17

Keywords

  • infrastructure
  • micro-service
  • scientific workflow system
  • self-adaptive system

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'MONAD: Self-Adaptive Micro-Service Infrastructure for Heterogeneous Scientific Workflows'. Together they form a unique fingerprint.

Cite this