A fresh perspective on developing and executing DAG-based distributed applications: A case-study of SAGA-based montage

Andre Merzky, Katerina Stamou, Shantenu Jha, Daniel S. Katz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Most workflow based applications currently have to adapt to available tools. While this keeps the cost of development low, it can lead to performance and flexibility tradeoffs that the application developer and deployer must make. In this paper, we use the Montage astronomical image mosaicking application as prototypical DAG-based workflow application to layout the development and deployment decisions for distributed applications. We discuss and explain the lack of simple (easy-to-use), scalable, and extensible distributed applications. We then introduce SAGA as a technology that permits the construction of abstractions that aid the development and execution of the applications, and thus addresses some of common shortcomings of traditional distributed applications development. We use Montage together with SAGA to examine how legacy applications can be made to run on distributed infrastructures, to see if our reasons are valid, and to compare potential new methods for creating distributed applications with existing technologies that are currently used. We demonstrate the ability to (i) scale-out and (ii) use different production infrastructure, while maintaining performance comparable to established systems. Our hope is that by demonstrating the simplicity of development along with other advantages (performance, scalability, extensibility, and infrastructure independence), this example will encourage others to think more broadly about how distributed applications are created and how new programming models such as Dryad can be supported in an infrastructure independent way, thus eventually leading to more applications that can seamlessly scale-out.

Original languageEnglish (US)
Title of host publicatione-Science 2009 - 5th IEEE International Conference on e-Science
Pages231-238
Number of pages8
DOIs
StatePublished - Dec 1 2009
Externally publishedYes
Event5th IEEE International Conference on e-Science, e-Science 2009 - Oxford, United Kingdom
Duration: Dec 9 2009Dec 11 2009

Publication series

Namee-Science 2009 - 5th IEEE International Conference on e-Science

Other

Other5th IEEE International Conference on e-Science, e-Science 2009
CountryUnited Kingdom
CityOxford
Period12/9/0912/11/09

ASJC Scopus subject areas

  • Arts and Humanities(all)
  • Computer Science(all)
  • Earth and Planetary Sciences(all)
  • Health Information Management

Fingerprint Dive into the research topics of 'A fresh perspective on developing and executing DAG-based distributed applications: A case-study of SAGA-based montage'. Together they form a unique fingerprint.

Cite this