Using application skeletons to improve eScience infrastructure

Zhao Zhang, Daniel S. Katz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Computer scientists who work on tools and systems to support eScience (a variety of parallel and distributed) applications usually use actual applications to prove that their systems will benefit science and engineering (e.g., improve application performance). Accessing and building the applications and necessary data sets can be difficult because of policy or technical issues, and it can be difficult to modify the characteristics of the applications to understand corner cases in the system design. In this paper, we present the Application Skeleton, a simple yet powerful tool to build synthetic applications that represent real applications, with runtime and I/O close to those of the real applications. This allows computer scientists to focus on the system they are building, they can work with the simpler skeleton applications and be sure that their work will also be applicable to the real applications. In addition, skeleton applications support simple reproducible system experiments since they are represented by a compact set of parameters. Our Application Skeleton tool (available as open source at https://github.com/applicationskeleton/Skeleton) currently can create easy-to-Access, easy-to-build, and easy-to-run bag-of-task, (iterative) map-reduce, and (iterative) multistage workflow applications. The tasks can be serial or parallel or a mix of both. We select three representative applications (Montage, BLAST, CyberShake Postprocessing), then describe and generate skeleton applications for each. We show that the skeleton applications have identical (or close) performance to that of the real applications. We then show examples of using skeleton applications to verify system optimizations such as data caching, I/O tuning, and task scheduling, as well as the system resilience mechanism, in some cases modifying the skeleton applications to emphasize some characteristic, and thus show that using skeleton applications simplifies the process of designing, implementing, and testing these optimizations.

Original languageEnglish (US)
Title of host publicationProceedings - 2014 IEEE 10th International Conference on eScience, eScience 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages111-118
Number of pages8
ISBN (Electronic)9781479942886
DOIs
StatePublished - Dec 2 2014
Externally publishedYes
Event10th IEEE International Conference on eScience, eScience 2014 - Guaruja, Brazil
Duration: Oct 20 2014Oct 24 2014

Publication series

NameProceedings - 2014 IEEE 10th International Conference on eScience, eScience 2014
Volume1

Other

Other10th IEEE International Conference on eScience, eScience 2014
Country/TerritoryBrazil
CityGuaruja
Period10/20/1410/24/14

ASJC Scopus subject areas

  • General Computer Science

Fingerprint

Dive into the research topics of 'Using application skeletons to improve eScience infrastructure'. Together they form a unique fingerprint.

Cite this