Modeling stream processing applications for dependability evaluation

Gabriela Jacques-Silva, Zbigniew T Kalbarczyk, Bugra Gedik, Henrique Andrade, Kun Lung Wu, Ravishankar K Iyer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper describes a modeling framework for evaluating the impact of faults on the output of streaming applications. Our model is based on three abstractions: stream operators, stream connections, and tuples. By composing these abstractions within a Stochastic Activity Network, we allow the modeling of complete applications. We consider faults that lead to data loss and to silent data corruption (SDC). Our framework captures how faults originating in one operator propagate to other operators down the stream processing graph. We demonstrate the extensibility of our framework by evaluating three different fault tolerance techniques: checkpointing, partial graph replication, and full graph replication. We show that under crashes that lead to data loss, partial graph replication has a great advantage in maintaining the accuracy of the application output when compared to checkpointing. We also show that SDC can break the no data duplication guarantees of a full graph replication-based fault tolerance technique.

Original languageEnglish (US)
Title of host publication2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks, DSN 2011
Pages430-441
Number of pages12
DOIs
StatePublished - Aug 26 2011
Event2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks, DSN 2011 - Hong Kong, Hong Kong
Duration: Jun 27 2011Jun 30 2011

Publication series

NameProceedings of the International Conference on Dependable Systems and Networks

Other

Other2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks, DSN 2011
CountryHong Kong
CityHong Kong
Period6/27/116/30/11

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Modeling stream processing applications for dependability evaluation'. Together they form a unique fingerprint.

  • Cite this

    Jacques-Silva, G., Kalbarczyk, Z. T., Gedik, B., Andrade, H., Wu, K. L., & Iyer, R. K. (2011). Modeling stream processing applications for dependability evaluation. In 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks, DSN 2011 (pp. 430-441). [5958256] (Proceedings of the International Conference on Dependable Systems and Networks). https://doi.org/10.1109/DSN.2011.5958256