Fault injection-based assessment of partial fault tolerance in stream processing applications

Gabriela Jacques-Silva, Bugra Gedik, Henrique Andrade, Kun Lung Wu, Ravishankar K. Iyer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper describes an experimental methodology used to evaluate the effectiveness of partial fault tolerance (PFT) techniques in data stream processing applications. Without a clear understanding of the impact of faults on the quality of the application output, applying PFT techniques in practice is not viable. We assess the impact of PFT by injecting faults into a synthetic financial engineering application running on top of IBM's stream processing middleware, System S. The application output quality degradation is evaluated via an application-specific output score function. In addition, we propose four metrics that are aimed at assessing the impact of faults in different stream operators of the application flow graph with respect to predictability and availability. These metrics help the developer to decide where in the application he should place redundant resources. We show that PFT is indeed viable, which opens the way for considerably reducing the resource consumption when compared to fully consistent replicas.

Original languageEnglish (US)
Title of host publicationDEBS'11 - Proceedings of the 5th ACM International Conference on Distributed Event-Based Systems
Pages231-242
Number of pages12
DOIs
StatePublished - 2011
Event5th ACM International Conference on Distributed Event-Based Systems, DEBS'11 - New York, NY, United States
Duration: Jul 11 2011Jul 15 2011

Publication series

NameDEBS'11 - Proceedings of the 5th ACM International Conference on Distributed Event-Based Systems

Other

Other5th ACM International Conference on Distributed Event-Based Systems, DEBS'11
Country/TerritoryUnited States
CityNew York, NY
Period7/11/117/15/11

Keywords

  • evaluation metrics
  • fault injection
  • fault tolerance
  • stream processing

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems

Fingerprint

Dive into the research topics of 'Fault injection-based assessment of partial fault tolerance in stream processing applications'. Together they form a unique fingerprint.

Cite this