On the use of cluster-based partial message logging to improve fault tolerance for MPI HPC applications

Thomas Ropars, Amina Guermouche, Bora Uçar, Esteban Meneses, Laxmikant V. Kalé, Franck Cappello

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Fault tolerance is becoming a major concern in HPC systems. The two traditional approaches for message passing applications, coordinated checkpointing and message logging, have severe scalability issues. Coordinated checkpointing protocols make all processes roll back after a failure. Message logging protocols log a huge amount of data and can induce an overhead on communication performance. Hierarchical rollback-recovery protocols based on the combination of coordinated checkpointing and message logging are an alternative. These partial message logging protocols are based on process clustering: only messages between clusters are logged to limit the consequence of a failure to one cluster. These protocols would work efficiently only if one can find clusters of processes in the applications such that the ratio of logged messages is very low. We study the communication patterns of message passing HPC applications to show that partial message logging is suitable in most cases. We propose a partitioning algorithm to find suitable clusters of processes given the communication pattern of an application. Finally, we evaluate the efficiency of partial message logging using two state of the art protocols on a set of representative applications.

Original languageEnglish (US)
Title of host publicationEuro-Par 2011 Parallel Processing - 17th International Conference, Proceedings
PublisherSpringer-Verlag Berlin Heidelberg
Pages567-578
Number of pages12
EditionPART 1
ISBN (Print)9783642233999
DOIs
StatePublished - 2011
Event17th International Conference on Parallel Processing, Euro-Par 2011 - Bordeaux, France
Duration: Aug 29 2011Sep 2 2011

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume6852 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other17th International Conference on Parallel Processing, Euro-Par 2011
CountryFrance
CityBordeaux
Period8/29/119/2/11

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'On the use of cluster-based partial message logging to improve fault tolerance for MPI HPC applications'. Together they form a unique fingerprint.

Cite this