FTC-Charm++: An in-memory checkpoint-based fault tolerant runtime for Charm++ and MPI

Gengbin Zheng, Lixia Shi, Laxmikant V. Kalé

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As high performance clusters continue to grow in size, the mean time between failure shrinks. Thus, the issues of fault tolerance and reliability are becoming one of the challenging factors for application scalability. The traditional disk-based method of dealing with faults is to checkpoint the state of the entire application periodically to reliable storage and restart from the recent checkpoint. The recovery of the application from faults involves (often manually) restarting applications on all processors and having it read the data from disks on all processors. The restart can therefore take minutes after it has been initiated. Such a strategy requires that the failed processor can be replaced so that the number of processors at checkpoint-time and recovery-time are the same. We present FTC-Charm++, afault-tolerant runtime based on a scheme for fast and scalable in-memory checkpoint and restart. At restart, when there is no extra processor, the program can continue to run on the remaining processors while minimizing the performance penalty due to losing processors. The method is useful for applications whose memory footprint is small at the checkpoint state, while a variation of this scheme - in-disk checkpoint/restart can be applied to applications with large memory footprint. The scheme does not require any individual component to be fault-free. We have implemented this scheme for Charm++ and MAPI (an adaptive version of MPI). This paper describes the scheme and shows performance data on a cluster using 128 processors.

Original languageEnglish (US)
Title of host publication2004 IEEE International Conference on Cluster Computing, ICCC 2004
Pages93-103
Number of pages11
DOIs
StatePublished - 2004
Event2004 IEEE International Conference on Cluster Computing, ICCC 2004 - San Diego, CA, United States
Duration: Sep 20 2004Sep 23 2004

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
ISSN (Print)1552-5244

Other

Other2004 IEEE International Conference on Cluster Computing, ICCC 2004
CountryUnited States
CitySan Diego, CA
Period9/20/049/23/04

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'FTC-Charm++: An in-memory checkpoint-based fault tolerant runtime for Charm++ and MPI'. Together they form a unique fingerprint.

Cite this