Preventive migration vs. preventive checkpointing for extreme scale supercomputers

Franck Cappello, Henri Casanova, Yves Robert

Research output: Contribution to journalArticlepeer-review

Abstract

An alternative to classical fault-tolerant approaches for large-scale clusters is failure avoidance, by which the occurrence of a fault is predicted and a preventive measure is taken. We develop analytical performance models for two types of preventive measures: preventive checkpointing and preventive migration. We instantiate these models for platform scenarios representative of current and future technology trends. We find that preventive migration is the better approach in the short term by orders of magnitude. However, in the longer term, both approaches have comparable merit with a marginal advantage for preventive checkpointing. We also develop an analytical model of the performance for fault tolerance based on periodic checkpointing and compare this approach to both failure avoidance techniques. We find that this comparison is sensitive to the nature of the stochastic distribution of the time between failures, and that failure avoidance is likely inferior to fault tolerance in the long term. Regardless, our result show that each approach is likely to achieve poor utilization for large-scale platforms (e.g., 220 nodes) unless the mean time between failures is large. We show how bounding parallel job size improves utilization, but conclude that achieving good utilization in future large-scale platforms will require a combination of techniques.

Original languageEnglish (US)
Pages (from-to)111-132
Number of pages22
JournalParallel Processing Letters
Volume21
Issue number2
DOIs
StatePublished - Jun 2011
Externally publishedYes

Keywords

  • checkpointing
  • failure prediction
  • migration
  • parallel jobs

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Preventive migration vs. preventive checkpointing for extreme scale supercomputers'. Together they form a unique fingerprint.

Cite this