SPBC: Leveraging the characteristics of MPI HPC applications for scalable checkpointing

The high failure rate expected for future supercomputers re-quires the design of new fault tolerant solutions. Most check-pointing protocols are designed to work with any message-passing application but suffer from scalability issues at ex-treme scale. We take a different approach: We identify a property common to many HPC applications, namely channel-determinism, and introduce a new partial order re-lation, called always-happens-before relation, between events of such applications. Leveraging these two concepts, we de-sign a protocol that combines an unprecedented set of fea-tures. Our protocol called SPBC combines in a hierarchical way coordinated checkpointing and message logging. It is the first protocol that provides failure containment without logging any information reliably apart from process check-points, and this, without penalizing recovery performance. Experiments run with a representative set of HPC work-loads demonstrate a good performance of our protocol during both, failure-free execution and recovery.