Can Applications Recover from fsync Failures?

Anthony Rebello; Yuvraj Patel; Ramnatthan Alagappan; Andrea C. Arpaci-Dusseau; Remzi H. Arpaci-Dusseau

doi:10.1145/3450338

Can Applications Recover from fsync Failures?

Anthony Rebello, Yuvraj Patel, Ramnatthan Alagappan, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau

Research output: Contribution to journal › Article › peer-review

Abstract

We analyze how file systems and modern data-intensive applications react to fsync failures. First, we characterize how three Linux file systems (ext4, XFS, Btrfs) behave in the presence of failures. We find commonalities across file systems (pages are always marked clean, certain block writes always lead to unavailability) as well as differences (page content and failure reporting is varied). Next, we study how five widely used applications (PostgreSQL, LMDB, LevelDB, SQLite, Redis) handle fsync failures. Our findings show that although applications use many failure-handling strategies, none are sufficient: fsync failures can cause catastrophic outcomes such as data loss and corruption. Our findings have strong implications for the design of file systems and applications that intend to provide strong durability guarantees.

Original language	English (US)
Article number	3450338
Journal	ACM Transactions on Storage
Volume	17
Issue number	2
DOIs	https://doi.org/10.1145/3450338
State	Published - Jun 2021
Externally published	Yes

Keywords

Durability
file system
fsync
fsync failures
persistence

ASJC Scopus subject areas

Hardware and Architecture

Online availability

10.1145/3450338

Library availability

Discover UIUC Full Text

Cite this

@article{57aca7dfa38744d6b70308918856b68e,

title = "Can Applications Recover from fsync Failures?",

abstract = "We analyze how file systems and modern data-intensive applications react to fsync failures. First, we characterize how three Linux file systems (ext4, XFS, Btrfs) behave in the presence of failures. We find commonalities across file systems (pages are always marked clean, certain block writes always lead to unavailability) as well as differences (page content and failure reporting is varied). Next, we study how five widely used applications (PostgreSQL, LMDB, LevelDB, SQLite, Redis) handle fsync failures. Our findings show that although applications use many failure-handling strategies, none are sufficient: fsync failures can cause catastrophic outcomes such as data loss and corruption. Our findings have strong implications for the design of file systems and applications that intend to provide strong durability guarantees.",

keywords = "Durability, file system, fsync, fsync failures, persistence",

author = "Anthony Rebello and Yuvraj Patel and Ramnatthan Alagappan and Arpaci-Dusseau, {Andrea C.} and Arpaci-Dusseau, {Remzi H.}",

note = "Funding Information: We thank our sponsors: VMware, Intel, Seagate, and Samsung, for their generous support. This material was also supported by funding from NSF under award numbers CNS-1421033, CNS-1763810, and CNS-1838733, and DOE DE-SC0014935. Funding Information: Ramnatthan Alagappan work done while at University of Wisconsin-Madison. We thank our sponsors: VMware, Intel, Seagate, and Samsung, for their generous support. This material was also supported by funding from NSF under award numbers CNS-1421033, CNS-1763810, and CNS-1838733, and DOE DE-SC0014935. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF, DOE, or any other institutions. Authors{\textquoteright} addresses: A. Rebello, Y. Patel, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, University of Wisconsin– Madison, Department of Computer Sciences, Madison, WI, 53706; emails: [email protected], {yuvraj, dusseau, remzi}@cs.wisc.edu; R. Alagappan, VMware Research Group, Madison, WI, 53706; email: [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. {\textcopyright} 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM. 1553-3077/2021/06-ART12 $15.00 https://doi.org/10.1145/3450338 Publisher Copyright: {\textcopyright} 2021 held by the owner/author(s). Publication rights licensed to ACM.",

year = "2021",

month = jun,

doi = "10.1145/3450338",

language = "English (US)",

volume = "17",

journal = "ACM Transactions on Storage",

issn = "1553-3077",