Accelerating Relative-error Bounded Lossy Compression for HPC datasets with Precomputation-Based Mechanisms

Xiangyu Zou, Tao Lu, Wen Xia, Xuan Wang, Weizhe Zhang, Sheng Di, Dingwen Tao, Franck Cappello

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Scientific simulations in high-performance computing (HPC) environments are producing vast volume of data, which may cause a severe I/O bottleneck at runtime and a huge burden on storage space for post-analysis. Unlike the traditional data reduction schemes (such as deduplication or lossless compression), not only can error-controlled lossy compression significantly reduce the data size but it can also hold the promise to satisfy user demand on error control. Point-wise relative error bounds (i.e., compression errors depends on the data values) are widely used by many scientific applications in the lossy compression, since error control can adapt to the precision in the dataset automatically. Point-wise relative error bounded compression is complicated and time consuming. In this work, we develop efficient precomputation-based mechanisms in the SZ lossy compression framework. Our mechanisms can avoid costly logarithmic transformation and identify quantization factor values via a fast table lookup, greatly accelerating the relative-error bounded compression with excellent compression ratios. In addition, our mechanisms also help reduce traversing operations for Huffman decoding, and thus significantly accelerate the decompression process in SZ. Experiments with four well-known real-world scientific simulation datasets show that our solution can improve the compression rate by about 30% and decompression rate by about 70% in most of cases, making our designed lossy compression strategy the best choice in class in most cases.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 35th Symposium on Mass Storage Systems and Technologies, MSST 2019
PublisherIEEE Computer Society
Pages65-78
Number of pages14
ISBN (Electronic)9781728139203
DOIs
StatePublished - May 2019
Event35th Symposium on Mass Storage Systems and Technologies, MSST 2019 - Santa Clara, United States
Duration: May 20 2019May 24 2019

Publication series

NameIEEE Symposium on Mass Storage Systems and Technologies
Volume2019-May
ISSN (Print)2160-1968

Conference

Conference35th Symposium on Mass Storage Systems and Technologies, MSST 2019
Country/TerritoryUnited States
CitySanta Clara
Period5/20/195/24/19

Keywords

  • compression rate
  • high-performance computing
  • Lossy compression
  • scientific data

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Accelerating Relative-error Bounded Lossy Compression for HPC datasets with Precomputation-Based Mechanisms'. Together they form a unique fingerprint.

Cite this