Improving floating point compression through binary masks

Leonardo A.Bautista Gomez, Franck Cappello

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Modern scientific technology such as particle accelerators, telescopes, and supercomputers are producing extremely large amounts of data. That scientific data needs to be processed by using systems with high computational capabilities such as supercomputers. Given that the scientific data is increasing in size at an exponential rate, storing and accessing the data are becoming expensive in both time and space. Most of this scientific data is stored by using floating point representation. Scientific applications executed on supercomputers spend a large amount of CPU cycles reading and writing floating point values, making data compression techniques an interesting way to increase computing efficiency. Given the accuracy requirements of scientific computing, we only focus on lossless data compression. In this paper we propose a masking technique that partially decreases the entropy of scientific datasets, allowing for a better compression ratio and higher throughput. We evaluate several data partitioning techniques for selective compression and compare these schemes with several existing compression strategies. Our approach shows up to 15% improvement in compression ratio while reducing the time spent in compression by half time in some cases.

Original languageEnglish (US)
Title of host publicationProceedings - 2013 IEEE International Conference on Big Data, Big Data 2013
Pages326-331
Number of pages6
DOIs
StatePublished - 2013
Externally publishedYes
Event2013 IEEE International Conference on Big Data, Big Data 2013 - Santa Clara, CA, United States
Duration: Oct 6 2013Oct 9 2013

Publication series

NameProceedings - 2013 IEEE International Conference on Big Data, Big Data 2013

Other

Other2013 IEEE International Conference on Big Data, Big Data 2013
Country/TerritoryUnited States
CitySanta Clara, CA
Period10/6/1310/9/13

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Improving floating point compression through binary masks'. Together they form a unique fingerprint.

Cite this