TY - GEN
T1 - Exploration of pattern-matching techniques for lossy compression on cosmology simulation data sets
AU - Tao, Dingwen
AU - Di, Sheng
AU - Chen, Zizhong
AU - Cappello, Franck
N1 - Publisher Copyright:
© Springer International Publishing AG 2017.
PY - 2017
Y1 - 2017
N2 - Because of the vast volume of data being produced by today’s scientific simulations, lossy compression allowing user-controlled information loss can significantly reduce the data size and the I/O burden. However, for large-scale cosmology simulation, such as the Hardware/Hybrid Accelerated Cosmology Code (HACC), where memory overhead constraints restrict compression to only one snapshot at a time, the lossy compression ratio is extremely limited because of the fairly low spatial coherence and high irregularity of the data. In this work, we propose a pattern-matching (similarity searching) technique to optimize the prediction accuracy and compression ratio of SZ lossy compressor on the HACC data sets. We evaluate our proposed method with different configurations and compare it with state-of-the-art lossy compressors. Experiments show that our proposed optimization approach can improve the prediction accuracy and reduce the compressed size of quantization codes compared with SZ. We present several lessons useful for future research involving pattern-matching techniques for lossy compression.
AB - Because of the vast volume of data being produced by today’s scientific simulations, lossy compression allowing user-controlled information loss can significantly reduce the data size and the I/O burden. However, for large-scale cosmology simulation, such as the Hardware/Hybrid Accelerated Cosmology Code (HACC), where memory overhead constraints restrict compression to only one snapshot at a time, the lossy compression ratio is extremely limited because of the fairly low spatial coherence and high irregularity of the data. In this work, we propose a pattern-matching (similarity searching) technique to optimize the prediction accuracy and compression ratio of SZ lossy compressor on the HACC data sets. We evaluate our proposed method with different configurations and compare it with state-of-the-art lossy compressors. Experiments show that our proposed optimization approach can improve the prediction accuracy and reduce the compressed size of quantization codes compared with SZ. We present several lessons useful for future research involving pattern-matching techniques for lossy compression.
UR - http://www.scopus.com/inward/record.url?scp=85032686508&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85032686508&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-67630-2_4
DO - 10.1007/978-3-319-67630-2_4
M3 - Conference contribution
AN - SCOPUS:85032686508
SN - 9783319676296
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 43
EP - 54
BT - High Performance Computing - ISC High Performance 2017 International Workshops, DRBSD, ExaComm, HCPM, HPC-IODC, IWOPH, IXPUG, P^3MA, VHPC, Visualization at Scale, WOPSSS, Revised Selected Papers
A2 - Yokota, Rio
A2 - Kunkel, Julian M.
A2 - Taufer, Michela
A2 - Shalf, John
PB - Springer
T2 - 32nd International Conference on High Performance Computing, ISC High Performance 2017
Y2 - 18 June 2017 through 22 June 2017
ER -