TY - GEN
T1 - SECRE
T2 - 30th Annual IEEE International Conference on High Performance Computing, Data, and Analytics, HiPC 2023
AU - Khan, Arham
AU - Di, Sheng
AU - Zhao, Kai
AU - Liu, Jinyang
AU - Chard, Kyle
AU - Foster, Ian
AU - Cappello, Franck
N1 - This research was supported by the Exascale Computing Project (ECP), Project Number: 17-SC-20-SC, a collaborative effort of two DOE organizations - the Office of Science and the National Nuclear Security Administration, responsible for the planning and preparation of a capable exascale ecosystem, including software, applications, hardware, advanced system engineering and early testbed platforms, to support the nation's exascale computing imperative. The material was supported by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research (ASCR), under contract DEAC02-06CH11357, and supported by the National Science Foundation under Grant OAC-2003709 and OAC-2104023. We acknowledge the computing resources provided on Bebop (operated by Laboratory Computing Resource Center at Argonne) and on Theta and JLSE (operated by Argonne Leadership Computing Facility).
PY - 2023
Y1 - 2023
N2 - Error-controlled lossy compression has been effective in reducing data storage/transfer costs while preserving reconstructed data fidelity based on user-defined error bounds. State-of-the-art error-controlled lossy compressors primarily fo-cus on error control rather than compression size, and thus, compression ratios are unknown until the compression operation is fully completed. Many use cases, however, require knowledge of compression ratios a priori, for example, pre-allocating appropri-ate memory for the compressed data at runtime. In this paper, we propose a novel, efficient Surrogate-based Error-controlled Lossy Compression Ratio Estimation Framework (SECRE), which includes three key features/contributions. (1) We carefully design the SECRE framework, which, in principle, can be applied to different error-bounded lossy compressors. (2) We implement a compression ratio estimation method for four state-of-the-art error-controlled lossy compressors-SZx, SZ3, ZFP, and SPERR-by devising a corresponding lightweight compression surrogate for each. (3) We evaluate the performance and accuracy of SECRE using four real-world scientific simulation datasets. Experiments show that SECREcan obtain highly accurate com-pression ratio estimates (e.g., 1 % estimation errors for SZx) with low execution overhead (e.g., 2 % estimation cost for SZx).
AB - Error-controlled lossy compression has been effective in reducing data storage/transfer costs while preserving reconstructed data fidelity based on user-defined error bounds. State-of-the-art error-controlled lossy compressors primarily fo-cus on error control rather than compression size, and thus, compression ratios are unknown until the compression operation is fully completed. Many use cases, however, require knowledge of compression ratios a priori, for example, pre-allocating appropri-ate memory for the compressed data at runtime. In this paper, we propose a novel, efficient Surrogate-based Error-controlled Lossy Compression Ratio Estimation Framework (SECRE), which includes three key features/contributions. (1) We carefully design the SECRE framework, which, in principle, can be applied to different error-bounded lossy compressors. (2) We implement a compression ratio estimation method for four state-of-the-art error-controlled lossy compressors-SZx, SZ3, ZFP, and SPERR-by devising a corresponding lightweight compression surrogate for each. (3) We evaluate the performance and accuracy of SECRE using four real-world scientific simulation datasets. Experiments show that SECREcan obtain highly accurate com-pression ratio estimates (e.g., 1 % estimation errors for SZx) with low execution overhead (e.g., 2 % estimation cost for SZx).
KW - compression ratio estimation
KW - error-controlled lossy compression
KW - sampling
KW - scientific datasets
UR - http://www.scopus.com/inward/record.url?scp=85178123400&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85178123400&partnerID=8YFLogxK
U2 - 10.1109/HiPC58850.2023.00029
DO - 10.1109/HiPC58850.2023.00029
M3 - Conference contribution
AN - SCOPUS:85178123400
T3 - Proceedings - 2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics, HiPC 2023
SP - 132
EP - 142
BT - Proceedings - 2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics, HiPC 2023
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 18 December 2023 through 21 December 2023
ER -