Understanding the Effects of Modern Compressors on the Community Earth Science Model

Robert Underwood, Julie Bessac, Sheng Di, Franck Cappello

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The Community Earth Science Model (CESM) is an important tool in climate modeling that produces a large volume of data on each simulation. Researchers have increasingly been turning to both lossless and lossy compression as an approach to reduce the volume of data for the CESM climate applications. Choosing the best-qualified compressor is nontrivial, however, especially because of the advent of many modern lossless and lossy compressors and complicated scientific integrity assessment of climate data model. In this paper we evaluate 11 state-of-the-art compressors using the quality assessments developed by climate scientists to understand the effectiveness of the compressors on the CESM climate datasets with four different models. Our work also identifies the best compression ratio that can be reasonably achieved while meeting these strict quality requirements.

Original languageEnglish (US)
Title of host publicationProceedings of DRBSD-8 2022
Subtitle of host publication8th International Workshop on Data Analysis and Reduction for Big Scientific Data, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1-10
Number of pages10
ISBN (Electronic)9781665463379
DOIs
StatePublished - 2022
Externally publishedYes
Event8th IEEE/ACM International Workshop on Data Analysis and Reduction for Big Scientific Data, DRBSD-8 2022 - Dallas, United States
Duration: Nov 13 2022Nov 18 2022

Publication series

NameProceedings of DRBSD-8 2022: 8th International Workshop on Data Analysis and Reduction for Big Scientific Data, Held in conjunction with SC 2022: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference8th IEEE/ACM International Workshop on Data Analysis and Reduction for Big Scientific Data, DRBSD-8 2022
Country/TerritoryUnited States
CityDallas
Period11/13/2211/18/22

Keywords

  • CESM
  • MGARD
  • ZFP
  • lossless compression
  • lossy compression
  • sz

ASJC Scopus subject areas

  • Computer Science Applications
  • Information Systems
  • Earth and Planetary Sciences (miscellaneous)

Fingerprint

Dive into the research topics of 'Understanding the Effects of Modern Compressors on the Community Earth Science Model'. Together they form a unique fingerprint.

Cite this