Fulfilling the promises of Lossy compression for scientific applications

Franck Cappello, Sheng Di, Ali Murat Gok

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Many scientific simulations, machine/deep learning applications and instruments are in need of significant data reduction. Errorbounded lossy compression has been identified as one solution and has been tested for many use-cases: Reducing streaming intensity (instruments), reducing storage and memory footprints, accelerating computation and accelerating data access and transfer. Ultimately, users’ trust in lossy compression relies on the preservation of science: same conclusions should be drawn from computations or analysis done from lossy compressed data. Experience from scientific simulations, Artificial Intelligence (AI) and instruments reveals several points: (i) there are important gaps in the understanding of the effects of lossy compressed data on computations, AI and analysis, (ii) each use-case, application and user has its own requirements in terms of compression ratio, speed and accuracy, and current generic monolithic compressors are not responding well to this need for specialization. This situation calls for more research and development on the lossy compression technologies. This paper addresses the most pressing research needs regarding the application of lossy compression in the scientific context.

Original languageEnglish (US)
Title of host publicationDriving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers
EditorsJeffrey Nichols, Arthur ‘Barney’ Maccabe, Suzanne Parete-Koon, Becky Verastegui, Oscar Hernandez, Theresa Ahearn
Number of pages18
ISBN (Print)9783030633929
StatePublished - 2021
Externally publishedYes
Event17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020 - Virtual, Online
Duration: Aug 26 2020Aug 28 2020

Publication series

NameCommunications in Computer and Information Science
Volume1315 CCIS
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937


Conference17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020
CityVirtual, Online


  • Lossy compression
  • Scientific data

ASJC Scopus subject areas

  • Computer Science(all)
  • Mathematics(all)


Dive into the research topics of 'Fulfilling the promises of Lossy compression for scientific applications'. Together they form a unique fingerprint.

Cite this