Automatic generation of benchmarks for I/O-intensive parallel applications

Meng Hao, Weizhe Zhang, You Zhang, Marc Snir, Laurence T. Yang

Research output: Contribution to journalArticlepeer-review

Abstract

The benchmarks of I/O-intensive parallel applications are important for evaluating and optimizing HPC softwares and hardwares. However, extracting high-fidelity benchmarks which can fully reflect computation, communication, and I/O behaviors of original I/O-intensive parallel applications is very difficult. This work contributes a framework which can automatically generate benchmarks for I/O-intensive parallel applications. We demonstrate our framework on Taub and TianHe-2 supercomputers with five NAS Parallel Benchmarks (NPB) and four I/O-intensive parallel applications. The results show that our trace merging algorithm and trace compressing algorithm are better than others, and the generated benchmarks can accurately mimic the computation, communication, and I/O behaviors of original I/O-intensive parallel applications. Also, these generated benchmarks can be used to predict the performance of original applications, while reducing the prediction overhead by scaling down the execution time of benchmark proportionally.

Original languageEnglish (US)
Pages (from-to)1-13
Number of pages13
JournalJournal of Parallel and Distributed Computing
Volume124
DOIs
StatePublished - Feb 2019

Keywords

  • Benchmark generation
  • I/O-intensive applications
  • Performance prediction
  • Trace compressing
  • Trace merging

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Automatic generation of benchmarks for I/O-intensive parallel applications'. Together they form a unique fingerprint.

Cite this