TY - JOUR
T1 - Automatic generation of benchmarks for I/O-intensive parallel applications
AU - Hao, Meng
AU - Zhang, Weizhe
AU - Zhang, You
AU - Snir, Marc
AU - Yang, Laurence T.
N1 - Funding Information:
Laurence T. Yang received the BE degree in computer science and technology from Tsinghua University, China and the PhD degree in computer science from the University of Victoria, Canada. He is a professor in the Department of Computer Science, St. Francis Xavier University, Canada. His research interests include parallel and distributed computing, embedded and ubiquitous computing, and big data. His research had been supported by the National Sciences and Engineering Research Council and Canada Foundation for Innovation.
Funding Information:
This work is supported by the National Key Research and Development Plan of China [grant number 2017YFB0202901 ]; the National Science Foundation of China (NSFC) [grant numbers 61672186 , 61472108 ].
Publisher Copyright:
© 2018 Elsevier Inc.
PY - 2019/2
Y1 - 2019/2
N2 - The benchmarks of I/O-intensive parallel applications are important for evaluating and optimizing HPC softwares and hardwares. However, extracting high-fidelity benchmarks which can fully reflect computation, communication, and I/O behaviors of original I/O-intensive parallel applications is very difficult. This work contributes a framework which can automatically generate benchmarks for I/O-intensive parallel applications. We demonstrate our framework on Taub and TianHe-2 supercomputers with five NAS Parallel Benchmarks (NPB) and four I/O-intensive parallel applications. The results show that our trace merging algorithm and trace compressing algorithm are better than others, and the generated benchmarks can accurately mimic the computation, communication, and I/O behaviors of original I/O-intensive parallel applications. Also, these generated benchmarks can be used to predict the performance of original applications, while reducing the prediction overhead by scaling down the execution time of benchmark proportionally.
AB - The benchmarks of I/O-intensive parallel applications are important for evaluating and optimizing HPC softwares and hardwares. However, extracting high-fidelity benchmarks which can fully reflect computation, communication, and I/O behaviors of original I/O-intensive parallel applications is very difficult. This work contributes a framework which can automatically generate benchmarks for I/O-intensive parallel applications. We demonstrate our framework on Taub and TianHe-2 supercomputers with five NAS Parallel Benchmarks (NPB) and four I/O-intensive parallel applications. The results show that our trace merging algorithm and trace compressing algorithm are better than others, and the generated benchmarks can accurately mimic the computation, communication, and I/O behaviors of original I/O-intensive parallel applications. Also, these generated benchmarks can be used to predict the performance of original applications, while reducing the prediction overhead by scaling down the execution time of benchmark proportionally.
KW - Benchmark generation
KW - I/O-intensive applications
KW - Performance prediction
KW - Trace compressing
KW - Trace merging
UR - http://www.scopus.com/inward/record.url?scp=85055754292&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85055754292&partnerID=8YFLogxK
U2 - 10.1016/j.jpdc.2018.10.004
DO - 10.1016/j.jpdc.2018.10.004
M3 - Article
AN - SCOPUS:85055754292
SN - 0743-7315
VL - 124
SP - 1
EP - 13
JO - Journal of Parallel and Distributed Computing
JF - Journal of Parallel and Distributed Computing
ER -