TY - GEN
T1 - DtCraft
T2 - 36th IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2017
AU - Huang, Tsung Wei
AU - Lin, Chun Xun
AU - Wong, Martin D.F.
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/12/13
Y1 - 2017/12/13
N2 - Recent years have seen rapid growth in data-driven distributed systems such as Hadoop MapReduce, Spark, and Dryad. However, the counterparts for high-performance or compute-intensive applications including large-scale optimizations, modeling, and simulations are still nascent. In this paper, we introduce DtCraft, a modern C+,+,17-based distributed execution engine that efficiently supports a new powerful programming model for building high-performance parallel applications. Users need no understanding of distributed computing and can focus on high-level developments, leaving difficult details such as concurrency controls, workload distribution, and fault tolerance handled by our system transparently. We have evaluated DtCraft on both micro-benchmarks and large-scale optimization problems, and shown promising performance on computer clusters. In a particular semicondictor design problem, we achieved 30 x speedup with 40 nodes and 15 x less development efforts over hand-crafted implementation.
AB - Recent years have seen rapid growth in data-driven distributed systems such as Hadoop MapReduce, Spark, and Dryad. However, the counterparts for high-performance or compute-intensive applications including large-scale optimizations, modeling, and simulations are still nascent. In this paper, we introduce DtCraft, a modern C+,+,17-based distributed execution engine that efficiently supports a new powerful programming model for building high-performance parallel applications. Users need no understanding of distributed computing and can focus on high-level developments, leaving difficult details such as concurrency controls, workload distribution, and fault tolerance handled by our system transparently. We have evaluated DtCraft on both micro-benchmarks and large-scale optimization problems, and shown promising performance on computer clusters. In a particular semicondictor design problem, we achieved 30 x speedup with 40 nodes and 15 x less development efforts over hand-crafted implementation.
UR - http://www.scopus.com/inward/record.url?scp=85043521519&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85043521519&partnerID=8YFLogxK
U2 - 10.1109/ICCAD.2017.8203853
DO - 10.1109/ICCAD.2017.8203853
M3 - Conference contribution
AN - SCOPUS:85043521519
T3 - IEEE/ACM International Conference on Computer-Aided Design, Digest of Technical Papers, ICCAD
SP - 757
EP - 765
BT - 2017 IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2017
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 13 November 2017 through 16 November 2017
ER -