TY - GEN
T1 - Scalability Challenges of an Industrial Implicit Finite Element Code
AU - Rouet, Francois Henry
AU - Ashcraft, Cleve
AU - Dawson, Jef
AU - Grimes, Roger
AU - Guleryuz, Erman
AU - Koric, Seid
AU - Lucas, Robert F.
AU - Ong, James S.
AU - Simons, Todd A.
AU - Zhu, Ting Ting
N1 - Funding Information:
ACKNOWLEDGMENT This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725. The authors would like to thank the NCSA Industry Program and the Blue Waters sustained-petascale computing project at the National Center for Supercomputing Applications (NCSA). Blue Waters is supported by the National Science Foundation (award numbers OCI 07-25070 and ACI-1238993) and the state of Illinois.
Publisher Copyright:
© 2020 IEEE.
PY - 2020/5
Y1 - 2020/5
N2 - LS-DYNA is a well-known multiphysics code with both explicit and implicit time stepping capabilities. Implicit simulations rely heavily on sparse matrix computations, in particular direct solvers, and are notoriously much harder to scale than explicit simulations. In this paper, we investigate the scalability challenges of the implicit structural mode of LS-DYNA. In particular, we focus on linear constraint analysis, sparse matrix reordering, symbolic factorization, and numerical factorization. Our problem of choice for this study is a thermomechanical simulation of jet engine models built by Rolls-Royce with up to 200 million degrees of freedom, or equations. The models are used for engine performance analysis and design optimization, in particular optimization of tip clearances in the compressor and turbine sections of the engine. We present results using as many as 131,072 cores on the Blue Waters Cray XE6/XK7 supercomputer at NCSA and the Titan Cray XK7 supercomputer at OLCF. Since the main focus is on general linear algebra problems, this work is of interest for all linear algebra practitioners, not only developers of implicit finite element codes.
AB - LS-DYNA is a well-known multiphysics code with both explicit and implicit time stepping capabilities. Implicit simulations rely heavily on sparse matrix computations, in particular direct solvers, and are notoriously much harder to scale than explicit simulations. In this paper, we investigate the scalability challenges of the implicit structural mode of LS-DYNA. In particular, we focus on linear constraint analysis, sparse matrix reordering, symbolic factorization, and numerical factorization. Our problem of choice for this study is a thermomechanical simulation of jet engine models built by Rolls-Royce with up to 200 million degrees of freedom, or equations. The models are used for engine performance analysis and design optimization, in particular optimization of tip clearances in the compressor and turbine sections of the engine. We present results using as many as 131,072 cores on the Blue Waters Cray XE6/XK7 supercomputer at NCSA and the Titan Cray XK7 supercomputer at OLCF. Since the main focus is on general linear algebra problems, this work is of interest for all linear algebra practitioners, not only developers of implicit finite element codes.
KW - Direct Solvers
KW - Finite Elements
KW - Graph Partitioning
KW - Sparse Matrices
UR - http://www.scopus.com/inward/record.url?scp=85088898103&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85088898103&partnerID=8YFLogxK
U2 - 10.1109/IPDPS47924.2020.00059
DO - 10.1109/IPDPS47924.2020.00059
M3 - Conference contribution
AN - SCOPUS:85088898103
T3 - Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020
SP - 505
EP - 514
BT - Proceedings - 2020 IEEE 34th International Parallel and Distributed Processing Symposium, IPDPS 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 34th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2020
Y2 - 18 May 2020 through 22 May 2020
ER -