TY - GEN
T1 - Petascale xct
T2 - 2020 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020
AU - Hidayetoglu, Mert
AU - Bicer, Tekin
AU - De Gonzalo, Simon Garcia
AU - Ren, Bin
AU - De Andrade, Vincent
AU - Gursoy, Doga
AU - Kettimuthu, Raj
AU - Foster, Ian T.
AU - Hwu, Wen Mei W.
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/11
Y1 - 2020/11
N2 - X-ray computed tomography is a commonly used technique for noninvasive imaging at synchrotron facilities. Iterative tomographic reconstruction algorithms are often preferred for recovering high quality 3D volumetric images from 2D X-ray images, however, their use has been limited to small/medium datasets due to their computational requirements. In this paper, we propose a high-performance iterative reconstruction system for terabyte(s)-scale 3D volumes. Our design involves three novel optimizations: (1) optimization of (back)projection operators by extending the 2D memory-centric approach to 3D;(2) performing hierarchical communications by exploiting 'fat-node' architecture with many GPUs; 3) utilization of mixed-precision types while preserving convergence rate and quality. We extensively evaluate the proposed optimizations and scaling on the Summit supercomputer. Our largest reconstruction is a mouse brain volume with 9×11K×11K voxels, where the total reconstruction time is under three minutes using 24,576 GPUs, reaching 65 PFLOPS: 34% of Summit's peak performance.
AB - X-ray computed tomography is a commonly used technique for noninvasive imaging at synchrotron facilities. Iterative tomographic reconstruction algorithms are often preferred for recovering high quality 3D volumetric images from 2D X-ray images, however, their use has been limited to small/medium datasets due to their computational requirements. In this paper, we propose a high-performance iterative reconstruction system for terabyte(s)-scale 3D volumes. Our design involves three novel optimizations: (1) optimization of (back)projection operators by extending the 2D memory-centric approach to 3D;(2) performing hierarchical communications by exploiting 'fat-node' architecture with many GPUs; 3) utilization of mixed-precision types while preserving convergence rate and quality. We extensively evaluate the proposed optimizations and scaling on the Summit supercomputer. Our largest reconstruction is a mouse brain volume with 9×11K×11K voxels, where the total reconstruction time is under three minutes using 24,576 GPUs, reaching 65 PFLOPS: 34% of Summit's peak performance.
UR - http://www.scopus.com/inward/record.url?scp=85099383820&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85099383820&partnerID=8YFLogxK
U2 - 10.1109/SC41405.2020.00041
DO - 10.1109/SC41405.2020.00041
M3 - Conference contribution
AN - SCOPUS:85099383820
T3 - International Conference for High Performance Computing, Networking, Storage and Analysis, SC
BT - Proceedings of SC 2020
PB - IEEE Computer Society
Y2 - 9 November 2020 through 19 November 2020
ER -