TY - GEN
T1 - An evaluative study on the effect of contention on message latencies in large supercomputers
AU - Bhatelé, Abhinav
AU - Kalé, Laxmikant V.
N1 - Copyright:
Copyright 2009 Elsevier B.V., All rights reserved.
PY - 2009
Y1 - 2009
N2 - Significant theoretical research was done on interconnect topologies and topology aware mapping for parallel computers in the 80s. With the deployment of virtual cutthrough, wormhole routing and faster interconnects, message latencies reduced and research in the area died down. This paper presents a study showing that with the emergence of very large supercomputers, typically connected as a 3D torus or mesh, topology effects have become important again. It presents an evaluative study on the effect of contention on message latencies on torus and mesh networks. The paper uses three MPI benchmarks to evaluate the effect of hops (links) traversed by messages, on their latencies. The benchmarks demonstrate that when multiple messages compete for network resources, link occupancy or contention can increase message latencies by up to a factor of 8 times. In other words, contention leads to increased message latencies and reduces effective available bandwidth for each message. This suggests that application developers should consider interconnect topologies when mapping tasks to processors in order to obtain the best performance. Results are shown for two parallel machines - ANL's Blue Gene/P and PSC's XT3.
AB - Significant theoretical research was done on interconnect topologies and topology aware mapping for parallel computers in the 80s. With the deployment of virtual cutthrough, wormhole routing and faster interconnects, message latencies reduced and research in the area died down. This paper presents a study showing that with the emergence of very large supercomputers, typically connected as a 3D torus or mesh, topology effects have become important again. It presents an evaluative study on the effect of contention on message latencies on torus and mesh networks. The paper uses three MPI benchmarks to evaluate the effect of hops (links) traversed by messages, on their latencies. The benchmarks demonstrate that when multiple messages compete for network resources, link occupancy or contention can increase message latencies by up to a factor of 8 times. In other words, contention leads to increased message latencies and reduces effective available bandwidth for each message. This suggests that application developers should consider interconnect topologies when mapping tasks to processors in order to obtain the best performance. Results are shown for two parallel machines - ANL's Blue Gene/P and PSC's XT3.
UR - http://www.scopus.com/inward/record.url?scp=70450091064&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=70450091064&partnerID=8YFLogxK
U2 - 10.1109/IPDPS.2009.5161094
DO - 10.1109/IPDPS.2009.5161094
M3 - Conference contribution
AN - SCOPUS:70450091064
SN - 9781424437504
T3 - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium
BT - IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium
T2 - 23rd IEEE International Parallel and Distributed Processing Symposium, IPDPS 2009
Y2 - 23 May 2009 through 29 May 2009
ER -