Shared Memory-Aware Latency-Sensitive Message Aggregation for Fine-Grained Communication

Message aggregation is widely used with a goal to reduce communication cost in HPC applications. The difference in the order of overhead of sending a message and cost of per byte transferred motivates the need for message aggregation, for several irregular fine-grained messaging applications like graph algorithms and parallel discrete event simulation (PDES). While the benefit of message aggregation is often analyzed in terms of reducing the overhead, specifically the per message cost, we also analyze different schemes that can aid in reducing the message latency, i.e. the time from when a message is sent to the time when it is received. Message latency can affect several applications like PDES with speculative execution where reducing message latency could result in fewer rollbacks. Specifically in our work, we demonstrate the effectiveness of process-aware message aggregation schemes for a range of proxy applications with respect to messaging overhead and latency.