A hierarchical approach for load balancing on parallel multi-core systems

Laércio L. Pilla, Christiane Pousa Ribeiro, Daniel Cordeiro, Chao Mei, Abhinav Bhatele, Philippe O.A. Navaux, François Broquedis, Jean François Méhaut, Laxmikant V. Kale

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Multi-core compute nodes with non-uniform memory access (NUMA) are now a common architecture in the assembly of large-scale parallel machines. On these machines, in addition to the network communication costs, the memory access costs within a compute node are also asymmetric. Ignoring this can lead to an increase in the data movement costs. Therefore, to fully exploit the potential of these nodes and reduce data access costs, it becomes crucial to have a complete view of the machine topology (i.e. the compute node topology and the interconnection network among the nodes). Furthermore, the parallel application behavior has an important role in determining how to utilize the machine efficiently. In this paper, we propose a hierarchical load balancing approach to improve the performance of applications on parallel multi-core systems. We introduce NucoLB, a topology-aware load balancer that focuses on redistributing work while reducing communication costs among and within compute nodes. NucoLB takes the asymmetric memory access costs present on NUMA multi-core compute nodes, the interconnection network overheads, and the application communication patterns into account in its balancing decisions. We have implemented NucoLB using the Charm++ parallel runtime system and evaluated its performance. Results show that our load balancer improves performance up to 20% when compared to state-of-the-art load balancers on three different NUMA parallel machines.

Original languageEnglish (US)
Title of host publicationProceedings - 41st International Conference on Parallel Processing, ICPP 2012
Number of pages10
StatePublished - 2012
Event41st International Conference on Parallel Processing, ICPP 2012 - Pittsburgh, PA, United States
Duration: Sep 10 2012Sep 13 2012

Publication series

NameProceedings of the International Conference on Parallel Processing
ISSN (Print)0190-3918


Other41st International Conference on Parallel Processing, ICPP 2012
Country/TerritoryUnited States
CityPittsburgh, PA


  • cluster
  • load balancing
  • memory affinity
  • multi-core
  • non-uniform memory access
  • topology

ASJC Scopus subject areas

  • Software
  • Mathematics(all)
  • Hardware and Architecture


Dive into the research topics of 'A hierarchical approach for load balancing on parallel multi-core systems'. Together they form a unique fingerprint.

Cite this