Weighted locality-sensitive scheduling for mitigating noise on multi-core clusters

Vivek Kale, Abhinav Bhatele, William D. Gropp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent studies have shown that operating system (OS) interference, popularly called OS noise can be a significant problem as we scale to a large number of processors. One solution for mitigating noise is to turn off certain OS services on the machine. However, this is typically infeasible because full-scale OS services may be required for some applications. Furthermore, it is not a choice that an end user can make. Thus, we need an application-level solution. Building upon previous work that demonstrated the utility of within-node light-weight load balancing, we discuss the technique of weighted micro-scheduling and provide insights based on experimentation for two different machines with very different noise signatures. Through careful enumeration of the search space of scheduler parameters, we allow our weighted micro-scheduler to be dynamic, adaptive and tunable for a specific application running on a specific architecture. By doing this, we show how we can enable running scientific applications efficiently on a very large number of processors, even in the presence of noise.

Original languageEnglish (US)
Title of host publication18th International Conference on High Performance Computing, HiPC 2011
DOIs
StatePublished - 2011
Event18th International Conference on High Performance Computing, HiPC 2011 - Bangalore, India
Duration: Dec 18 2011Dec 21 2011

Publication series

Name18th International Conference on High Performance Computing, HiPC 2011

Other

Other18th International Conference on High Performance Computing, HiPC 2011
CountryIndia
CityBangalore
Period12/18/1112/21/11

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'Weighted locality-sensitive scheduling for mitigating noise on multi-core clusters'. Together they form a unique fingerprint.

Cite this