Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web services

Yi Lu, Qiaomin Xie, Gabriel Kliot, Alan Geller, James R. Larus, Albert Greenberg

Research output: Contribution to journalArticlepeer-review


The prevalence of dynamic-content web services, exemplified by search and online social networking, has motivated an increasingly wide web-facing front end. Horizontal scaling in the Cloud is favored for its elasticity, and distributed design of load balancers is highly desirable. Existing algorithms with a centralized design, such as Join-the-Shortest-Queue (JSQ), incur high communication overhead for distributed dispatchers. We propose a novel class of algorithms called Join-Idle-Queue (JIQ) for distributed load balancing in large systems. Unlike algorithms such as Power-of-Two, the JIQ algorithm incurs no communication overhead between the dispatchers and processors at job arrivals. We analyze the JIQ algorithm in the large system limit and find that it effectively results in a reduced system load, which produces 30-fold reduction in queueing overhead compared to Power-of-Two at medium to high load. An extension of the basic JIQ algorithm deals with very high loads using only local information of server load.

Original languageEnglish (US)
Pages (from-to)1056-1071
Number of pages16
JournalPerformance Evaluation
Issue number11
StatePublished - Nov 2011


  • Cloud computing
  • Load balancing
  • Queueing analysis
  • Randomized algorithm

ASJC Scopus subject areas

  • Software
  • Modeling and Simulation
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web services'. Together they form a unique fingerprint.

Cite this