Henge: Intent-driven multi-tenant stream processing

Faria Kalim, Le Xu, Sharanya Bathey, Richa Meherwal, Indranil Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We present Henge, a system to support intent-based multi-tenancy in modern distributed stream processing systems. Henge supports multi-tenancy as a first-class citizen: everyone in an organization can now submit their stream processing jobs to a single, shared, consolidated cluster. Secondly, Henge allows each job to specify its own intents (i.e., requirements) as a Service Level Objective (SLO) that captures latency and/or throughput needs. In such an intent-driven multi-tenant cluster, the Henge scheduler adapts continually to meet jobs’ respective SLOs in spite of limited cluster resources, and under dynamically varying workloads. SLOs are soft and are based on utility functions. Henge’s overall goal is to maximize the total system utility achieved by all jobs in the system. Henge is integrated into Apache Storm and we present experimental results using both production jobs from Yahoo! and real datasets from Twitter.

Original languageEnglish (US)
Title of host publicationSoCC 2018 - Proceedings of the 2018 ACM Symposium on Cloud Computing
PublisherAssociation for Computing Machinery
Number of pages14
ISBN (Electronic)9781450360111
StatePublished - Oct 11 2018
Event2018 ACM Symposium on Cloud Computing, SoCC 2018 - Carlsbad, United States
Duration: Oct 11 2018Oct 13 2018

Publication series

NameSoCC 2018 - Proceedings of the 2018 ACM Symposium on Cloud Computing


Other2018 ACM Symposium on Cloud Computing, SoCC 2018
Country/TerritoryUnited States


  • Intents
  • Multi-Tenancy
  • Resource Management
  • Service Level Objectives
  • Stream Processing

ASJC Scopus subject areas

  • Artificial Intelligence
  • Information Systems
  • Software


Dive into the research topics of 'Henge: Intent-driven multi-tenant stream processing'. Together they form a unique fingerprint.

Cite this