Control considerations for scalable event processing

Wei Xu, Joseph L. Hellerstein, Bill Kramer, David Patterson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The growth in the scale of systems and networks has created many challenges for their management, especially for event processing. Our premise is that scaling event processing requires parallelism. To this end, we observe that event processing can be divided into intra-event processing such as filtering and inter-event processing such as root cause analysis. Since intra-event processing is easily parallelized, we propose an architecture in which intra-event processing elements (IAPs) are replicated to scale to larger event input rates. We address two challenges in this architecture. First, the IAPs are subject to overloads that require effective flow control, a capability that was not present in the components we used to build IAPs. Second, we need to balance the loads on IAPs to avoid creating resource bottlenecks. These challenges are further complicated by the presence of disturbances such as CPU intensive administrative tasks that reduce event processing rates. We address these challenges using designs based on control theory, a technique for analyzing stability, accuracy, and settling times. We demonstrate the effectiveness of our approaches with testbed experiments that include a disturbance in the form of a CPU intensive application.

Original languageEnglish (US)
Title of host publicationAmbient Networks - 16th IFIP/IEEE International Workshop on Distributed Systems
Subtitle of host publicationOperations and Management, DSOM 2005, Proceedings
EditorsJürgen Schönwälder, Joan Serrat
PublisherSpringer-Verlag Berlin Heidelberg
Pages233-244
Number of pages12
ISBN (Print)3540293884, 9783540293880
DOIs
StatePublished - 2005
Externally publishedYes
Event16th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management, DSOM 2005 - Ambient Networks - Barcelona, Spain
Duration: Oct 24 2005Oct 26 2005

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3775 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other16th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management, DSOM 2005 - Ambient Networks
CountrySpain
CityBarcelona
Period10/24/0510/26/05

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Control considerations for scalable event processing'. Together they form a unique fingerprint.

Cite this