Sieve: Semantically encoded video analytics on edge and cloud

Tarek Elgamal, Shu Shi, Varun Gupta, Rittwik Jana, Klara Nahrstedt

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Recent advances in computer vision and neural networks have made it possible for more surveillance videos to be automatically searched and analyzed by algorithms rather than humans. This happened in parallel with advances in edge computing where videos are analyzed over hierarchical clusters that contain edge devices, close to the video source. However, the current video analysis pipeline has several disadvantages when dealing with such advances. For example, video encoders have been designed for a long time to please human viewers and be agnostic of the downstream analysis task (e.g., object detection). Moreover, most of the video analytics systems leverage 2-tier architecture where the encoded video is sent to either a remote cloud or a private edge server but does not efficiently leverage both of them. In response to these advances, we present SIEVE, a 3-tier video analytics system to reduce the latency and increase the throughput of analytics over video streams. In SIEVE, we present a novel technique to detect objects in compressed video streams. We refer to this technique as semantic video encoding because it allows video encoders to be aware of the semantics of the downstream task (e.g., object detection). Our results show that by leveraging semantic video encoding, we achieve close to 100% object detection accuracy with decompressing only 3.5% of the video frames which results in more than 100x speedup compared to classical approaches that decompress every video frame.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE 40th International Conference on Distributed Computing Systems, ICDCS 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781728170022
StatePublished - Nov 2020
Event40th IEEE International Conference on Distributed Computing Systems, ICDCS 2020 - Singapore, Singapore
Duration: Nov 29 2020Dec 1 2020

Publication series

NameProceedings - International Conference on Distributed Computing Systems


Conference40th IEEE International Conference on Distributed Computing Systems, ICDCS 2020

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Sieve: Semantically encoded video analytics on edge and cloud'. Together they form a unique fingerprint.

Cite this