Vesper: Learning to Manage Uncertainty in Video Streaming

Bo Chen, Mingyuan Wu, Hongpeng Guo, Zhisheng Yan, Klara Nahrstedt

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Video codecs are crucial in video streaming systems. However, the quantization operation in existing codecs introduces irreversible jitters. Moreover, the common practice of fitting a single codec to diverse video content lacks the flexibility to adapt the parameters of a codec for specific content. They lead to the problem of quantization and content uncertainty. Our preliminary study shows an ideal codec without uncertainty gains a significant advantage over the conventional codec with uncertainty. However, realizing the ideal codec presents tremendous challenges in the generalizability and the costs of computation, transmission, and delay. In this paper, we present Vesper, a video streaming system that innovatively tackles uncertainty with two learning-based components, super-precision and self-evolution. The super-precision module builds a neural network that predicts original feature values from quantized feature values, which effectively mitigates the impact of quantization without inducing generalizability issues. The self-evolution module performs content-aware adaptation on the encoder and replaces non-content-aware video segments with content-aware ones on the fly, which addresses content uncertainty without adding significant costs to on-demand streaming. Evaluations demonstrate Vesper's superior Quality of Experience compared to streaming systems built with state-of-the-art codecs.

Original languageEnglish (US)
Title of host publicationMMSys 2024 - Proceedings of the 2024 ACM Multimedia Systems Conference
PublisherAssociation for Computing Machinery
Number of pages12
ISBN (Electronic)9798400704123
StatePublished - Apr 15 2024
Event15th ACM Multimedia Systems Conference, MMSys 2024 - Bari, Italy
Duration: Apr 15 2024Apr 18 2024

Publication series

NameMMSys 2024 - Proceedings of the 2024 ACM Multimedia Systems Conference


Conference15th ACM Multimedia Systems Conference, MMSys 2024


  • Machine Learning
  • Video Compression
  • Video Streaming

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software


Dive into the research topics of 'Vesper: Learning to Manage Uncertainty in Video Streaming'. Together they form a unique fingerprint.

Cite this