SEAWARE: Semantic Aware View Prediction System for 360-degree Video Streaming

Jounsup Park, Mingyuan Wu, Kuan Ying Lee, Bo Chen, Klara Nahrstedt, Michael Zink, Ramesh Sitaraman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Future view prediction for a 360-degree video streaming system is important to save the network bandwidth and improve the Quality of Experience (QoE). Historical view data of a single viewer and multiple viewers have been used for future view prediction. Video semantic information is also useful to predict the viewer's future behavior. However, extracting video semantic information requires powerful computing hardware and large memory space to perform deep learning-based video analysis. It is not a desirable condition for most of client devices, such as small mobile devices or Head Mounted Display (HMD). Therefore, we develop an approach where video semantic analysis is executed on the media server, and the analysis results are shared with clients via the Semantic Flow Descriptor (SFD) and View-Object State Machine (VOSM). SFD and VOSM become new descriptive additions of the Media Presentation Description (MPD) and Spatial Relation Description (SRD) to support 360-degree video streaming. Using the semantic-based approach, we design the Semantic-Aware View Prediction System (SEAWARE) to improve the overall view prediction performance. The evaluation results of 360-degree videos and real HMD view traces show that the SEAWARE system improves the view prediction performance and streams high-quality video with limited network bandwidth.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 IEEE International Symposium on Multimedia, ISM 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages57-64
Number of pages8
ISBN (Electronic)9781728186979
DOIs
StatePublished - Dec 2020
Event22nd IEEE International Symposium on Multimedia, ISM 2020 - Virtual, Naples, Italy
Duration: Dec 2 2020Dec 4 2020

Publication series

NameProceedings - 2020 IEEE International Symposium on Multimedia, ISM 2020

Conference

Conference22nd IEEE International Symposium on Multimedia, ISM 2020
Country/TerritoryItaly
CityVirtual, Naples
Period12/2/2012/4/20

Keywords

  • 360 degree video
  • semantic
  • streaming
  • view prediction

ASJC Scopus subject areas

  • Media Technology

Fingerprint

Dive into the research topics of 'SEAWARE: Semantic Aware View Prediction System for 360-degree Video Streaming'. Together they form a unique fingerprint.

Cite this