Semantic-Aware View Prediction for 360-Degree Videos at the 5G Edge

Shivi Vats, Jounsup Park, Klara Nahrstedt, Michael Zink, Ramesh Sitaraman, Hermann Hellwagner

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In a 5G testbed, we use 360° video streaming to test, measure, and demonstrate the 5G infrastructure, including the capabilities and challenges of edge computing support. Specifically, we use the SEAWARE (Semantic-Aware View Prediction) software system, originally described in [1], at the edge of the 5G network to support a 360° video player (handling tiled videos) by view prediction. Originally, SEAWARE performs semantic analysis of a 360° video on the media server, by extracting, e.g., important objects and events. This video semantic information is encoded in specific data structures and shared with the client in a DASH streaming framework. Making use of these data structures, the client/player can perform view prediction without in-depth, computationally expensive semantic video analysis. In this paper, the SEAWARE system was ported and adapted to run (partially) on the edge where it can be used to predict views and prefetch predicted segments/tiles in high quality in order to have them available close to the client when requested. The paper gives an overview of the 5G testbed, the overall architecture, and the implementation of SEAWARE at the edge server. Since an important goal of this work is to achieve low motion-to-glass latencies, we developed and describe "tile postloading", a technique that allows non-predicted tiles to be fetched in high quality into a segment already available in the player buffer. The performance of 360° tiled video playback on the 5G infrastructure is evaluated and presented. Current limitations of the 5G network in use and some challenges of DASH-based streaming and of edge-assisted viewport prediction under "real-world"constraints are pointed out; further, the performance benefits of tile postloading are disclosed.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 IEEE International Symposium on Multimedia, ISM 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages121-128
Number of pages8
ISBN (Electronic)9781665471725
DOIs
StatePublished - 2022
Event24th IEEE International Symposium on Multimedia, ISM 2022 - Virtual, Online, Italy
Duration: Dec 5 2022Dec 7 2022

Publication series

NameProceedings - 2022 IEEE International Symposium on Multimedia, ISM 2022

Conference

Conference24th IEEE International Symposium on Multimedia, ISM 2022
Country/TerritoryItaly
CityVirtual, Online
Period12/5/2212/7/22

Keywords

  • 5G networks
  • Tile-based 360° video streaming
  • edge computing
  • tile postloading
  • viewport prediction

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Signal Processing
  • Media Technology

Fingerprint

Dive into the research topics of 'Semantic-Aware View Prediction for 360-Degree Videos at the 5G Edge'. Together they form a unique fingerprint.

Cite this