Event-driven stitching for tile-based live 360 video streaming

Bo Chen, Zhisheng Yan, Haiming Jin, Klara Nahrstedt

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

360 video streaming is gaining popularity because of the new type of experience it creates. Tile-based approaches have been widely used in VoD 360 video streaming to save the network bandwidth. However, they cannot be extended to the case of live streaming because they assume the 360 videos stitched offline before streaming. Instead, stitching has to be done in real-time in live 360 video streaming. More importantly, the stitching speed as shown in our experiments is one order of magnitude lower than the network transmission speed, making stitching more of a deciding factor of the overall frame rate than the network transmission speed. In this paper, we design a stitching algorithm for tile-based live 360 video streaming that adapts stitching quality to make the best use of the timing budget. There are two main challenges. First, existing tile-based approaches do not consider various semantic information in different scenarios. Second, the decision of tiling schemes for tile-based stitching is non-trivial. To solve the above two challenges, we present an event-driven stitching algorithm for tile-based 360 video live streaming, which consists of such an event-driven model to abstract various semantic information as events and a tile actuator to make tiling scheme decisions. We implement a streaming system based on event-driven stitching called LiveTexture. To evaluate the proposed algorithm, we compare LiveTexture with other baseline systems and show that LiveTexture adapts well to various timing budgets by meeting 89.4% of the timing constraints. We also demonstrate that LiveTexture utilizes the timing budget more efficiently than others.

Original languageEnglish (US)
Title of host publicationProceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019
PublisherAssociation for Computing Machinery, Inc
Pages1-12
Number of pages12
ISBN (Electronic)9781450362979
DOIs
StatePublished - Jun 18 2019
Event10th ACM Multimedia Systems Conference, MMSys 2019 - Amherst, United States
Duration: Jun 18 2019Jun 21 2019

Publication series

NameProceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019

Conference

Conference10th ACM Multimedia Systems Conference, MMSys 2019
CountryUnited States
CityAmherst
Period6/18/196/21/19

Fingerprint

Video streaming
Tile
Electric power transmission networks
Semantics
Actuators
Bandwidth
Experiments

Keywords

  • 360 video
  • Live streaming
  • Tile-based stitching

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software

Cite this

Chen, B., Yan, Z., Jin, H., & Nahrstedt, K. (2019). Event-driven stitching for tile-based live 360 video streaming. In Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019 (pp. 1-12). (Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019). Association for Computing Machinery, Inc. https://doi.org/10.1145/3304109.3306234

Event-driven stitching for tile-based live 360 video streaming. / Chen, Bo; Yan, Zhisheng; Jin, Haiming; Nahrstedt, Klara.

Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019. Association for Computing Machinery, Inc, 2019. p. 1-12 (Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chen, B, Yan, Z, Jin, H & Nahrstedt, K 2019, Event-driven stitching for tile-based live 360 video streaming. in Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019. Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019, Association for Computing Machinery, Inc, pp. 1-12, 10th ACM Multimedia Systems Conference, MMSys 2019, Amherst, United States, 6/18/19. https://doi.org/10.1145/3304109.3306234
Chen B, Yan Z, Jin H, Nahrstedt K. Event-driven stitching for tile-based live 360 video streaming. In Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019. Association for Computing Machinery, Inc. 2019. p. 1-12. (Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019). https://doi.org/10.1145/3304109.3306234
Chen, Bo ; Yan, Zhisheng ; Jin, Haiming ; Nahrstedt, Klara. / Event-driven stitching for tile-based live 360 video streaming. Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019. Association for Computing Machinery, Inc, 2019. pp. 1-12 (Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019).
@inproceedings{210ed2d5e7284edd92b7f3a38cc109fc,
title = "Event-driven stitching for tile-based live 360 video streaming",
abstract = "360 video streaming is gaining popularity because of the new type of experience it creates. Tile-based approaches have been widely used in VoD 360 video streaming to save the network bandwidth. However, they cannot be extended to the case of live streaming because they assume the 360 videos stitched offline before streaming. Instead, stitching has to be done in real-time in live 360 video streaming. More importantly, the stitching speed as shown in our experiments is one order of magnitude lower than the network transmission speed, making stitching more of a deciding factor of the overall frame rate than the network transmission speed. In this paper, we design a stitching algorithm for tile-based live 360 video streaming that adapts stitching quality to make the best use of the timing budget. There are two main challenges. First, existing tile-based approaches do not consider various semantic information in different scenarios. Second, the decision of tiling schemes for tile-based stitching is non-trivial. To solve the above two challenges, we present an event-driven stitching algorithm for tile-based 360 video live streaming, which consists of such an event-driven model to abstract various semantic information as events and a tile actuator to make tiling scheme decisions. We implement a streaming system based on event-driven stitching called LiveTexture. To evaluate the proposed algorithm, we compare LiveTexture with other baseline systems and show that LiveTexture adapts well to various timing budgets by meeting 89.4{\%} of the timing constraints. We also demonstrate that LiveTexture utilizes the timing budget more efficiently than others.",
keywords = "360 video, Live streaming, Tile-based stitching",
author = "Bo Chen and Zhisheng Yan and Haiming Jin and Klara Nahrstedt",
year = "2019",
month = "6",
day = "18",
doi = "10.1145/3304109.3306234",
language = "English (US)",
series = "Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019",
publisher = "Association for Computing Machinery, Inc",
pages = "1--12",
booktitle = "Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019",

}

TY - GEN

T1 - Event-driven stitching for tile-based live 360 video streaming

AU - Chen, Bo

AU - Yan, Zhisheng

AU - Jin, Haiming

AU - Nahrstedt, Klara

PY - 2019/6/18

Y1 - 2019/6/18

N2 - 360 video streaming is gaining popularity because of the new type of experience it creates. Tile-based approaches have been widely used in VoD 360 video streaming to save the network bandwidth. However, they cannot be extended to the case of live streaming because they assume the 360 videos stitched offline before streaming. Instead, stitching has to be done in real-time in live 360 video streaming. More importantly, the stitching speed as shown in our experiments is one order of magnitude lower than the network transmission speed, making stitching more of a deciding factor of the overall frame rate than the network transmission speed. In this paper, we design a stitching algorithm for tile-based live 360 video streaming that adapts stitching quality to make the best use of the timing budget. There are two main challenges. First, existing tile-based approaches do not consider various semantic information in different scenarios. Second, the decision of tiling schemes for tile-based stitching is non-trivial. To solve the above two challenges, we present an event-driven stitching algorithm for tile-based 360 video live streaming, which consists of such an event-driven model to abstract various semantic information as events and a tile actuator to make tiling scheme decisions. We implement a streaming system based on event-driven stitching called LiveTexture. To evaluate the proposed algorithm, we compare LiveTexture with other baseline systems and show that LiveTexture adapts well to various timing budgets by meeting 89.4% of the timing constraints. We also demonstrate that LiveTexture utilizes the timing budget more efficiently than others.

AB - 360 video streaming is gaining popularity because of the new type of experience it creates. Tile-based approaches have been widely used in VoD 360 video streaming to save the network bandwidth. However, they cannot be extended to the case of live streaming because they assume the 360 videos stitched offline before streaming. Instead, stitching has to be done in real-time in live 360 video streaming. More importantly, the stitching speed as shown in our experiments is one order of magnitude lower than the network transmission speed, making stitching more of a deciding factor of the overall frame rate than the network transmission speed. In this paper, we design a stitching algorithm for tile-based live 360 video streaming that adapts stitching quality to make the best use of the timing budget. There are two main challenges. First, existing tile-based approaches do not consider various semantic information in different scenarios. Second, the decision of tiling schemes for tile-based stitching is non-trivial. To solve the above two challenges, we present an event-driven stitching algorithm for tile-based 360 video live streaming, which consists of such an event-driven model to abstract various semantic information as events and a tile actuator to make tiling scheme decisions. We implement a streaming system based on event-driven stitching called LiveTexture. To evaluate the proposed algorithm, we compare LiveTexture with other baseline systems and show that LiveTexture adapts well to various timing budgets by meeting 89.4% of the timing constraints. We also demonstrate that LiveTexture utilizes the timing budget more efficiently than others.

KW - 360 video

KW - Live streaming

KW - Tile-based stitching

UR - http://www.scopus.com/inward/record.url?scp=85069053609&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069053609&partnerID=8YFLogxK

U2 - 10.1145/3304109.3306234

DO - 10.1145/3304109.3306234

M3 - Conference contribution

AN - SCOPUS:85069053609

T3 - Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019

SP - 1

EP - 12

BT - Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019

PB - Association for Computing Machinery, Inc

ER -