Using graphics rendering contexts to enhance the real-time video coding for mobile cloud gaming

Shu Shi, Cheng Hsin Hsu, Klara Nahrstedt, Roy H. Campbell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The emerging cloud gaming service has been growing rapidly, but not yet able to reach mobile customers due to many limitations, such as bandwidth and latency. We introduce a 3D image warping assisted real-time video coding method that can potentially meet all the requirements of mobile cloud gaming. The proposed video encoder selects a set of key frames in the video sequence, uses the 3D image warping algorithm to interpolate other non-key frames, and encodes the key frames and the residues frames with an H.264/AVC encoder. Our approach is novel in taking advantage of the run-time graphics rendering contexts (rendering viewpoint, pixel depth, camera motion, etc.) from the 3D game engine to enhance the performance of video encoding for the cloud gaming service. The experiments indicate that our proposed video encoder has the potential to beat the state-of-art x264 encoder in the scenario of real-time cloud gaming. For example, by implementing the proposed method in a 3D tank battle game, we experimentally show that more than 2 dB quality improvement is possible.

Original languageEnglish (US)
Title of host publicationMM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops
Pages103-112
Number of pages10
DOIs
StatePublished - 2011
Event19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11 - Scottsdale, AZ, United States
Duration: Nov 28 2011Dec 1 2011

Other

Other19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11
CountryUnited States
CityScottsdale, AZ
Period11/28/1112/1/11

    Fingerprint

Keywords

  • 3D image warping
  • Cloud gaming
  • Mobile devices
  • Real-time video coding

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction

Cite this

Shi, S., Hsu, C. H., Nahrstedt, K., & Campbell, R. H. (2011). Using graphics rendering contexts to enhance the real-time video coding for mobile cloud gaming. In MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops (pp. 103-112) https://doi.org/10.1145/2072298.2072313