Abstract

Mobile devices are gradually changing people's computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to desktops. Many applications with intensive graphics rendering workloads are unable to run on mobile platforms directly. This issue can be addressed with the idea of remote rendering: the heavy 3D graphics rendering computation runs on a powerful server and the rendering results are transmitted to the mobile client for display. However, the simple remote rendering solution inevitably suffers from the large interaction latency caused by wireless networks, and is not acceptable for many applications that have very strict latency requirements. In this article, we present an advanced low-latency remote rendering system that assists mobile devices to render interactive 3D graphics in real-time. Our design takes advantage of an image based rendering technique: 3D image warping, to synthesize the mobile display from the depth images generated on the server. The research indicates that the system can successfully reduce the interaction latency while maintaining the high rendering quality by generating multiple depth images at the carefully selected viewpoints. We study the problem of viewpoint selection, propose a real-time reference viewpoint prediction algorithm, and evaluate the algorithm performance with real-device experiments.

Original languageEnglish (US)
Article number46
JournalACM Transactions on Multimedia Computing, Communications and Applications
Volume8
Issue number3 SUPPL. 3
DOIs
StatePublished - Sep 1 2012

Fingerprint

Mobile devices
Servers
Display devices
Wireless networks
Electric power utilization
Experiments

Keywords

  • 3D image warping
  • Interaction latency
  • Mobile devices
  • Remote rendering

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

@article{e2e631bc6af44a53ac534909f4de5bb4,
title = "A real-time remote rendering system for interactive mobile graphics",
abstract = "Mobile devices are gradually changing people's computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to desktops. Many applications with intensive graphics rendering workloads are unable to run on mobile platforms directly. This issue can be addressed with the idea of remote rendering: the heavy 3D graphics rendering computation runs on a powerful server and the rendering results are transmitted to the mobile client for display. However, the simple remote rendering solution inevitably suffers from the large interaction latency caused by wireless networks, and is not acceptable for many applications that have very strict latency requirements. In this article, we present an advanced low-latency remote rendering system that assists mobile devices to render interactive 3D graphics in real-time. Our design takes advantage of an image based rendering technique: 3D image warping, to synthesize the mobile display from the depth images generated on the server. The research indicates that the system can successfully reduce the interaction latency while maintaining the high rendering quality by generating multiple depth images at the carefully selected viewpoints. We study the problem of viewpoint selection, propose a real-time reference viewpoint prediction algorithm, and evaluate the algorithm performance with real-device experiments.",
keywords = "3D image warping, Interaction latency, Mobile devices, Remote rendering",
author = "Shu Shi and Klara Nahrstedt and Roy Campbell",
year = "2012",
month = "9",
day = "1",
doi = "10.1145/2348816.2348825",
language = "English (US)",
volume = "8",
journal = "ACM Transactions on Multimedia Computing, Communications and Applications",
issn = "1551-6857",
publisher = "Association for Computing Machinery (ACM)",
number = "3 SUPPL. 3",

}

TY - JOUR

T1 - A real-time remote rendering system for interactive mobile graphics

AU - Shi, Shu

AU - Nahrstedt, Klara

AU - Campbell, Roy

PY - 2012/9/1

Y1 - 2012/9/1

N2 - Mobile devices are gradually changing people's computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to desktops. Many applications with intensive graphics rendering workloads are unable to run on mobile platforms directly. This issue can be addressed with the idea of remote rendering: the heavy 3D graphics rendering computation runs on a powerful server and the rendering results are transmitted to the mobile client for display. However, the simple remote rendering solution inevitably suffers from the large interaction latency caused by wireless networks, and is not acceptable for many applications that have very strict latency requirements. In this article, we present an advanced low-latency remote rendering system that assists mobile devices to render interactive 3D graphics in real-time. Our design takes advantage of an image based rendering technique: 3D image warping, to synthesize the mobile display from the depth images generated on the server. The research indicates that the system can successfully reduce the interaction latency while maintaining the high rendering quality by generating multiple depth images at the carefully selected viewpoints. We study the problem of viewpoint selection, propose a real-time reference viewpoint prediction algorithm, and evaluate the algorithm performance with real-device experiments.

AB - Mobile devices are gradually changing people's computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to desktops. Many applications with intensive graphics rendering workloads are unable to run on mobile platforms directly. This issue can be addressed with the idea of remote rendering: the heavy 3D graphics rendering computation runs on a powerful server and the rendering results are transmitted to the mobile client for display. However, the simple remote rendering solution inevitably suffers from the large interaction latency caused by wireless networks, and is not acceptable for many applications that have very strict latency requirements. In this article, we present an advanced low-latency remote rendering system that assists mobile devices to render interactive 3D graphics in real-time. Our design takes advantage of an image based rendering technique: 3D image warping, to synthesize the mobile display from the depth images generated on the server. The research indicates that the system can successfully reduce the interaction latency while maintaining the high rendering quality by generating multiple depth images at the carefully selected viewpoints. We study the problem of viewpoint selection, propose a real-time reference viewpoint prediction algorithm, and evaluate the algorithm performance with real-device experiments.

KW - 3D image warping

KW - Interaction latency

KW - Mobile devices

KW - Remote rendering

UR - http://www.scopus.com/inward/record.url?scp=84870232628&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84870232628&partnerID=8YFLogxK

U2 - 10.1145/2348816.2348825

DO - 10.1145/2348816.2348825

M3 - Article

AN - SCOPUS:84870232628

VL - 8

JO - ACM Transactions on Multimedia Computing, Communications and Applications

JF - ACM Transactions on Multimedia Computing, Communications and Applications

SN - 1551-6857

IS - 3 SUPPL. 3

M1 - 46

ER -