Holoportation: Virtual 3D Teleportation in Real-time

Sergio Orts-Escolano; David Kim; Qin Cai; Christoph Rhemann; Philip Davidson; Philip Chou; Sean Fanello; Sameh Khamis; Sarah Mennicken; Wayne Chang; Mingsong Dou; Julien Valentin; Adarsh Kowdle; Vladimir Tankovich; Vivek Pradeep; Yury Degtyarev; Charles Loop; Shenlong Wang; Sing Bing Kang; Pushmeet Kohli; Yuliya Lutchyn; Cem Keskin; Shahram Izadi

doi:10.1145/2984511.2984517

Holoportation: Virtual 3D Teleportation in Real-time

Sergio Orts-Escolano, David Kim, Qin Cai, Christoph Rhemann, Philip Davidson, Philip Chou, Sean Fanello, Sameh Khamis, Sarah Mennicken, Wayne Chang, Mingsong Dou, Julien Valentin, Adarsh Kowdle, Vladimir Tankovich, Vivek Pradeep, Yury Degtyarev, Charles Loop, Shenlong Wang, Sing Bing Kang, Pushmeet KohliYuliya Lutchyn, Cem Keskin, Shahram Izadi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.

Original language	English (US)
Title of host publication	UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology
Publisher	Association for Computing Machinery
Pages	741-754
Number of pages	14
ISBN (Electronic)	9781450345316
DOIs	https://doi.org/10.1145/2984511.2984517
State	Published - Oct 16 2016
Externally published	Yes
Event	29th Annual Symposium on User Interface Software and Technology, UIST 2016 - Tokyo, Japan Duration: Oct 16 2016 → Oct 19 2016

Publication series

Name	UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology

Other

Other	29th Annual Symposium on User Interface Software and Technology, UIST 2016
Country/Territory	Japan
City	Tokyo
Period	10/16/16 → 10/19/16

Keywords

3d capture
Depth cameras
GPU
Mixed reality
Non-rigid reconstruction
Real-time
Telepresence

ASJC Scopus subject areas

Human-Computer Interaction
Software

Online availability

10.1145/2984511.2984517

Library availability

Discover UIUC Full Text

Cite this

Orts-Escolano, S., Kim, D., Cai, Q., Rhemann, C., Davidson, P., Chou, P., Fanello, S., Khamis, S., Mennicken, S., Chang, W., Dou, M., Valentin, J., Kowdle, A., Tankovich, V., Pradeep, V., Degtyarev, Y., Loop, C., Wang, S., Kang, S. B., ... Izadi, S. (2016). Holoportation: Virtual 3D Teleportation in Real-time. In UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 741-754). (UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology). Association for Computing Machinery. https://doi.org/10.1145/2984511.2984517

Holoportation: Virtual 3D Teleportation in Real-time. / Orts-Escolano, Sergio; Kim, David; Cai, Qin et al.
UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology. Association for Computing Machinery, 2016. p. 741-754 (UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Orts-Escolano, S, Kim, D, Cai, Q, Rhemann, C, Davidson, P, Chou, P, Fanello, S, Khamis, S, Mennicken, S, Chang, W, Dou, M, Valentin, J, Kowdle, A, Tankovich, V, Pradeep, V, Degtyarev, Y, Loop, C, Wang, S, Kang, SB, Kohli, P, Lutchyn, Y, Keskin, C & Izadi, S 2016, Holoportation: Virtual 3D Teleportation in Real-time. in UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology. UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Association for Computing Machinery, pp. 741-754, 29th Annual Symposium on User Interface Software and Technology, UIST 2016, Tokyo, Japan, 10/16/16. https://doi.org/10.1145/2984511.2984517

Orts-Escolano S, Kim D, Cai Q, Rhemann C, Davidson P, Chou P et al. Holoportation: Virtual 3D Teleportation in Real-time. In UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology. Association for Computing Machinery. 2016. p. 741-754. (UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology). doi: 10.1145/2984511.2984517

@inproceedings{6fef587ce1194f02850b90c510ed8451,

title = "Holoportation: Virtual 3D Teleportation in Real-time",

abstract = "We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.",

keywords = "3d capture, Depth cameras, GPU, Mixed reality, Non-rigid reconstruction, Real-time, Telepresence",

author = "Sergio Orts-Escolano and David Kim and Qin Cai and Christoph Rhemann and Philip Davidson and Philip Chou and Sean Fanello and Sameh Khamis and Sarah Mennicken and Wayne Chang and Mingsong Dou and Julien Valentin and Adarsh Kowdle and Vladimir Tankovich and Vivek Pradeep and Yury Degtyarev and Charles Loop and Shenlong Wang and Kang, {Sing Bing} and Pushmeet Kohli and Yuliya Lutchyn and Cem Keskin and Shahram Izadi",

year = "2016",

month = oct,

day = "16",

doi = "10.1145/2984511.2984517",

language = "English (US)",

series = "UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology",

publisher = "Association for Computing Machinery",

pages = "741--754",

booktitle = "UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology",

address = "United States",

note = "29th Annual Symposium on User Interface Software and Technology, UIST 2016 ; Conference date: 16-10-2016 Through 19-10-2016",

}

TY - GEN

T1 - Holoportation

T2 - 29th Annual Symposium on User Interface Software and Technology, UIST 2016

AU - Orts-Escolano, Sergio

AU - Kim, David

AU - Cai, Qin

AU - Rhemann, Christoph

AU - Davidson, Philip

AU - Chou, Philip

AU - Fanello, Sean

AU - Khamis, Sameh

AU - Mennicken, Sarah

AU - Chang, Wayne

AU - Dou, Mingsong

AU - Valentin, Julien

AU - Kowdle, Adarsh

AU - Tankovich, Vladimir

AU - Pradeep, Vivek

AU - Degtyarev, Yury

AU - Loop, Charles

AU - Wang, Shenlong

AU - Kang, Sing Bing

AU - Kohli, Pushmeet

AU - Lutchyn, Yuliya

AU - Keskin, Cem

AU - Izadi, Shahram

PY - 2016/10/16

Y1 - 2016/10/16

N2 - We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.

AB - We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.

KW - 3d capture

KW - Depth cameras

KW - GPU

KW - Mixed reality

KW - Non-rigid reconstruction

KW - Real-time

KW - Telepresence

UR - http://www.scopus.com/inward/record.url?scp=84995777898&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84995777898&partnerID=8YFLogxK

U2 - 10.1145/2984511.2984517

DO - 10.1145/2984511.2984517

M3 - Conference contribution

AN - SCOPUS:84995777898

T3 - UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology

SP - 741

EP - 754

BT - UIST 2016 - Proceedings of the 29th Annual Symposium on User Interface Software and Technology

PB - Association for Computing Machinery

Y2 - 16 October 2016 through 19 October 2016

ER -

Holoportation: Virtual 3D Teleportation in Real-time

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this