Teleimmersive audio-visual communication using commodity hardware

Viet Anh Nguyen, Jiangbo Lu, Shengkui Zhao, Douglas L Jones, Minh N Do

Research output: Contribution to journalArticle

Abstract

Natural human communication involves complex visual and audio behavior, and often context and joint interaction with the surrounding environment, to create a rich and satisfying experience. However, widely used virtual meeting systems such as WebEx and Skype still provide rather limited functionalities and hardly maintain the experience of an in-person meeting. In particular, traditional systems lack a sense of colocation and interaction as in a face-to-face meeting due to the separate displays of remote participants and poor integration with the shared collaborative contents. As a result, teleimmersive (TI) systems that aim to provide natural user experiences and interaction have attracted increasing research interest [1]. High-end telepresence products such as Cisco TelePresence or HP?s Halo were expressly designed to create the perception of meeting in the same physical space. But to achieve such an experience, these systems require a proprietary installation and high setup costs. Recently, some three-dimensional (3-D) TI systems have been developed to enhance remote collaboration by merging remote participants into the same 3-D virtual space [2]?[4]. However, these systems still fall short of simulating a face-to-face collaboration with the presence of shared contents. Also, the required bulky and expensive hardware with nontrivial calibration and setup hinders their wide adoption. With the wide availability of low-cost, commodity computing devices with embedded video cameras, microphones, and ubiquitous Internet access adequate for real-time media, the dream of high-quality TI communication should finally be within our reach.

Original languageEnglish (US)
Article number6923517
Pages (from-to)118-136
Number of pages19
JournalIEEE Signal Processing Magazine
Volume31
Issue number6
DOIs
StatePublished - Nov 2014

Fingerprint

Visual communication
Hardware
Communication
Video cameras
Microphones
Merging
Telepresence
Costs
Display devices
Availability
Calibration
Internet
Setup Cost
User Experience
User Interaction
Interaction
3D
Vision
Display
Person

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering
  • Applied Mathematics

Cite this

Teleimmersive audio-visual communication using commodity hardware. / Nguyen, Viet Anh; Lu, Jiangbo; Zhao, Shengkui; Jones, Douglas L; Do, Minh N.

In: IEEE Signal Processing Magazine, Vol. 31, No. 6, 6923517, 11.2014, p. 118-136.

Research output: Contribution to journalArticle

Nguyen, Viet Anh ; Lu, Jiangbo ; Zhao, Shengkui ; Jones, Douglas L ; Do, Minh N. / Teleimmersive audio-visual communication using commodity hardware. In: IEEE Signal Processing Magazine. 2014 ; Vol. 31, No. 6. pp. 118-136.
@article{2a3854b3ce674eb482beff4642bbf52e,
title = "Teleimmersive audio-visual communication using commodity hardware",
abstract = "Natural human communication involves complex visual and audio behavior, and often context and joint interaction with the surrounding environment, to create a rich and satisfying experience. However, widely used virtual meeting systems such as WebEx and Skype still provide rather limited functionalities and hardly maintain the experience of an in-person meeting. In particular, traditional systems lack a sense of colocation and interaction as in a face-to-face meeting due to the separate displays of remote participants and poor integration with the shared collaborative contents. As a result, teleimmersive (TI) systems that aim to provide natural user experiences and interaction have attracted increasing research interest [1]. High-end telepresence products such as Cisco TelePresence or HP?s Halo were expressly designed to create the perception of meeting in the same physical space. But to achieve such an experience, these systems require a proprietary installation and high setup costs. Recently, some three-dimensional (3-D) TI systems have been developed to enhance remote collaboration by merging remote participants into the same 3-D virtual space [2]?[4]. However, these systems still fall short of simulating a face-to-face collaboration with the presence of shared contents. Also, the required bulky and expensive hardware with nontrivial calibration and setup hinders their wide adoption. With the wide availability of low-cost, commodity computing devices with embedded video cameras, microphones, and ubiquitous Internet access adequate for real-time media, the dream of high-quality TI communication should finally be within our reach.",
author = "Nguyen, {Viet Anh} and Jiangbo Lu and Shengkui Zhao and Jones, {Douglas L} and Do, {Minh N}",
year = "2014",
month = "11",
doi = "10.1109/MSP.2014.2340232",
language = "English (US)",
volume = "31",
pages = "118--136",
journal = "IEEE Signal Processing Magazine",
issn = "1053-5888",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

TY - JOUR

T1 - Teleimmersive audio-visual communication using commodity hardware

AU - Nguyen, Viet Anh

AU - Lu, Jiangbo

AU - Zhao, Shengkui

AU - Jones, Douglas L

AU - Do, Minh N

PY - 2014/11

Y1 - 2014/11

N2 - Natural human communication involves complex visual and audio behavior, and often context and joint interaction with the surrounding environment, to create a rich and satisfying experience. However, widely used virtual meeting systems such as WebEx and Skype still provide rather limited functionalities and hardly maintain the experience of an in-person meeting. In particular, traditional systems lack a sense of colocation and interaction as in a face-to-face meeting due to the separate displays of remote participants and poor integration with the shared collaborative contents. As a result, teleimmersive (TI) systems that aim to provide natural user experiences and interaction have attracted increasing research interest [1]. High-end telepresence products such as Cisco TelePresence or HP?s Halo were expressly designed to create the perception of meeting in the same physical space. But to achieve such an experience, these systems require a proprietary installation and high setup costs. Recently, some three-dimensional (3-D) TI systems have been developed to enhance remote collaboration by merging remote participants into the same 3-D virtual space [2]?[4]. However, these systems still fall short of simulating a face-to-face collaboration with the presence of shared contents. Also, the required bulky and expensive hardware with nontrivial calibration and setup hinders their wide adoption. With the wide availability of low-cost, commodity computing devices with embedded video cameras, microphones, and ubiquitous Internet access adequate for real-time media, the dream of high-quality TI communication should finally be within our reach.

AB - Natural human communication involves complex visual and audio behavior, and often context and joint interaction with the surrounding environment, to create a rich and satisfying experience. However, widely used virtual meeting systems such as WebEx and Skype still provide rather limited functionalities and hardly maintain the experience of an in-person meeting. In particular, traditional systems lack a sense of colocation and interaction as in a face-to-face meeting due to the separate displays of remote participants and poor integration with the shared collaborative contents. As a result, teleimmersive (TI) systems that aim to provide natural user experiences and interaction have attracted increasing research interest [1]. High-end telepresence products such as Cisco TelePresence or HP?s Halo were expressly designed to create the perception of meeting in the same physical space. But to achieve such an experience, these systems require a proprietary installation and high setup costs. Recently, some three-dimensional (3-D) TI systems have been developed to enhance remote collaboration by merging remote participants into the same 3-D virtual space [2]?[4]. However, these systems still fall short of simulating a face-to-face collaboration with the presence of shared contents. Also, the required bulky and expensive hardware with nontrivial calibration and setup hinders their wide adoption. With the wide availability of low-cost, commodity computing devices with embedded video cameras, microphones, and ubiquitous Internet access adequate for real-time media, the dream of high-quality TI communication should finally be within our reach.

UR - http://www.scopus.com/inward/record.url?scp=85032752058&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85032752058&partnerID=8YFLogxK

U2 - 10.1109/MSP.2014.2340232

DO - 10.1109/MSP.2014.2340232

M3 - Article

AN - SCOPUS:85032752058

VL - 31

SP - 118

EP - 136

JO - IEEE Signal Processing Magazine

JF - IEEE Signal Processing Magazine

SN - 1053-5888

IS - 6

M1 - 6923517

ER -