Roto texture: Automated tools for texturing raw video

Hui Fang, John C. Hart

Research output: Contribution to journalArticle

Abstract

We propose a video editing system that allows a user to apply a time-coherent texture to a surface depicted in the raw video from a single uncalibrated camera, including the surface texture mapping of a texture image and the surface texture synthesis from a texture swatch. Our system avoids the construction of a 3D shape model and instead uses the recovered normal field to deform the texture so that it plausibly adheres to the undulations of the depicted surface. The texture mapping method uses the nonlinear least-squares optimization of a spring model to control the behavior of the texture image as it is deformed to match the evolving normal field through the video. The texture synthesis method uses a coarse optical flow to advect clusters of pixels corresponding to patches of similarly oriented surface points. These clusters are organized into a minimum advection tree to account for the dynamic visibility of clusters. We take a rather crude approach to normal recovering and optical flow estimation, yet the results are robust and plausible for nearly diffuse surfaces such as faces and t-shirts.

Original languageEnglish (US)
Article number1703377
Pages (from-to)1580-1589
Number of pages10
JournalIEEE Transactions on Visualization and Computer Graphics
Volume12
Issue number6
DOIs
StatePublished - Nov 1 2006

Keywords

  • Shape from shading
  • Texture synthesis
  • Video editing

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Roto texture: Automated tools for texturing raw video'. Together they form a unique fingerprint.

  • Cite this