Interactive 3D model acquisition and tracking of building block structures

Andrew Miller, Brandyn White, Emiko Charbonneau, Zach Kanzler, Joseph J. Laviola

Research output: Contribution to journalArticlepeer-review


We present a prototype system for interactive construction and modification of 3D physical models using building blocks.Our system uses a depth sensing camera and a novel algorithm for acquiring and tracking the physical models. The algorithm,LatticeFirst, is based on the fact that building block structures can be arranged in a 3D point lattice where the smallest block unitis a basis in which to derive all the pieces of the model. The algorithm also makes it possible for users to interact naturally with thephysical model as it is acquired, using their bare hands to add and remove pieces. We present the details of our algorithm, along withexamples of the models we can acquire using the interactive system. We also show the results of an experiment where participantsmodify a block structure in the absence of visual feedback. Finally, we discuss two proofofconcept applications: a collaborativeguided assembly system where one user is interactively guided to build a structure based on another users design, and a gamewhere the player must build a structure that matches an onscreen silhouette.

Original languageEnglish (US)
Article number6165147
Pages (from-to)651-659
Number of pages9
JournalIEEE Transactions on Visualization and Computer Graphics
Issue number4
StatePublished - 2012
Externally publishedYes


  • 3D model acquisition
  • Interactive physical model building
  • building block structures
  • depth cameras
  • object tracking

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Interactive 3D model acquisition and tracking of building block structures'. Together they form a unique fingerprint.

Cite this