Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms

Dominic Roberts, Yunpeng Wang, Ali Sabet, Mani Golparvar-Fard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The availability of inexpensive and high-quality cameras has enabled research towards computer vision systems for tracking construction productivity and safety by detecting, tracking, and estimating pose of construction resources and recognizing their activities. To verify these algorithms can generalize to novel visual scenes before deployment, large amounts of labelled training and real-world validation samples are needed. While automatic generation of synthetic data helps with the former, obtaining the latter is less practical. To address this gap, we introduce a tool with which designated annotators perform pose annotation by interactively aligning 3D kinematically configurable equipment models with their depictions on 2D images. Multiple types of informative visual annotation can be retrieved using this technique, notably segmentation masks and equipment pose. Experiments demonstrate our tool's effectiveness for annotating long-form videos of earthmoving operations. We demonstrate how our tool can provide ground-truth annotations for the evaluation of a variety of computer-vision algorithms.

Original languageEnglish (US)
Title of host publicationComputing in Civil Engineering 2019
Subtitle of host publicationVisualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019
EditorsYong K. Cho, Fernanda Leite, Amir Behzadan, Chao Wang
PublisherAmerican Society of Civil Engineers
Pages32-38
Number of pages7
ISBN (Electronic)9780784482421
DOIs
StatePublished - Jan 1 2019
EventASCE International Conference on Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation, i3CE 2019 - Atlanta, United States
Duration: Jun 17 2019Jun 19 2019

Publication series

NameComputing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019

Conference

ConferenceASCE International Conference on Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation, i3CE 2019
CountryUnited States
CityAtlanta
Period6/17/196/19/19

    Fingerprint

ASJC Scopus subject areas

  • Computer Science(all)
  • Civil and Structural Engineering

Cite this

Roberts, D., Wang, Y., Sabet, A., & Golparvar-Fard, M. (2019). Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms. In Y. K. Cho, F. Leite, A. Behzadan, & C. Wang (Eds.), Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019 (pp. 32-38). (Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019). American Society of Civil Engineers. https://doi.org/10.1061/9780784482421.005