Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms

Dominic Roberts, Yunpeng Wang, Ali Sabet, Mani Golparvar Fard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The availability of inexpensive and high-quality cameras has enabled research towards computer vision systems for tracking construction productivity and safety by detecting, tracking, and estimating pose of construction resources and recognizing their activities. To verify these algorithms can generalize to novel visual scenes before deployment, large amounts of labelled training and real-world validation samples are needed. While automatic generation of synthetic data helps with the former, obtaining the latter is less practical. To address this gap, we introduce a tool with which designated annotators perform pose annotation by interactively aligning 3D kinematically configurable equipment models with their depictions on 2D images. Multiple types of informative visual annotation can be retrieved using this technique, notably segmentation masks and equipment pose. Experiments demonstrate our tool's effectiveness for annotating long-form videos of earthmoving operations. We demonstrate how our tool can provide ground-truth annotations for the evaluation of a variety of computer-vision algorithms.

Original languageEnglish (US)
Title of host publicationComputing in Civil Engineering 2019
Subtitle of host publicationVisualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019
EditorsChao Wang, Yong K. Cho, Fernanda Leite, Amir Behzadan
PublisherAmerican Society of Civil Engineers (ASCE)
Pages32-38
Number of pages7
ISBN (Electronic)9780784482421
DOIs
StatePublished - Jan 1 2019
EventASCE International Conference on Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation, i3CE 2019 - Atlanta, United States
Duration: Jun 17 2019Jun 19 2019

Publication series

NameComputing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019

Conference

ConferenceASCE International Conference on Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation, i3CE 2019
CountryUnited States
CityAtlanta
Period6/17/196/19/19

Fingerprint

Construction equipment
Computer vision
Monitoring
Masks
Productivity
Cameras
Availability
Experiments

ASJC Scopus subject areas

  • Computer Science(all)
  • Civil and Structural Engineering

Cite this

Roberts, D., Wang, Y., Sabet, A., & Golparvar Fard, M. (2019). Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms. In C. Wang, Y. K. Cho, F. Leite, & A. Behzadan (Eds.), Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019 (pp. 32-38). (Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019). American Society of Civil Engineers (ASCE). https://doi.org/10.1061/9780784482421.005

Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms. / Roberts, Dominic; Wang, Yunpeng; Sabet, Ali; Golparvar Fard, Mani.

Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. ed. / Chao Wang; Yong K. Cho; Fernanda Leite; Amir Behzadan. American Society of Civil Engineers (ASCE), 2019. p. 32-38 (Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Roberts, D, Wang, Y, Sabet, A & Golparvar Fard, M 2019, Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms. in C Wang, YK Cho, F Leite & A Behzadan (eds), Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019, American Society of Civil Engineers (ASCE), pp. 32-38, ASCE International Conference on Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation, i3CE 2019, Atlanta, United States, 6/17/19. https://doi.org/10.1061/9780784482421.005
Roberts D, Wang Y, Sabet A, Golparvar Fard M. Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms. In Wang C, Cho YK, Leite F, Behzadan A, editors, Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. American Society of Civil Engineers (ASCE). 2019. p. 32-38. (Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019). https://doi.org/10.1061/9780784482421.005
Roberts, Dominic ; Wang, Yunpeng ; Sabet, Ali ; Golparvar Fard, Mani. / Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms. Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. editor / Chao Wang ; Yong K. Cho ; Fernanda Leite ; Amir Behzadan. American Society of Civil Engineers (ASCE), 2019. pp. 32-38 (Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019).
@inproceedings{9cfb643c22c54517b58d131a1bacea63,
title = "Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms",
abstract = "The availability of inexpensive and high-quality cameras has enabled research towards computer vision systems for tracking construction productivity and safety by detecting, tracking, and estimating pose of construction resources and recognizing their activities. To verify these algorithms can generalize to novel visual scenes before deployment, large amounts of labelled training and real-world validation samples are needed. While automatic generation of synthetic data helps with the former, obtaining the latter is less practical. To address this gap, we introduce a tool with which designated annotators perform pose annotation by interactively aligning 3D kinematically configurable equipment models with their depictions on 2D images. Multiple types of informative visual annotation can be retrieved using this technique, notably segmentation masks and equipment pose. Experiments demonstrate our tool's effectiveness for annotating long-form videos of earthmoving operations. We demonstrate how our tool can provide ground-truth annotations for the evaluation of a variety of computer-vision algorithms.",
author = "Dominic Roberts and Yunpeng Wang and Ali Sabet and {Golparvar Fard}, Mani",
year = "2019",
month = "1",
day = "1",
doi = "10.1061/9780784482421.005",
language = "English (US)",
series = "Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019",
publisher = "American Society of Civil Engineers (ASCE)",
pages = "32--38",
editor = "Chao Wang and Cho, {Yong K.} and Fernanda Leite and Amir Behzadan",
booktitle = "Computing in Civil Engineering 2019",

}

TY - GEN

T1 - Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms

AU - Roberts, Dominic

AU - Wang, Yunpeng

AU - Sabet, Ali

AU - Golparvar Fard, Mani

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The availability of inexpensive and high-quality cameras has enabled research towards computer vision systems for tracking construction productivity and safety by detecting, tracking, and estimating pose of construction resources and recognizing their activities. To verify these algorithms can generalize to novel visual scenes before deployment, large amounts of labelled training and real-world validation samples are needed. While automatic generation of synthetic data helps with the former, obtaining the latter is less practical. To address this gap, we introduce a tool with which designated annotators perform pose annotation by interactively aligning 3D kinematically configurable equipment models with their depictions on 2D images. Multiple types of informative visual annotation can be retrieved using this technique, notably segmentation masks and equipment pose. Experiments demonstrate our tool's effectiveness for annotating long-form videos of earthmoving operations. We demonstrate how our tool can provide ground-truth annotations for the evaluation of a variety of computer-vision algorithms.

AB - The availability of inexpensive and high-quality cameras has enabled research towards computer vision systems for tracking construction productivity and safety by detecting, tracking, and estimating pose of construction resources and recognizing their activities. To verify these algorithms can generalize to novel visual scenes before deployment, large amounts of labelled training and real-world validation samples are needed. While automatic generation of synthetic data helps with the former, obtaining the latter is less practical. To address this gap, we introduce a tool with which designated annotators perform pose annotation by interactively aligning 3D kinematically configurable equipment models with their depictions on 2D images. Multiple types of informative visual annotation can be retrieved using this technique, notably segmentation masks and equipment pose. Experiments demonstrate our tool's effectiveness for annotating long-form videos of earthmoving operations. We demonstrate how our tool can provide ground-truth annotations for the evaluation of a variety of computer-vision algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85068800590&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068800590&partnerID=8YFLogxK

U2 - 10.1061/9780784482421.005

DO - 10.1061/9780784482421.005

M3 - Conference contribution

T3 - Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019

SP - 32

EP - 38

BT - Computing in Civil Engineering 2019

A2 - Wang, Chao

A2 - Cho, Yong K.

A2 - Leite, Fernanda

A2 - Behzadan, Amir

PB - American Society of Civil Engineers (ASCE)

ER -