Vision-based construction activity analysis in long video sequences via hidden markov models: Experiments on earthmoving operations

Dominic Roberts, Mani Golparvar-Fard, Juan Carlos Niebles, Junyoung Gwak, Ruxiao Bao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a new method for detailed activity analysis of dynamic construction resources in highly varying videos obtained from construction site cameras. Toward this goal, we propose a Hidden Markov Model (HMM) that is able to automatically discover and assign sequences of activities that are most discriminative for an observed construction operation. To do so, the algorithm leverages dense trajectory features from a detected dynamic resource (e.g., excavator) in a video. Using these dense trajectory features, we train a Gaussian mixture model (GMM) to estimate the probability density function of each activity with multiple one-versus-all support vector machine classifiers. The proposed HMM also models duration of each activity, and the transition between activities (e.g., "swing bucket loaded" after "load bucket" for earth-moving activities of an excavator). As a proof-of-concept, we train and test our HMM+GMM model on an unprecedented dataset of 10 real-world long video sequences of interacting pairs of excavators and dumptrucks. Our preliminary experimental results on long-sequence activity recognition in presence of noise, occlusions, and scene clutter demonstrate the effectiveness of our method.

Original languageEnglish (US)
Title of host publicationConstruction Research Congress 2018
Subtitle of host publicationSafety and Disaster Management - Selected Papers from the Construction Research Congress 2018
EditorsChristofer Harper, Yongcheol Lee, Rebecca Harris, Charles Berryman, Chao Wang
PublisherAmerican Society of Civil Engineers
Pages164-173
Number of pages10
ISBN (Electronic)9780784481288
DOIs
StatePublished - 2018
EventConstruction Research Congress 2018: Safety and Disaster Management, CRC 2018 - New Orleans, United States
Duration: Apr 2 2018Apr 4 2018

Publication series

NameConstruction Research Congress 2018: Safety and Disaster Management - Selected Papers from the Construction Research Congress 2018
Volume2018-April

Other

OtherConstruction Research Congress 2018: Safety and Disaster Management, CRC 2018
CountryUnited States
CityNew Orleans
Period4/2/184/4/18

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Building and Construction

Fingerprint Dive into the research topics of 'Vision-based construction activity analysis in long video sequences via hidden markov models: Experiments on earthmoving operations'. Together they form a unique fingerprint.

Cite this