Vision-based workface assessment using depth images for activity analysis of interior construction operations

Ardalan Khosrowpour, Juan Carlos Niebles, Mani Golparvar-Fard

Research output: Contribution to journalArticlepeer-review

Abstract

Workface assessment - the process of determining the overall activity rates of onsite construction workers throughout a day - typically involves manual visual observations which are time-consuming and labor-intensive. To minimize subjectivity and the time required for conducting detailed assessments, and allowing managers to spend their time on the more important task of assessing and implementing improvements, we propose a new inexpensive vision-based method using RGB-D sensors that is applicable to interior construction operations. This is a particularly challenging task as construction activities have a large range of intra-class variability including varying sequences of body posture and time-spent on each individual activity. The skeleton extraction algorithms from RGB-D sequences produce noisy outputs when workers interact with tools or when there is a significant body occlusion within the camera's field-of-view. Existing vision-based methods are also limited as they can primarily classify "atomic" activities from RGB-D sequences involving one worker conducting a single activity. To address these limitations, our method includes three components: 1) an algorithm for detecting, tracking, and extracting body skeleton features from depth images; 2) a discriminative bag-of-poses activity classifier for classifying single visual activities from a given body skeleton sequence; and 3) a Hidden Markov Model to represent emission probabilities in the form of a statistical distribution of single activity classifiers. For training and testing purposes, we introduce a new dataset of eleven RGB-D sequences for interior drywall construction operations involving three actual construction workers conducting eight different activities in various interior locations. Our results with an average accuracy of 76% on the testing dataset show the promise of vision-based methods using RGB-D sequences for facilitating the activity analysis workface assessment.

Original languageEnglish (US)
Pages (from-to)74-87
Number of pages14
JournalAutomation in Construction
Volume48
DOIs
StatePublished - Dec 2014

Keywords

  • Activity analysis
  • Hidden Markov Model
  • RGB-D cameras
  • Workface assessment

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Civil and Structural Engineering
  • Building and Construction

Fingerprint Dive into the research topics of 'Vision-based workface assessment using depth images for activity analysis of interior construction operations'. Together they form a unique fingerprint.

Cite this