Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases

Shuai Tang, Mani Golparvar Fard, Milind Naphade, Murali M. Gopalakrishna

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Forecasting activities of construction workers and equipment is immensely desirable for construction safety management, as it can enable capturing near-miss of accidents and right-time intervention of collision, struck-by, trespassing, and improper use of tools. In this paper, an activity forecasting framework for construction safety management is presented, and an application to forecast workers and equipment's motion trajectory from previously observed motion is implemented. A long short-term memory (LSTM) encoder-decoder network is developed to forecast future locations with mixture density network (MDN) used to model uncertainty in predictions. Two contextual cues are proved to help activity forecasting: (1) worker/equipment placement and distances; (2) object type attributes. A joint training schema is employed to forecast target's locations at different future times. The proposed framework can handle very long sequences (around 2000 steps) well and accurately predict future locations in maximum 40 frames (2 seconds). Our experiment results show that the proposed model significantly outperforms conventional time-series analysis models. In 1080p high-definition videos the final model achieves average localization error in 10, 20, 40 future frames with 7.30, 12.71, and 24.22 pixels, respectively.

Original languageEnglish (US)
Title of host publicationComputing in Civil Engineering 2019
Subtitle of host publicationSmart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019
EditorsChao Wang, Yong K. Cho, Fernanda Leite, Amir Behzadan
PublisherAmerican Society of Civil Engineers (ASCE)
Pages204-210
Number of pages7
ISBN (Electronic)9780784482445
DOIs
StatePublished - Jan 1 2019
EventASCE International Conference on Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience, i3CE 2019 - Atlanta, United States
Duration: Jun 17 2019Jun 19 2019

Publication series

NameComputing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019

Conference

ConferenceASCE International Conference on Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience, i3CE 2019
CountryUnited States
CityAtlanta
Period6/17/196/19/19

Fingerprint

Monitoring
Time series analysis
Accidents
Pixels
Trajectories
Experiments
Uncertainty
Long short-term memory

ASJC Scopus subject areas

  • Computer Science(all)
  • Civil and Structural Engineering

Cite this

Tang, S., Golparvar Fard, M., Naphade, M., & Gopalakrishna, M. M. (2019). Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases. In C. Wang, Y. K. Cho, F. Leite, & A. Behzadan (Eds.), Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019 (pp. 204-210). (Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019). American Society of Civil Engineers (ASCE). https://doi.org/10.1061/9780784482445.026

Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases. / Tang, Shuai; Golparvar Fard, Mani; Naphade, Milind; Gopalakrishna, Murali M.

Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. ed. / Chao Wang; Yong K. Cho; Fernanda Leite; Amir Behzadan. American Society of Civil Engineers (ASCE), 2019. p. 204-210 (Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tang, S, Golparvar Fard, M, Naphade, M & Gopalakrishna, MM 2019, Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases. in C Wang, YK Cho, F Leite & A Behzadan (eds), Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019, American Society of Civil Engineers (ASCE), pp. 204-210, ASCE International Conference on Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience, i3CE 2019, Atlanta, United States, 6/17/19. https://doi.org/10.1061/9780784482445.026
Tang S, Golparvar Fard M, Naphade M, Gopalakrishna MM. Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases. In Wang C, Cho YK, Leite F, Behzadan A, editors, Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. American Society of Civil Engineers (ASCE). 2019. p. 204-210. (Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019). https://doi.org/10.1061/9780784482445.026
Tang, Shuai ; Golparvar Fard, Mani ; Naphade, Milind ; Gopalakrishna, Murali M. / Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases. Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019. editor / Chao Wang ; Yong K. Cho ; Fernanda Leite ; Amir Behzadan. American Society of Civil Engineers (ASCE), 2019. pp. 204-210 (Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019).
@inproceedings{adc94dfb9576468c9e184fc578d943a4,
title = "Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases",
abstract = "Forecasting activities of construction workers and equipment is immensely desirable for construction safety management, as it can enable capturing near-miss of accidents and right-time intervention of collision, struck-by, trespassing, and improper use of tools. In this paper, an activity forecasting framework for construction safety management is presented, and an application to forecast workers and equipment's motion trajectory from previously observed motion is implemented. A long short-term memory (LSTM) encoder-decoder network is developed to forecast future locations with mixture density network (MDN) used to model uncertainty in predictions. Two contextual cues are proved to help activity forecasting: (1) worker/equipment placement and distances; (2) object type attributes. A joint training schema is employed to forecast target's locations at different future times. The proposed framework can handle very long sequences (around 2000 steps) well and accurately predict future locations in maximum 40 frames (2 seconds). Our experiment results show that the proposed model significantly outperforms conventional time-series analysis models. In 1080p high-definition videos the final model achieves average localization error in 10, 20, 40 future frames with 7.30, 12.71, and 24.22 pixels, respectively.",
author = "Shuai Tang and {Golparvar Fard}, Mani and Milind Naphade and Gopalakrishna, {Murali M.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1061/9780784482445.026",
language = "English (US)",
series = "Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019",
publisher = "American Society of Civil Engineers (ASCE)",
pages = "204--210",
editor = "Chao Wang and Cho, {Yong K.} and Fernanda Leite and Amir Behzadan",
booktitle = "Computing in Civil Engineering 2019",

}

TY - GEN

T1 - Video-Based Activity Forecasting for Construction Safety Monitoring Use Cases

AU - Tang, Shuai

AU - Golparvar Fard, Mani

AU - Naphade, Milind

AU - Gopalakrishna, Murali M.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Forecasting activities of construction workers and equipment is immensely desirable for construction safety management, as it can enable capturing near-miss of accidents and right-time intervention of collision, struck-by, trespassing, and improper use of tools. In this paper, an activity forecasting framework for construction safety management is presented, and an application to forecast workers and equipment's motion trajectory from previously observed motion is implemented. A long short-term memory (LSTM) encoder-decoder network is developed to forecast future locations with mixture density network (MDN) used to model uncertainty in predictions. Two contextual cues are proved to help activity forecasting: (1) worker/equipment placement and distances; (2) object type attributes. A joint training schema is employed to forecast target's locations at different future times. The proposed framework can handle very long sequences (around 2000 steps) well and accurately predict future locations in maximum 40 frames (2 seconds). Our experiment results show that the proposed model significantly outperforms conventional time-series analysis models. In 1080p high-definition videos the final model achieves average localization error in 10, 20, 40 future frames with 7.30, 12.71, and 24.22 pixels, respectively.

AB - Forecasting activities of construction workers and equipment is immensely desirable for construction safety management, as it can enable capturing near-miss of accidents and right-time intervention of collision, struck-by, trespassing, and improper use of tools. In this paper, an activity forecasting framework for construction safety management is presented, and an application to forecast workers and equipment's motion trajectory from previously observed motion is implemented. A long short-term memory (LSTM) encoder-decoder network is developed to forecast future locations with mixture density network (MDN) used to model uncertainty in predictions. Two contextual cues are proved to help activity forecasting: (1) worker/equipment placement and distances; (2) object type attributes. A joint training schema is employed to forecast target's locations at different future times. The proposed framework can handle very long sequences (around 2000 steps) well and accurately predict future locations in maximum 40 frames (2 seconds). Our experiment results show that the proposed model significantly outperforms conventional time-series analysis models. In 1080p high-definition videos the final model achieves average localization error in 10, 20, 40 future frames with 7.30, 12.71, and 24.22 pixels, respectively.

UR - http://www.scopus.com/inward/record.url?scp=85068777384&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068777384&partnerID=8YFLogxK

U2 - 10.1061/9780784482445.026

DO - 10.1061/9780784482445.026

M3 - Conference contribution

AN - SCOPUS:85068777384

T3 - Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019

SP - 204

EP - 210

BT - Computing in Civil Engineering 2019

A2 - Wang, Chao

A2 - Cho, Yong K.

A2 - Leite, Fernanda

A2 - Behzadan, Amir

PB - American Society of Civil Engineers (ASCE)

ER -