Automatic identification of window regions on indoor point clouds using LiDAR and cameras

In this paper, we propose an algorithm to automatically identify window regions on exterior facing facades of buildings using interior 3D point cloud resulting from an ambulatory backpack sensor system, outfitted with multiple LiDAR sensors and cameras. We develop a set of discriminative features for the task, namely visual brightness, infrared opaqueness, and an occlusion indicator, within a Markov Random Field (MRF) framework to provide structured prediction for window or glass regions. A preprocessing classifier is trained on the features to produce node potentials, and large margin parameter training is used to boost performance. Our algorithm has been trained on data taken at the 3^rd floor of Cory Hall at UC Berkeley, with a total façade area of 269.1 m², and has been tested on walls taken on the 2 ^nd floor of Cory Hall, a Walgreens, and an office building in San Francisco, with a total exterior façade area of 454.6 m². Window regions are successfully identified with 85.5% F₁-score and 94.2% accuracy.