Recognition and 3D localization of traffic signs via image-based point cloud models

Recently, the US Departments of Transportation have pro-actively looked into videotaping roadway assets. Using inspection vehicles equipped with high resolution cameras, accurate information on location and condition of high quantity and low cost roadway assets are being collected. While many efforts have focused on streamlining the data collection, the analysis is still manual and involves painstaking and subjective processes. Their high cost has also limited the scope of the visual assessments to critical roadways only. To address current limitations, this paper presents an automated method to detect, classify, and accurately localize traffic signs in 3D using existing visual data. Using a discriminative learning method based on Histograms of Oriented Gradients and Color, traffic signs are detected and classified into multiple categories. Then, a Structure from Motion procedure creates a 3D point cloud from the street level images, and triangulates the location of the detected signs in 3D. The experimental results show that the method reliably detects and localizes traffic signs and demonstrate a strong potential in improving assessments and lowering cost in practical applications.