Segmentation and recognition of roadway assets from car-mounted camera video streams using a scalable non-parametric image parsing method

This paper presents a non-parametric image parsing method for segmentation and recognition of roadway assets such as traffic signs, traffic lights, pavement markings, and guardrails from 2D car-mounted video streams. The method can be easily scaled to thousands of video frames captured during data collection and does not need training. Instead, it retrieves a set of most relevant video frames (e.g. highway vs. secondary road) which serve as candidates for superpixel-level annotation. It then obtains superpixels from the video frames and using the retrieval set encodes their visual characteristics using a histogram of different shape, appearance, and color descriptors. Neighborhood contexts are incorporated by using Markov Random Field (MRF) optimization and two types of semantic (e.g. guardrail) and geometric (e.g. horizontal) labels are simultaneously assigned to the superpixels. We introduce a new dataset from I-57 together with its ground truth and present experimental results on both I-57 and SmartRoad datasets. Experimental results with an average accuracy of 88.24% for recognition and 82.02% for segmentation show that our local visual features provide acceptable performance, while the method overall does not require any significant supervised training. This scalable method has potential to reduce the time and effort required for developing road inventories, especially for those such as guardrails and traffic lights that are not typically considered in 2D asset recognition methods.