Evaluating railroad ballast degradation trends using machine vision and machine learning techniques

Recently, automatic ballast sampling (ABS) methods have been introduced to the railroad industry to obtain a sample of ballast and underlying layers. Currently, manual-visual classification methods are used by experts to identify fouling conditions and degradation trends in the collected ballast samples. This paper presents an innovative approach developed for objective classification of ballast degradation using the combination of machine vision and machine learning techniques. Initially, various computer vision algorithms were used to generate features associated with images of ballast cross sections at different degradation levels. Next, the generated features were used alongside a visual classification database provided by experts to develop, train, validate, and test a feed forward artificial neural network (ANN) using a supervised learning method. This work was further extended by implementing convolutional neural networks (CNNs) to serve as automatic feature generators. The findings of this study showed that the proposed CNNs with an optimized topology could successfully classify ballast fouling in an effective and repeatable fashion with reasonable error levels. Further improvement of this technology holds the potential to provide a tool for consistent and automated ballast inspection and life cycle analysis intended to improve the safety and network reliability of US railroad transportation system.