Bayesian Data Augmentation and Training for Perception DNN in Autonomous Aerial Vehicles

Learning-based solutions have enabled incredible capabilities for autonomous systems. Autonomous vehicles, both aerial and ground, rely on Deep Neural Networks (DNN) for various integral tasks, including perception. The efficacy of supervised learning solutions, such as the DNNusedforperceptiontasks, hinges on the quality of the training data. Discrepancies between training data and operating conditions result in faults that can lead to catastrophic incidents. However, collecting and labeling vast amounts of context-sensitive data, with broad coverage of possible variations in the operating environment, is prohibitively difficult. To overcome this limitation, synthetic data generation techniques for DNN training emerged, allowing for the easy exploration of diverse scenarios. While significant synthetic data generation solutions exist for ground vehicles, for aerial vehicles such support is still lacking. This work presents a data augmentation framework for aerial vehicle’s perception training, leveraging photorealistic simulation seamlessly integrated with high-fidelity vehicle dynamics, control, and planning algorithms. Safe landing in urban environments is a crucial challenge in the development of autonomous air taxis, and therefore, landing maneuver is chosen as the focus of this work. With repeated simulations of landing maneuvers in scenarios with varying vehicle states, weather conditions and time of day, we assess the landing performance of the VTOL (Vertical Take off and Landing) type UAV and gather valuable data. The landing performance is used as the objective function to optimize the DNN through retraining. Given the high computational cost of DNN retraining, we incorporated Bayesian Optimization in our framework that systematically explores the data augmentation parameter space to retrain the best-performing models. The framework allowed us to identify high-performing data augmentation parameters that are consistently effective across different landing scenarios. Utilizing the capabilities of this data augmentation framework, we obtained a robust perception model. The model consistently improved the perception-based landing success rate by at least 20%under different lighting and weather conditions.