Online Learning and Planning in Time-Varying Environments: An Aircraft Case Study

Aerospace vehicles routinely encounter uncertain, time-varying, and partially observable environments, presenting considerable challenges for autonomous operation and planning. Traditional learning methods, which excel in static contexts, often falter in such highly dynamic settings. Building on recently established Time-Varying Partially Observable Markov Decision Processes (TV-POMDP) and Memory Prioritized State Estimation (MPSE) methodologies, this work demonstrates their application in the advanced GUAM simulation environment, which models NASA’s Generic UAM concept. The contribution of this paper lies in refining these approaches to suit the complexity and unpredictability of aerospace contexts, where conventional learning strategies are insufficient. By applying MPSE, we enhance the estimation of environmental states with a weighted approach that respects the temporality and informational value of observations. The subsequent policy optimization process is informed by the estimations of these time-varying transition functions, leading to better long-term strategies that are aware of the rapid environmental shifts characteristic of aerospace scenarios. The validation of these methods through the GUAM simulator confirms their effectiveness, marking a positive step towards their practical implementation in autonomous aerospace vehicles that encounter continual, stochastic changes.