Online Learning and Planning in Time-Varying Environments: An Aircraft Case Study

Gokul Puthumanaillam, Yuvraj Mamik, Melkior Ornik

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Aerospace vehicles routinely encounter uncertain, time-varying, and partially observable environments, presenting considerable challenges for autonomous operation and planning. Traditional learning methods, which excel in static contexts, often falter in such highly dynamic settings. Building on recently established Time-Varying Partially Observable Markov Decision Processes (TV-POMDP) and Memory Prioritized State Estimation (MPSE) methodologies, this work demonstrates their application in the advanced GUAM simulation environment, which models NASA’s Generic UAM concept. The contribution of this paper lies in refining these approaches to suit the complexity and unpredictability of aerospace contexts, where conventional learning strategies are insufficient. By applying MPSE, we enhance the estimation of environmental states with a weighted approach that respects the temporality and informational value of observations. The subsequent policy optimization process is informed by the estimations of these time-varying transition functions, leading to better long-term strategies that are aware of the rapid environmental shifts characteristic of aerospace scenarios. The validation of these methods through the GUAM simulator confirms their effectiveness, marking a positive step towards their practical implementation in autonomous aerospace vehicles that encounter continual, stochastic changes.

Original languageEnglish (US)
Title of host publicationAIAA SciTech Forum and Exposition, 2024
PublisherAmerican Institute of Aeronautics and Astronautics Inc, AIAA
ISBN (Print)9781624107115
StatePublished - 2024
EventAIAA SciTech Forum and Exposition, 2024 - Orlando, United States
Duration: Jan 8 2024Jan 12 2024

Publication series

NameAIAA SciTech Forum and Exposition, 2024


ConferenceAIAA SciTech Forum and Exposition, 2024
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Aerospace Engineering


Dive into the research topics of 'Online Learning and Planning in Time-Varying Environments: An Aircraft Case Study'. Together they form a unique fingerprint.

Cite this