Reinforcement learning for spacecraft attitude control

Vedant, James T. Allison, Matthew West, Alexander Ghosh

Research output: Contribution to journalConference article

Abstract

Reinforcement learning (RL) has recently shown promise in solving difficult numerical problems and has discovered non-intuitive solutions to existing problems. This study investigates the ability of a general RL agent to find an optimal control strategy for spacecraft attitude control problems. Two main types of Attitude Control Systems (ACS) are presented. First, the general ACS problem with full actuation is considered, but with saturation constraints on the applied torques, representing thruster-based ACSs. Second, an attitude control problem with reaction wheel based ACS is considered, which has more constraints on control authority. The agent is trained using the Proximal Policy Optimization (PPO) RL method to obtain an attitude control policy. To ensure robustness, the inertia of the satellite is unknown to the control agent and is randomized for each simulation. To achieve efficient learning, the agent is trained using curriculum learning. We compare the RL based controller to a QRF (quaternion rate feedback) attitude controller, a well-established state feedback control strategy. We investigate the nominal performance and robustness with respect to uncertainty in system dynamics. Our RL based attitude control agent adapts to any spacecraft mass without needing to re-train. In the range of 0.1 to 100,000 kg, our agent achieves 2% better performance to a QRF controller tuned for the same mass range, and similar performance to the QRF controller tuned specifically for a given mass. The performance of the trained RL agent for the reaction wheel based ACS achieved 10 higher better reward then that of a tuned QRF controller.

Original languageEnglish (US)
Article numberIAC-19_C1_IP_4_x49857
JournalProceedings of the International Astronautical Congress, IAC
Volume2019-October
StatePublished - Jan 1 2019
Event70th International Astronautical Congress, IAC 2019 - Washington, United States
Duration: Oct 21 2019Oct 25 2019

Keywords

  • Adaptive control
  • Artificial Intelligence
  • Attitude control
  • Machine learning
  • Reinforcement learning
  • Robust control

ASJC Scopus subject areas

  • Aerospace Engineering
  • Astronomy and Astrophysics
  • Space and Planetary Science

Fingerprint Dive into the research topics of 'Reinforcement learning for spacecraft attitude control'. Together they form a unique fingerprint.

  • Cite this