Adaptive optimal control of partially-unknown constrained-input systems using policy iteration with experience replay

Hamidreza Modares, Frank L. Lewis, Mohammad Bagher Naghibi-Sistani, Girish Chowdhary, Tansel Yucelen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper develops an online learning algorithm to find optimal control solutions for partially-unknown continuous-time systems subject to input constraints. The input constraints are encoded into the optimal control problem through a nonquadratic performance functional. An online policy iteration algorithm that uses integral reinforcement knowledge is developed to learn the solution to the optimal control problem online without knowing the full dynamics model. The policy iteration algorithm is implemented on an actor-critic structure, where two neural network approximators are tuned online and simultaneously to generate the optimal control law. A novel technique based on experience replay is introduced to retain past data in updating the neural network weights. This uses the recorded data concurrently with current data for adaptation of the critic neural network weights. Concurrent learning provides an easy-to-check real-time condition for persistence of excitation that is sufficient to guarantee convergence to a near optimal control law. Stability of the proposed feedback control law is shown and its performance is evaluated through simulations.

Original languageEnglish (US)
Title of host publicationAIAA Guidance, Navigation, and Control (GNC) Conference
StatePublished - Sep 16 2013
Externally publishedYes
EventAIAA Guidance, Navigation, and Control (GNC) Conference - Boston, MA, United States
Duration: Aug 19 2013Aug 22 2013

Publication series

NameAIAA Guidance, Navigation, and Control (GNC) Conference

Other

OtherAIAA Guidance, Navigation, and Control (GNC) Conference
CountryUnited States
CityBoston, MA
Period8/19/138/22/13

ASJC Scopus subject areas

  • Aerospace Engineering
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Adaptive optimal control of partially-unknown constrained-input systems using policy iteration with experience replay'. Together they form a unique fingerprint.

Cite this