Adaptive Risk Sensitive Path Integral for Model Predictive Control via Reinforcement Learning

Hyung Jin Yoon, Chuyuan Tao, Hunmin Kim, Naira Hovakimyan, Petros G Voulgaris

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a reinforcement learning framework where an agent uses an internal nominal model for stochastic model predictive control (MPC) while compensating for a disturbance. Our work builds on the existing risk-aware optimal control with stochastic differential equations (SDEs) that aims to deal with such disturbance. However, the risk sensitivity and the noise strength of the nominal SDE in the riskaware optimal control are often heuristically chosen. In the proposed framework, the risk-taking policy determines the behavior of the MPC to be risk-seeking (exploration) or risk-averse (exploitation). Specifically, we employ the risk-aware path integral control that can be implemented as a Monte-Carlo (MC) sampling with fast parallel simulations using a GPU. The MC sampling implementations of the MPC have been successful in robotic applications due to their real-time computation capability. The proposed framework that adapts the noise model and the risk sensitivity outperforms the standard model predictive path integral in simulation environments that have disturbances.

Original languageEnglish (US)
Title of host publication2023 31st Mediterranean Conference on Control and Automation, MED 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages926-931
Number of pages6
ISBN (Electronic)9798350315431
DOIs
StatePublished - 2023
Event31st Mediterranean Conference on Control and Automation, MED 2023 - Limassol, Cyprus
Duration: Jun 26 2023Jun 29 2023

Publication series

Name2023 31st Mediterranean Conference on Control and Automation, MED 2023

Conference

Conference31st Mediterranean Conference on Control and Automation, MED 2023
Country/TerritoryCyprus
CityLimassol
Period6/26/236/29/23

ASJC Scopus subject areas

  • Aerospace Engineering
  • Automotive Engineering
  • Safety, Risk, Reliability and Quality
  • Control and Optimization

Fingerprint

Dive into the research topics of 'Adaptive Risk Sensitive Path Integral for Model Predictive Control via Reinforcement Learning'. Together they form a unique fingerprint.

Cite this