Controlled interacting particle algorithms for simulation-based reinforcement learning

Anant A. Joshi, Amirhossein Taghvaei, Prashant G. Mehta, Sean P. Meyn

Research output: Contribution to journalArticlepeer-review


This paper is concerned with optimal control problems for control systems in continuous time, and interacting particle system methods designed to construct approximate control solutions. Particular attention is given to the linear quadratic (LQ) control problem. There is a growing interest in re-visiting this classical problem, in part due to the successes of reinforcement learning (RL). The main question of this body of research (and also of our paper) is to approximate the optimal control law without explicitly solving the Riccati equation. A novel simulation-based algorithm, namely a dual ensemble Kalman filter (EnKF), is introduced. The algorithm is used to obtain formulae for optimal control, expressed entirely in terms of the EnKF particles. An extension to the nonlinear case is also presented. The theoretical results and algorithms are illustrated with numerical experiments.

Original languageEnglish (US)
Article number105392
JournalSystems and Control Letters
StatePublished - Dec 2022


  • Duality
  • Linear quadratic (LQ)
  • Optimal control
  • Reinforcement learning
  • Riccati equation

ASJC Scopus subject areas

  • Control and Systems Engineering
  • General Computer Science
  • Mechanical Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'Controlled interacting particle algorithms for simulation-based reinforcement learning'. Together they form a unique fingerprint.

Cite this