Empirical Dynamic Programming for Controlled Diffusion Processes

Sambhu H. Karumanchi, Mohamed A. Belabbas, Naira Hovakimyan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider Markov chain approximation for optimal control of diffusion processes under infinite horizon discounted cost optimality and apply the simulation-based Empirical Value Iteration to estimate the value function of each approximating chain. We follow a nested multi-grid discretization of the state space to establish weak convergence of the value function sequence to the value function of the original controlled diffusion. We illustrate the convergence performance of the model on the popular Benes' bang-bang control problem [Beneš (1974)].

Original languageEnglish (US)
Title of host publicationIFAC-PapersOnLine
EditorsHideaki Ishii, Yoshio Ebihara, Jun-ichi Imura, Masaki Yamakita
PublisherElsevier B.V.
Pages11235-11241
Number of pages7
Edition2
ISBN (Electronic)9781713872344
DOIs
StatePublished - Jul 1 2023
Externally publishedYes
Event22nd IFAC World Congress - Yokohama, Japan
Duration: Jul 9 2023Jul 14 2023

Publication series

NameIFAC-PapersOnLine
Number2
Volume56
ISSN (Electronic)2405-8963

Conference

Conference22nd IFAC World Congress
Country/TerritoryJapan
CityYokohama
Period7/9/237/14/23

Keywords

  • Diffusion processes
  • Markov decision process
  • Numerical methods for optimal control
  • Reinforcement learning
  • Stochastic optimal control
  • Value iteration

ASJC Scopus subject areas

  • Control and Systems Engineering

Fingerprint

Dive into the research topics of 'Empirical Dynamic Programming for Controlled Diffusion Processes'. Together they form a unique fingerprint.

Cite this