A forward-backward algorithm for stochastic control problems: Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming

Stephan E. Ludwig, Justin A Sirignano, Ruojun Huang, George Papanicolaou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

An algorithm for solving continuous-time stochastic optimal control problems is presented. The numerical scheme is based on the stochastic maximum principle (SMP) as an alternative to the widely studied dynamic programming principle (DDP). By using the SMP, (Peng, 1990) obtained a system of coupled forward-backward stochastic differential equations (FBSDE) with an external optimality condition. We extend the numerical scheme of (Delarue and Menozzi, 2006) by a Newton-Raphson method to solve the FBSDE system and the optimality condition simultaneously. As far as the authors are aware, this is the first fully explicit numerical scheme for the solution of optimal control problems through the solution of the corresponding extended FBSDE system. We discuss possible numerical advantages to the DDP approach and consider an optimal investment-consumption problem as an example.

Original languageEnglish (US)
Title of host publicationICORES 2012 - Proceedings of the 1st International Conference on Operations Research and Enterprise Systems
Pages83-89
Number of pages7
StatePublished - 2012
Externally publishedYes
Event1st International Conference on Operations Research and Enterprise Systems, ICORES 2012 - Vilamoura, Algarve, Portugal
Duration: Feb 4 2012Feb 6 2012

Other

Other1st International Conference on Operations Research and Enterprise Systems, ICORES 2012
CountryPortugal
CityVilamoura, Algarve
Period2/4/122/6/12

Keywords

  • Forward-Backward stochastic differential equations
  • Optimal stochastic control
  • Stochastic maximum principle

ASJC Scopus subject areas

  • Management Science and Operations Research

Fingerprint Dive into the research topics of 'A forward-backward algorithm for stochastic control problems: Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming'. Together they form a unique fingerprint.

Cite this