A forward-backward algorithm for stochastic control problems: Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming

Stephan E. Ludwig; Justin A Sirignano; Ruojun Huang; George Papanicolaou

A forward-backward algorithm for stochastic control problems: Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming

Stephan E. Ludwig, Justin A Sirignano, Ruojun Huang, George Papanicolaou

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

An algorithm for solving continuous-time stochastic optimal control problems is presented. The numerical scheme is based on the stochastic maximum principle (SMP) as an alternative to the widely studied dynamic programming principle (DDP). By using the SMP, (Peng, 1990) obtained a system of coupled forward-backward stochastic differential equations (FBSDE) with an external optimality condition. We extend the numerical scheme of (Delarue and Menozzi, 2006) by a Newton-Raphson method to solve the FBSDE system and the optimality condition simultaneously. As far as the authors are aware, this is the first fully explicit numerical scheme for the solution of optimal control problems through the solution of the corresponding extended FBSDE system. We discuss possible numerical advantages to the DDP approach and consider an optimal investment-consumption problem as an example.

Original language	English (US)
Title of host publication	ICORES 2012 - Proceedings of the 1st International Conference on Operations Research and Enterprise Systems
Pages	83-89
Number of pages	7
State	Published - 2012
Externally published	Yes
Event	1st International Conference on Operations Research and Enterprise Systems, ICORES 2012 - Vilamoura, Algarve, Portugal Duration: Feb 4 2012 → Feb 6 2012

Other

Other	1st International Conference on Operations Research and Enterprise Systems, ICORES 2012
Country/Territory	Portugal
City	Vilamoura, Algarve
Period	2/4/12 → 2/6/12

Keywords

Forward-Backward stochastic differential equations
Optimal stochastic control
Stochastic maximum principle

ASJC Scopus subject areas

Management Science and Operations Research

Library availability

Discover UIUC Full Text

Cite this

A forward-backward algorithm for stochastic control problems : Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming. / Ludwig, Stephan E.; Sirignano, Justin A; Huang, Ruojun et al.

ICORES 2012 - Proceedings of the 1st International Conference on Operations Research and Enterprise Systems. 2012. p. 83-89.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ludwig, SE, Sirignano, JA, Huang, R & Papanicolaou, G 2012, A forward-backward algorithm for stochastic control problems: Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming. in ICORES 2012 - Proceedings of the 1st International Conference on Operations Research and Enterprise Systems. pp. 83-89, 1st International Conference on Operations Research and Enterprise Systems, ICORES 2012, Vilamoura, Algarve, Portugal, 2/4/12.

@inproceedings{8211d323053f4b10a185d09abffddbde,

title = "A forward-backward algorithm for stochastic control problems: Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming",

abstract = "An algorithm for solving continuous-time stochastic optimal control problems is presented. The numerical scheme is based on the stochastic maximum principle (SMP) as an alternative to the widely studied dynamic programming principle (DDP). By using the SMP, (Peng, 1990) obtained a system of coupled forward-backward stochastic differential equations (FBSDE) with an external optimality condition. We extend the numerical scheme of (Delarue and Menozzi, 2006) by a Newton-Raphson method to solve the FBSDE system and the optimality condition simultaneously. As far as the authors are aware, this is the first fully explicit numerical scheme for the solution of optimal control problems through the solution of the corresponding extended FBSDE system. We discuss possible numerical advantages to the DDP approach and consider an optimal investment-consumption problem as an example.",

keywords = "Forward-Backward stochastic differential equations, Optimal stochastic control, Stochastic maximum principle",

author = "Ludwig, {Stephan E.} and Sirignano, {Justin A} and Ruojun Huang and George Papanicolaou",

year = "2012",

language = "English (US)",

isbn = "9789898425973",

pages = "83--89",

booktitle = "ICORES 2012 - Proceedings of the 1st International Conference on Operations Research and Enterprise Systems",

note = "1st International Conference on Operations Research and Enterprise Systems, ICORES 2012 ; Conference date: 04-02-2012 Through 06-02-2012",

}

TY - GEN

T1 - A forward-backward algorithm for stochastic control problems

T2 - 1st International Conference on Operations Research and Enterprise Systems, ICORES 2012

AU - Ludwig, Stephan E.

AU - Sirignano, Justin A

AU - Huang, Ruojun

AU - Papanicolaou, George

PY - 2012

Y1 - 2012

N2 - An algorithm for solving continuous-time stochastic optimal control problems is presented. The numerical scheme is based on the stochastic maximum principle (SMP) as an alternative to the widely studied dynamic programming principle (DDP). By using the SMP, (Peng, 1990) obtained a system of coupled forward-backward stochastic differential equations (FBSDE) with an external optimality condition. We extend the numerical scheme of (Delarue and Menozzi, 2006) by a Newton-Raphson method to solve the FBSDE system and the optimality condition simultaneously. As far as the authors are aware, this is the first fully explicit numerical scheme for the solution of optimal control problems through the solution of the corresponding extended FBSDE system. We discuss possible numerical advantages to the DDP approach and consider an optimal investment-consumption problem as an example.

AB - An algorithm for solving continuous-time stochastic optimal control problems is presented. The numerical scheme is based on the stochastic maximum principle (SMP) as an alternative to the widely studied dynamic programming principle (DDP). By using the SMP, (Peng, 1990) obtained a system of coupled forward-backward stochastic differential equations (FBSDE) with an external optimality condition. We extend the numerical scheme of (Delarue and Menozzi, 2006) by a Newton-Raphson method to solve the FBSDE system and the optimality condition simultaneously. As far as the authors are aware, this is the first fully explicit numerical scheme for the solution of optimal control problems through the solution of the corresponding extended FBSDE system. We discuss possible numerical advantages to the DDP approach and consider an optimal investment-consumption problem as an example.

KW - Forward-Backward stochastic differential equations

KW - Optimal stochastic control

KW - Stochastic maximum principle

UR - http://www.scopus.com/inward/record.url?scp=84861996353&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84861996353&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84861996353

SN - 9789898425973

SP - 83

EP - 89

BT - ICORES 2012 - Proceedings of the 1st International Conference on Operations Research and Enterprise Systems

Y2 - 4 February 2012 through 6 February 2012

ER -

A forward-backward algorithm for stochastic control problems: Using the Stochastic Maximum Principle as an Alternative to Dynamic Programming

Abstract

Other

Keywords

ASJC Scopus subject areas

Library availability

Related links

Fingerprint

Cite this