Controlling a Markov Decision Process with an Abrupt Change in the Transition Kernel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider the control of a Markov decision process (MDP) that undergoes an abrupt change in its transition kernel (mode). We formulate the problem of minimizing regret under control switching based on mode change detection, compared to a mode-observing controller, as an optimal stopping problem. Using a sequence of approximations, we reduce it to a quickest change detection (QCD) problem with Markovian data, for which we characterize a state-dependent threshold-type optimal change detection policy. Numerical experiments illustrate various properties of our control-switching policy.

Original languageEnglish (US)
Title of host publication2023 American Control Conference, ACC 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3401-3408
Number of pages8
ISBN (Electronic)9798350328066
DOIs
StatePublished - 2023
Event2023 American Control Conference, ACC 2023 - San Diego, United States
Duration: May 31 2023Jun 2 2023

Publication series

NameProceedings of the American Control Conference
Volume2023-May
ISSN (Print)0743-1619

Conference

Conference2023 American Control Conference, ACC 2023
Country/TerritoryUnited States
CitySan Diego
Period5/31/236/2/23

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Controlling a Markov Decision Process with an Abrupt Change in the Transition Kernel'. Together they form a unique fingerprint.

Cite this