Secure Linear Quadratic Regulator Using Sparse Model-Free Reinforcement Learning

Bahare Kiumarsi, Tamer Basar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a resilient model-free reinforcement learning solution to linear quadratic regulator control of cyber-physical systems under sensor attacks. To guarantee resiliency to sensor attacks, a sparse least-squares optimization is introduced to solve the Bellman equation. While the Bellman equation does not involve any dynamics, it implicitly solves a Lyapunov equation which depends on the system dynamics. Thus, if the data are corrupted and do not follow the dynamics, that causes an error in the Bellman equation. Therefore, assuming a strong system observability, i.e., s-sparse observability, the proposed sparse optimization assures that the data from compromised sensors that lead to a sizable error in the Bellman equation have no effect in reconstructing the state of the system, and, thus on evaluation of the policy. That is, only sensory outputs that result in a small error in the Bellman equation affect the policy evaluation. Once the optimal control policy is found, it can be applied to the system, until a surprise signal depending on the Bellman error is activated to indicate a change caused by a new attack or a change in the system dynamics.

Original languageEnglish (US)
Title of host publication2019 IEEE 58th Conference on Decision and Control, CDC 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3641-3647
Number of pages7
ISBN (Electronic)9781728113982
DOIs
StatePublished - Dec 2019
Event58th IEEE Conference on Decision and Control, CDC 2019 - Nice, France
Duration: Dec 11 2019Dec 13 2019

Publication series

NameProceedings of the IEEE Conference on Decision and Control
Volume2019-December
ISSN (Print)0743-1546

Conference

Conference58th IEEE Conference on Decision and Control, CDC 2019
CountryFrance
CityNice
Period12/11/1912/13/19

Keywords

  • Linear quadratic regulator
  • Reinforcement Learning
  • Resilient control

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Modeling and Simulation
  • Control and Optimization

Fingerprint Dive into the research topics of 'Secure Linear Quadratic Regulator Using Sparse Model-Free Reinforcement Learning'. Together they form a unique fingerprint.

Cite this