Batch-to-Batch Finite-Horizon LQ Control for Unknown Discrete-Time Linear Systems Via Stochastic Extremum Seeking

Shu Jun Liu, Miroslav Krstic, Tamer Basar

Research output: Contribution to journalArticlepeer-review

Abstract

We employ our recent discrete-time stochastic averaging theorems and stochastic extremum seeking to iteratively (batch-to-batch) optimize open-loop control sequences for unknown but reachable discrete-time linear systems with a scalar input and without known system dimension, for a cost that is quadratic in the measurable output and the input. First, for a multivariable gradient-based stochastic extremum seeking algorithm we prove local exponential convergence to the optimal open-loop control sequence. Second, to remove the convergence rate's dependence on the Hessian matrix of the cost function, which is unknown since the system's model (the system matrices $(A,B,C)$) is unknown, we develop a multivariable discrete-time Newton-based stochastic extremum seeking method, design the Newton-based algorithm for the iteration of the input sequence, and prove local exponential convergence to the optimal input sequence. Finally, two simulation examples are given to illustrate the effectiveness of the two methods.

Original languageEnglish (US)
Article number7605544
Pages (from-to)4116-4123
Number of pages8
JournalIEEE Transactions on Automatic Control
Volume62
Issue number8
DOIs
StatePublished - Aug 2017

Keywords

  • LQ control
  • stochastic averaging
  • stochastic extremum seeking

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Batch-to-Batch Finite-Horizon LQ Control for Unknown Discrete-Time Linear Systems Via Stochastic Extremum Seeking'. Together they form a unique fingerprint.

Cite this