Batch-to-Batch Finite-Horizon LQ Control for Unknown Discrete-Time Linear Systems Via Stochastic Extremum Seeking

We employ our recent discrete-time stochastic averaging theorems and stochastic extremum seeking to iteratively (batch-to-batch) optimize open-loop control sequences for unknown but reachable discrete-time linear systems with a scalar input and without known system dimension, for a cost that is quadratic in the measurable output and the input. First, for a multivariable gradient-based stochastic extremum seeking algorithm we prove local exponential convergence to the optimal open-loop control sequence. Second, to remove the convergence rate's dependence on the Hessian matrix of the cost function, which is unknown since the system's model (the system matrices $(A,B,C)$) is unknown, we develop a multivariable discrete-time Newton-based stochastic extremum seeking method, design the Newton-based algorithm for the iteration of the input sequence, and prove local exponential convergence to the optimal input sequence. Finally, two simulation examples are given to illustrate the effectiveness of the two methods.