Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach

Dušan M. Stipanović, Mirna N. Kapetina, Milan R. Rapaić, Boris Murmann

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known in machine learning applications. From the technical perspective, the analyses exploit the systems similarities to a convex combination of discrete-time systems, where one of the systems is trivial, and thus, the overall properties are mostly dependent on the other one. Stability results are formulated for the nonlinear system and its linearization with respect to the systems, in general, multiple equilibria. To motivate and illustrate the potential of these results in applications, some particular results are derived for the gated recurrent unit neural network models and a connection between local stability analysis and learning is provided.

Original languageEnglish (US)
Pages (from-to)291-306
Number of pages16
JournalJournal of Optimization Theory and Applications
Volume188
Issue number1
DOIs
StatePublished - Jan 2021

Keywords

  • Difference equations
  • Neural networks
  • Stability

ASJC Scopus subject areas

  • Control and Optimization
  • Management Science and Operations Research
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach'. Together they form a unique fingerprint.

Cite this