Control interpretations for first-order optimization methods

Bin Hu, Laurent Lessard

Research output: Chapter in Book/Report/Conference proceedingConference contribution


First-order iterative optimization methods play a fundamental role in large scale optimization and machine learning. This paper presents control interpretations for such optimization methods. First, we give loop-shaping interpretations for several existing optimization methods and show that they are composed of basic control elements such as PID and lag compensators. Next, we apply the small gain theorem to draw a connection between the convergence rate analysis of optimization methods and the input-output gain computations of certain complementary sensitivity functions. These connections suggest that standard classical control synthesis tools may be brought to bear on the design of optimization algorithms.

Original languageEnglish (US)
Title of host publication2017 American Control Conference, ACC 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781509059928
StatePublished - Jun 29 2017
Externally publishedYes
Event2017 American Control Conference, ACC 2017 - Seattle, United States
Duration: May 24 2017May 26 2017

Publication series

NameProceedings of the American Control Conference
ISSN (Print)0743-1619


Other2017 American Control Conference, ACC 2017
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Electrical and Electronic Engineering


Dive into the research topics of 'Control interpretations for first-order optimization methods'. Together they form a unique fingerprint.

Cite this