A Robust Accelerated Optimization Algorithm for Strongly Convex Functions

Saman Cyrus, Bin Hu, Bryan Van Scoy, Laurent Lessard

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This work proposes an accelerated first-order algorithm we call the Robust Momentum Method for optimizing smooth strongly convex functions. The algorithm has a single scalar parameter that can be tuned to trade off robustness to gradient noise versus worst-case convergence rate. At one extreme, the algorithm is faster than Nesterov's Fast Gradient Method by a constant factor but more fragile to noise. At the other extreme, the algorithm reduces to the Gradient Method and is very robust to noise. The algorithm design technique is inspired by methods from classical control theory and the resulting algorithm has a simple analytical form. Algorithm performance is verified on a series of numerical simulations in both noise-free and relative gradient noise cases.

Original languageEnglish (US)
Title of host publication2018 Annual American Control Conference, ACC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1376-1381
Number of pages6
ISBN (Print)9781538654286
DOIs
StatePublished - Aug 9 2018
Externally publishedYes
Event2018 Annual American Control Conference, ACC 2018 - Milwauke, United States
Duration: Jun 27 2018Jun 29 2018

Publication series

NameProceedings of the American Control Conference
Volume2018-June
ISSN (Print)0743-1619

Other

Other2018 Annual American Control Conference, ACC 2018
Country/TerritoryUnited States
CityMilwauke
Period6/27/186/29/18

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'A Robust Accelerated Optimization Algorithm for Strongly Convex Functions'. Together they form a unique fingerprint.

Cite this