TY - JOUR

T1 - Stochastic gradient descent in continuous time

T2 - A central limit theorem

AU - Sirignano, Justin

AU - Spiliopoulos, Konstantinos

N1 - Funding Information:
Research of K. Spiliopoulos supported in part by the National Science Foundation [DMS 1550918].The authors thank seminar participants at Princeton University and the University of Colorado Boulder for their comments.
Publisher Copyright:
© 2020 The Author(s).

PY - 2020/6

Y1 - 2020/6

N2 - Stochastic gradient descent in continuous time (SGDCT) provides a computa-tionally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter updates occur in continuous time and satisfy a stochastic differential equation. This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well. An Lp convergence rate is also proven for the algorithm in the strongly convex case. The mathematical analysis lies at the intersection of stochastic analysis and statistical learning.

AB - Stochastic gradient descent in continuous time (SGDCT) provides a computa-tionally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter updates occur in continuous time and satisfy a stochastic differential equation. This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well. An Lp convergence rate is also proven for the algorithm in the strongly convex case. The mathematical analysis lies at the intersection of stochastic analysis and statistical learning.

KW - Central limit theorem

KW - Machine learning

KW - Statistical learning

KW - Stochastic differential equations

KW - Stochastic gradient descent

UR - http://www.scopus.com/inward/record.url?scp=85090992702&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85090992702&partnerID=8YFLogxK

U2 - 10.1287/stsy.2019.0050

DO - 10.1287/stsy.2019.0050

M3 - Article

AN - SCOPUS:85090992702

VL - 10

SP - 124

EP - 151

JO - Stochastic Systems

JF - Stochastic Systems

SN - 1946-5238

IS - 2

ER -