TY - GEN
T1 - The Role of Regularization in Overparameterized Neural Networks*
AU - Satpathi, Siddhartha
AU - Gupta, Harsh
AU - Liang, Shiyu
AU - Srikant, R.
N1 - Funding Information:
*This work was supported by NSF Grants NeTS 1718203, CPS ECCS 1739189, ECCS 16-09370, CCF 1934986, NSF/USDA Grant AG 2018-67007-28379, ARO W911NF-19-1-0379, and ONR Grant Navy N00014-19-1-2566 1S Satpathi, H Gupta, S Liang and R Srikant are with the Department of Electrical and Computer Engineering and the Coordinated Science Lab at the University of Illinois at Urbana Champaign, Champaign, IL 61820 USA.
PY - 2020/12/14
Y1 - 2020/12/14
N2 - In this paper, we consider gradient descent on a regularized loss function for training an overparametrized neural network. We model the algorithm as an ODE and show how overparameterization and regularization work together to provide the right tradeoff between training and generalization errors.
AB - In this paper, we consider gradient descent on a regularized loss function for training an overparametrized neural network. We model the algorithm as an ODE and show how overparameterization and regularization work together to provide the right tradeoff between training and generalization errors.
UR - http://www.scopus.com/inward/record.url?scp=85099886171&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85099886171&partnerID=8YFLogxK
U2 - 10.1109/CDC42340.2020.9304386
DO - 10.1109/CDC42340.2020.9304386
M3 - Conference contribution
AN - SCOPUS:85099886171
T3 - Proceedings of the IEEE Conference on Decision and Control
SP - 4683
EP - 4688
BT - 2020 59th IEEE Conference on Decision and Control, CDC 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 59th IEEE Conference on Decision and Control, CDC 2020
Y2 - 14 December 2020 through 18 December 2020
ER -