TY - JOUR
T1 - Adding one neuron can eliminate all bad local minima
AU - Liang, Shiyu
AU - Sun, Ruoyu
AU - Lee, Jason D.
AU - Srikant, R.
N1 - Funding Information:
Research is supported by the following grants: USDA/NSF CPS grant AG 2018-67007-2837, NSF NeTS 1718203, NSF CPS ECCS 1739189, DTRA Grant DTRA grant HDTRA1-15-1-0003, NSF CCF 1755847 and a start-up grant from Dept. of ISE, University of Illinois Urbana-Champaign.
PY - 2018
Y1 - 2018
N2 - One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection to the output, or one special neuron per layer, every local minimum is a global minimum.
AB - One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection to the output, or one special neuron per layer, every local minimum is a global minimum.
UR - http://www.scopus.com/inward/record.url?scp=85064842273&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85064842273&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85064842273
SN - 1049-5258
VL - 2018-December
SP - 4350
EP - 4360
JO - Advances in Neural Information Processing Systems
JF - Advances in Neural Information Processing Systems
T2 - 32nd Conference on Neural Information Processing Systems, NeurIPS 2018
Y2 - 2 December 2018 through 8 December 2018
ER -