The Global Landscape of Neural Networks: An Overview

Ruoyu Sun; Dawei Li; Shiyu Liang; Tian Ding; Rayadurgam Srikant

doi:10.1109/MSP.2020.3004124

The Global Landscape of Neural Networks: An Overview

Ruoyu Sun, Dawei Li, Shiyu Liang, Tian Ding, Rayadurgam Srikant

Research output: Contribution to journal › Article › peer-review

Abstract

One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

Original language	English (US)
Article number	9194023
Pages (from-to)	95-108
Number of pages	14
Journal	IEEE Signal Processing Magazine
Volume	37
Issue number	5
DOIs	https://doi.org/10.1109/MSP.2020.3004124
State	Published - Sep 2020

ASJC Scopus subject areas

Signal Processing
Electrical and Electronic Engineering
Applied Mathematics

Online availability

10.1109/MSP.2020.3004124

Library availability

Discover UIUC Full Text

Cite this

@article{13300cd29ef64388a8f273f172532327,

title = "The Global Landscape of Neural Networks: An Overview",

abstract = "One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.",

author = "Ruoyu Sun and Dawei Li and Shiyu Liang and Tian Ding and Rayadurgam Srikant",

note = "Publisher Copyright: {\textcopyright} 1991-2012 IEEE.",

year = "2020",

month = sep,

doi = "10.1109/MSP.2020.3004124",

language = "English (US)",

volume = "37",

pages = "95--108",

journal = "IEEE Signal Processing Magazine",

issn = "1053-5888",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "5",

}

TY - JOUR

T1 - The Global Landscape of Neural Networks

T2 - An Overview

AU - Sun, Ruoyu

AU - Li, Dawei

AU - Liang, Shiyu

AU - Ding, Tian

AU - Srikant, Rayadurgam

PY - 2020/9

Y1 - 2020/9

N2 - One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

AB - One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.

UR - http://www.scopus.com/inward/record.url?scp=85091317036&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85091317036&partnerID=8YFLogxK

U2 - 10.1109/MSP.2020.3004124

DO - 10.1109/MSP.2020.3004124

M3 - Article

AN - SCOPUS:85091317036

SN - 1053-5888

VL - 37

SP - 95

EP - 108

JO - IEEE Signal Processing Magazine

JF - IEEE Signal Processing Magazine

IS - 5

M1 - 9194023

ER -

The Global Landscape of Neural Networks: An Overview

Abstract

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this