Provably robust deep learning via adversarially trained smoothed classifiers

Hadi Salman; Greg Yang; Jerry Li; Pengchuan Zhang; Huan Zhang; Ilya Razenshteyn; Sébastien Bubeck

Provably robust deep learning via adversarially trained smoothed classifiers

Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, Sébastien Bubeck

Research output: Contribution to journal › Conference article › peer-review

Abstract

Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to `₂-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adapted attack for smoothed classifiers, and we show how this attack can be used in an adversarial training setting to boost the provable robustness of smoothed classifiers. We demonstrate through extensive experimentation that our method consistently outperforms all existing provably `₂-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable `₂-defenses. Moreover, we find that pre-training and semi-supervised learning boost adversarially trained smoothed classifiers even further.

Original language	English (US)
Journal	Advances in Neural Information Processing Systems
Volume	32
State	Published - 2019
Externally published	Yes
Event	33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019 - Vancouver, Canada Duration: Dec 8 2019 → Dec 14 2019

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Signal Processing

Library availability

Discover UIUC Full Text

Cite this

@article{f822e45eed6e461c91ed92121abc4305,

title = "Provably robust deep learning via adversarially trained smoothed classifiers",

abstract = "Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to `2-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adapted attack for smoothed classifiers, and we show how this attack can be used in an adversarial training setting to boost the provable robustness of smoothed classifiers. We demonstrate through extensive experimentation that our method consistently outperforms all existing provably `2-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable `2-defenses. Moreover, we find that pre-training and semi-supervised learning boost adversarially trained smoothed classifiers even further.",

author = "Hadi Salman and Greg Yang and Jerry Li and Pengchuan Zhang and Huan Zhang and Ilya Razenshteyn and S{\'e}bastien Bubeck",

note = "Publisher Copyright: {\textcopyright} 2019 Neural information processing systems foundation. All rights reserved.; 33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019 ; Conference date: 08-12-2019 Through 14-12-2019",

year = "2019",

language = "English (US)",

volume = "32",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

publisher = "Neural information processing systems foundation",

}

TY - JOUR

T1 - Provably robust deep learning via adversarially trained smoothed classifiers

AU - Salman, Hadi

AU - Yang, Greg

AU - Li, Jerry

AU - Zhang, Pengchuan

AU - Zhang, Huan

AU - Razenshteyn, Ilya

AU - Bubeck, Sébastien

PY - 2019

Y1 - 2019

N2 - Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to `2-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adapted attack for smoothed classifiers, and we show how this attack can be used in an adversarial training setting to boost the provable robustness of smoothed classifiers. We demonstrate through extensive experimentation that our method consistently outperforms all existing provably `2-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable `2-defenses. Moreover, we find that pre-training and semi-supervised learning boost adversarially trained smoothed classifiers even further.

AB - Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to `2-norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adapted attack for smoothed classifiers, and we show how this attack can be used in an adversarial training setting to boost the provable robustness of smoothed classifiers. We demonstrate through extensive experimentation that our method consistently outperforms all existing provably `2-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable `2-defenses. Moreover, we find that pre-training and semi-supervised learning boost adversarially trained smoothed classifiers even further.

UR - http://www.scopus.com/inward/record.url?scp=85090176285&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85090176285&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85090176285

SN - 1049-5258

VL - 32

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

T2 - 33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019

Y2 - 8 December 2019 through 14 December 2019

ER -

Provably robust deep learning via adversarially trained smoothed classifiers

Abstract

ASJC Scopus subject areas

Library availability

Related links

Fingerprint

Cite this