Continual learning of new sound classes using generative replay

Zhepei Wang, Cem Subakan, Efthymios Tzinis, Paris Smaragdis, Laurent Charlin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Continual learning consists in incrementally training a model on a sequence of datasets and testing on the union of all datasets. In this paper, we examine continual learning for the problem of sound classification, in which we wish to refine already trained models to learn new sound classes. In practice one does not want to maintain all past training data and retrain from scratch, but naively updating a model with new data(sets) results in a degradation of already learned tasks, which is referred to as "catastrophic forgetting." We develop a generative replay procedure for generating training audio spectrogram data, in place of keeping older training datasets. We show that by incrementally refining a classifier with generative replay a generator that is 4% of the size of all previous training data matches the performance of refining the classifier keeping 20% of all previous training data. We thus conclude that we can extend a trained sound classifier to learn new classes without having to keep previously used datasets.

Original languageEnglish (US)
Title of host publication2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages308-312
Number of pages5
ISBN (Electronic)9781728111230
DOIs
StatePublished - Oct 2019
Event2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2019 - New Paltz, United States
Duration: Oct 20 2019Oct 23 2019

Publication series

NameIEEE Workshop on Applications of Signal Processing to Audio and Acoustics
Volume2019-October
ISSN (Print)1931-1168
ISSN (Electronic)1947-1629

Conference

Conference2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2019
CountryUnited States
CityNew Paltz
Period10/20/1910/23/19

Keywords

  • continual learning
  • generative replay
  • neural networks
  • Sound classification

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Continual learning of new sound classes using generative replay'. Together they form a unique fingerprint.

  • Cite this

    Wang, Z., Subakan, C., Tzinis, E., Smaragdis, P., & Charlin, L. (2019). Continual learning of new sound classes using generative replay. In 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2019 (pp. 308-312). [8937236] (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics; Vol. 2019-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WASPAA.2019.8937236