Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Data

Efthymios Tzinis, Jonah Casebeer, Zhepei Wang, Paris Smaragdis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose FedEnhance, an unsupervised federated learning (FL) approach for speech enhancement and separation with non-IID distributed data across multiple clients. We simulate a realworld scenario where each client only has access to a few noisy recordings from a limited and disjoint number of speakers (hence non-IID). Each client trains their model in isolation using mixture invariant training while periodically providing updates to a central server. Our experiments show that our approach achieves competitive enhancement performance compared to IID training on a single device and that we can further facilitate the convergence speed and the overall performance using transfer learning on the server-side. Moreover, we show that we can effectively combine updates from clients trained locally with supervised and unsupervised losses. We also release a new dataset LibriFSD50K and its creation recipe in order to facilitate FL research for source separation problems.

Original languageEnglish (US)
Title of host publication2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages46-50
Number of pages5
ISBN (Electronic)9781665448703
DOIs
StatePublished - 2021
Externally publishedYes
Event2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2021 - New Paltz, United States
Duration: Oct 17 2021Oct 20 2021

Publication series

NameIEEE Workshop on Applications of Signal Processing to Audio and Acoustics
Volume2021-October
ISSN (Print)1931-1168
ISSN (Electronic)1947-1629

Conference

Conference2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2021
Country/TerritoryUnited States
CityNew Paltz
Period10/17/2110/20/21

Keywords

  • Speech enhancement
  • federated learning
  • non-IID learning
  • source separation
  • unsupervised learning

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Data'. Together they form a unique fingerprint.

Cite this