Efficient source separation using bitwise neural networks

Minje Kim, Paris Smaragdis

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Efficiency is one of the key issues in single-channel source separation systems due to the fact that they are often employed for real-time processing. More computationally demanding approaches tend to produce better results, but often not fast enough to be deployed in practical systems. For example, as opposed to the iterative separation algorithms using source-specific dictionaries, a Deep Neural Network (DNN) performs separation via an iteration-free feedforward process. However, even the feedforward process can be very complex depending on the size of the network. In this chapter, we introduce Bitwise Neural Networks (BNN) as an extremely compact form of neural networks, whose feedforward pass uses only efficient bitwise operations (e.g. XNOR instead of multiplication) on binary weight matrices and quantized input signals. As a result, we show that BNNs can perform denoising with a negnigible loss of quality as compared to a corresponding network with the same structure, while reducing the network complexity significantly.

Original languageEnglish (US)
Title of host publicationSignals and Communication Technology
PublisherSpringer
Pages187-206
Number of pages20
DOIs
StatePublished - Jan 1 2018

Publication series

NameSignals and Communication Technology
ISSN (Print)1860-4862
ISSN (Electronic)1860-4870

Fingerprint

Source separation
Feedforward neural networks
Glossaries
Neural networks
Processing
Deep neural networks

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Computer Networks and Communications

Cite this

Kim, M., & Smaragdis, P. (2018). Efficient source separation using bitwise neural networks. In Signals and Communication Technology (pp. 187-206). (Signals and Communication Technology). Springer. https://doi.org/10.1007/978-3-319-73031-8_8

Efficient source separation using bitwise neural networks. / Kim, Minje; Smaragdis, Paris.

Signals and Communication Technology. Springer, 2018. p. 187-206 (Signals and Communication Technology).

Research output: Chapter in Book/Report/Conference proceedingChapter

Kim, M & Smaragdis, P 2018, Efficient source separation using bitwise neural networks. in Signals and Communication Technology. Signals and Communication Technology, Springer, pp. 187-206. https://doi.org/10.1007/978-3-319-73031-8_8
Kim M, Smaragdis P. Efficient source separation using bitwise neural networks. In Signals and Communication Technology. Springer. 2018. p. 187-206. (Signals and Communication Technology). https://doi.org/10.1007/978-3-319-73031-8_8
Kim, Minje ; Smaragdis, Paris. / Efficient source separation using bitwise neural networks. Signals and Communication Technology. Springer, 2018. pp. 187-206 (Signals and Communication Technology).
@inbook{7a93ded53658428da0b56772fbba8a79,
title = "Efficient source separation using bitwise neural networks",
abstract = "Efficiency is one of the key issues in single-channel source separation systems due to the fact that they are often employed for real-time processing. More computationally demanding approaches tend to produce better results, but often not fast enough to be deployed in practical systems. For example, as opposed to the iterative separation algorithms using source-specific dictionaries, a Deep Neural Network (DNN) performs separation via an iteration-free feedforward process. However, even the feedforward process can be very complex depending on the size of the network. In this chapter, we introduce Bitwise Neural Networks (BNN) as an extremely compact form of neural networks, whose feedforward pass uses only efficient bitwise operations (e.g. XNOR instead of multiplication) on binary weight matrices and quantized input signals. As a result, we show that BNNs can perform denoising with a negnigible loss of quality as compared to a corresponding network with the same structure, while reducing the network complexity significantly.",
author = "Minje Kim and Paris Smaragdis",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-319-73031-8_8",
language = "English (US)",
series = "Signals and Communication Technology",
publisher = "Springer",
pages = "187--206",
booktitle = "Signals and Communication Technology",

}

TY - CHAP

T1 - Efficient source separation using bitwise neural networks

AU - Kim, Minje

AU - Smaragdis, Paris

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Efficiency is one of the key issues in single-channel source separation systems due to the fact that they are often employed for real-time processing. More computationally demanding approaches tend to produce better results, but often not fast enough to be deployed in practical systems. For example, as opposed to the iterative separation algorithms using source-specific dictionaries, a Deep Neural Network (DNN) performs separation via an iteration-free feedforward process. However, even the feedforward process can be very complex depending on the size of the network. In this chapter, we introduce Bitwise Neural Networks (BNN) as an extremely compact form of neural networks, whose feedforward pass uses only efficient bitwise operations (e.g. XNOR instead of multiplication) on binary weight matrices and quantized input signals. As a result, we show that BNNs can perform denoising with a negnigible loss of quality as compared to a corresponding network with the same structure, while reducing the network complexity significantly.

AB - Efficiency is one of the key issues in single-channel source separation systems due to the fact that they are often employed for real-time processing. More computationally demanding approaches tend to produce better results, but often not fast enough to be deployed in practical systems. For example, as opposed to the iterative separation algorithms using source-specific dictionaries, a Deep Neural Network (DNN) performs separation via an iteration-free feedforward process. However, even the feedforward process can be very complex depending on the size of the network. In this chapter, we introduce Bitwise Neural Networks (BNN) as an extremely compact form of neural networks, whose feedforward pass uses only efficient bitwise operations (e.g. XNOR instead of multiplication) on binary weight matrices and quantized input signals. As a result, we show that BNNs can perform denoising with a negnigible loss of quality as compared to a corresponding network with the same structure, while reducing the network complexity significantly.

UR - http://www.scopus.com/inward/record.url?scp=85063225680&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063225680&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-73031-8_8

DO - 10.1007/978-3-319-73031-8_8

M3 - Chapter

AN - SCOPUS:85063225680

T3 - Signals and Communication Technology

SP - 187

EP - 206

BT - Signals and Communication Technology

PB - Springer

ER -