TY - JOUR
T1 - A general network architecture for sound event localization and detection using transfer learning and recurrent neural network
AU - Nguyen, Thi Ngoc Tho
AU - Nguyen, Ngoc Khanh
AU - Phan, Huy
AU - Pham, Lam
AU - Ooi, Kenneth
AU - Jones, Douglas L.
AU - Gan, Woon Seng
N1 - Funding Information:
∗This research was conducted at Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU), which is a collaboration between Singapore Telecommunications Limited (Singtel) and Nanyang Technological University (NTU) that is funded by the Singapore Government through the Industry Alignment Fund - Industry Collaboration Projects Grant.
Publisher Copyright:
© 2021 IEEE
PY - 2021
Y1 - 2021
N2 - Polyphonic sound event detection and localization (SELD) task is challenging because it is difficult to jointly optimize sound event detection (SED) and direction-of-arrival (DOA) estimation in the same network. We propose a general network architecture for SELD in which the SELD network comprises sub-networks that are pretrained to solve SED and DOA estimation independently, and a recurrent layer that combines the SED and DOA estimation outputs into SELD outputs. The recurrent layer does the alignment between the sound classes and DOAs of sound events while being unaware of how these outputs are produced by the upstream SED and DOA estimation algorithms. This simple network architecture is compatible with different existing SED and DOA estimation algorithms. It is highly practical since the sub-networks can be improved independently. The experimental results using the DCASE 2020 SELD dataset show that the performances of our proposed network architecture using different SED and DOA estimation algorithms and different audio formats are competitive with other state-of-the-art SELD algorithms. The source code for the proposed SELD network architecture is available at Github.
AB - Polyphonic sound event detection and localization (SELD) task is challenging because it is difficult to jointly optimize sound event detection (SED) and direction-of-arrival (DOA) estimation in the same network. We propose a general network architecture for SELD in which the SELD network comprises sub-networks that are pretrained to solve SED and DOA estimation independently, and a recurrent layer that combines the SED and DOA estimation outputs into SELD outputs. The recurrent layer does the alignment between the sound classes and DOAs of sound events while being unaware of how these outputs are produced by the upstream SED and DOA estimation algorithms. This simple network architecture is compatible with different existing SED and DOA estimation algorithms. It is highly practical since the sub-networks can be improved independently. The experimental results using the DCASE 2020 SELD dataset show that the performances of our proposed network architecture using different SED and DOA estimation algorithms and different audio formats are competitive with other state-of-the-art SELD algorithms. The source code for the proposed SELD network architecture is available at Github.
KW - Direction-of-arrival estimation
KW - Network architecture
KW - Recurrent neural network
KW - Sound event detection
UR - http://www.scopus.com/inward/record.url?scp=85109062221&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85109062221&partnerID=8YFLogxK
U2 - 10.1109/ICASSP39728.2021.9414602
DO - 10.1109/ICASSP39728.2021.9414602
M3 - Conference article
AN - SCOPUS:85109062221
SN - 0736-7791
VL - 2021-June
SP - 935
EP - 939
JO - Proceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing
JF - Proceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing
T2 - 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021
Y2 - 6 June 2021 through 11 June 2021
ER -