Joint Optimization of Deep Neural Network-Based Dereverberation and Beamforming for Sound Event Detection in Multi-Channel Environmentsopen access
- Authors
- Noh, Kyoungjin; Chang, Joon-Hyuk
- Issue Date
- Apr-2020
- Publisher
- MDPI
- Keywords
- sound event detection; dereverberation; acoustic beamforming; convolutional recurrent neural network; joint optimization
- Citation
- SENSORS, v.20, no.7, pp.1 - 13
- Indexed
- SCIE
SCOPUS
- Journal Title
- SENSORS
- Volume
- 20
- Number
- 7
- Start Page
- 1
- End Page
- 13
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/9920
- DOI
- 10.3390/s20071883
- ISSN
- 1424-8220
- Abstract
- In this paper, we propose joint optimization of deep neural network (DNN)-supported dereverberation and beamforming for the convolutional recurrent neural network (CRNN)-based sound event detection (SED) in multi-channel environments. First, the short-time Fourier transform (STFT) coefficients are calculated from multi-channel audio signals under the noisy and reverberant environments, which are then enhanced by the DNN-supported weighted prediction error (WPE) dereverberation with the estimated masks. Next, the STFT coefficients of the dereverberated multi-channel audio signals are conveyed to the DNN-supported minimum variance distortionless response (MVDR) beamformer in which DNN-supported MVDR beamforming is carried out with the source and noise masks estimated by the DNN. As a result, the single-channel enhanced STFT coefficients are shown at the output and tossed to the CRNN-based SED system, and then, the three modules are jointly trained by the single loss function designed for SED. Furthermore, to ease the difficulty of training a deep learning model for SED caused by the imbalance in the amount of data for each class, the focal loss is used as a loss function. Experimental results show that joint training of DNN-supported dereverberation and beamforming with the SED model under the supervision of focal loss significantly improves the performance under the noisy and reverberant environments.
- Files in This Item
-
- Appears in
Collections - 서울 공과대학 > 서울 융합전자공학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/9920)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.