Ver publicación 

Mostrar el registro sencillo de la publicación

Multi-channel speech enhancement using labelled random finite sets and a neural beamformer in cocktail party scenario

dc.contributor.authorDatta, Jayanta
dc.contributor.authorDehghan Firoozabadi, Ali
dc.contributor.authorZabala-Blanco, David
dc.contributor.authorCastillo-Soria, Francisco R.
dc.date.accessioned2025-06-12T14:15:43Z
dc.date.available2025-06-12T14:15:43Z
dc.date.issued2025
dc.identifier.urihttp://repositorio.ucm.cl/handle/ucm/6125
dc.description.abstractIn this research, a multi-channel target speech enhancement scheme is proposed that is based on deep learning (DL) architecture and assisted by multi-source tracking using a labeled random finite set (RFS) framework. A neural network based on minimum variance distortionless response (MVDR) beamformer is considered as the beamformer of choice, where a residual dense convolutional graph-U-Net is applied in a generative adversarial network (GAN) setting to model the beamformer for target speech enhancement under reverberant conditions involving multiple moving speech sources. The input dataset for this neural architecture is constructed by applying multi-source tracking using multi-sensor generalized labeled multi-Bernoulli (MS-GLMB) filtering, which belongs to the labeled RFS framework, to obtain estimations of the sources’ positions and the associated labels (corresponding to each source) at each time frame with high accuracy under the effect of undesirable factors like reverberation and background noise. The tracked sources’ positions and associated labels help to correctly discriminate the target source from the interferers across all time frames and generate time–frequency (T-F) masks corresponding to the target source from the output of a time-varying, minimum variance distortionless response (MVDR) beamformer. These T-F masks constitute the target label set used to train the proposed deep neural architecture to perform target speech enhancement. The exploitation of MS-GLMB filtering and a time-varying MVDR beamformer help in providing the spatial information of the sources, in addition to the spectral information, within the neural speech enhancement framework during the training phase. Moreover, the application of the GAN framework takes advantage of adversarial optimization as an alternative to maximum likelihood (ML)-based frameworks, which further boosts the performance of target speech enhancement under reverberant conditions. The computer simulations demonstrate that the proposed approach leads to better target speech enhancement performance compared with existing state-of-the-art DL-based methodologies which do not incorporate the labeled RFS-based approach, something which is evident from the 75% ESTOI and PESQ of 2.70 achieved by the proposed approach as compared with the 46.74% ESTOI and PESQ of 1.84 achieved by Mask-MVDR with self-attention mechanism at a reverberation time (RT60) of 550 ms.es_CL
dc.language.isoenes_CL
dc.rightsAtribución-NoComercial-SinDerivadas 3.0 Chile*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/3.0/cl/*
dc.sourceApplied Sciences, 15(6), 2944es_CL
dc.subjectSRP-PHATes_CL
dc.subjectDeep learninges_CL
dc.subjectMicrophone arrayes_CL
dc.subjectMS-GLMB filteringes_CL
dc.subjectBeamforminges_CL
dc.titleMulti-channel speech enhancement using labelled random finite sets and a neural beamformer in cocktail party scenarioes_CL
dc.typeArticlees_CL
dc.ucm.facultadFacultad de Ciencias de la Ingenieríaes_CL
dc.ucm.indexacionScopuses_CL
dc.ucm.indexacionIsies_CL
dc.ucm.urimdpi.com/2076-3417/15/6/2944es_CL
dc.ucm.doidoi.org/10.3390/app15062944es_CL


Ficheros en la publicación

FicherosTamañoFormatoVer

No hay ficheros asociados a esta publicación.

Esta publicación aparece en la(s) siguiente(s) colección(ones)

Mostrar el registro sencillo de la publicación

Atribución-NoComercial-SinDerivadas 3.0 Chile
Excepto si se señala otra cosa, la licencia de la publicación se describe como Atribución-NoComercial-SinDerivadas 3.0 Chile