Editing State-of-the-art (section)

=== Article summaries ===
* Article summaries and analyses: Each article receives a subsection including a summary (reference to RQ and hypothesis), critical analysis, and discuss its relevance to your theme.

==== Donahue, C., Li, B., & Prabhavalkar, R. (2018). ''Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition'' (arXiv:1711.05747). arXiv. <nowiki>http://arxiv.org/abs/1711.05747</nowiki> ====
* '''Summary:''' This paper investigates the application of Generative Adversarial Networks (GANs) for speech enhancement, particularly for improving the noise robustness of ASR systems. Through comprehensive experiments, it introduces a frequency-domain approach (FSEGAN) to speech enhancement that shows improved ASR performance over traditional time-domain methods (SEGAN).
* '''RQ:''' Can GAN-based speech enhancement techniques effectively improve the noise robustness of ASR systems compared to traditional noise suppression methods?
* '''Hypothesis:''' The paper hypothesizes that GAN-based speech enhancement, especially when operating on log-Mel filterbank spectra rather than waveforms, will provide significant improvements in ASR system performance in noisy conditions.
* '''Conclusion:''' The study concludes that while GAN-based speech enhancement methods, particularly FSEGAN, can improve ASR performance in noisy conditions, they do not outperform multi-style training (MTR) methods. Retraining the ASR system using both the original noisy audio and the audio improved by GANs leads to better performance. This suggests that GAN-enhanced audio could be a valuable addition to improve ASR systems when used alongside the original noisy input.
* '''Critical observations:''' SEGAN, while effective in removing additive noise, is less effective in reverberant conditions compared to the frequency-domain approach (FSEGAN). On the contrast, FSEGAN significantly improves ASR performance but does not outperform traditional MTR alone. However, combining noisy and enhanced features for retraining enhances the system's robustness.
* '''Relevance:''' This article is relevant to techniques used to bolster the performance of ASR systems, highlighting the significant potential of innovating GAN-based model in this field.

==== Y. Koizumi, H. Zen, S. Karita, et al. (2023). Miipher: A robust speech restoration model integrating self-supervised speech and text representations, arXiv:2303.01664. ====
*'''Summary:''' The paper presents Miipher, a robust speech restoration (SR) model that integrates self-supervised speech and text representations to enhance the quality of degraded speech signals. The model is designed to address two primary challenges in SR: phoneme masking and deletion.
* '''RQ:''' How to develop a robust speech restoration (SR) model that can convert degraded speech signals into high-quality ones, with a focus on handling difficult degradations such as phoneme masking and deletion?
* '''Hypothesis:''' The proposed SR model, Miipher, will be robust against various audio degradations and enable the training of high-quality text-to-speech (TTS) models from restored speech samples.
* '''Conclusion:''' The study concludes that Miipher is effective in restoring speech samples in-the-wild and can increase the value of speech samples by improving their quality as training data for speech generation tasks.
* '''Critical observations:''' The use of w2v-BERT features significantly improves SR performance compared to log-mel spectrogram-based methods, the effectiveness of PnG-BERT features in preserving text content, and the importance of speaker embeddings for retaining speaker characteristics in restored speech.
* '''Relevance:''' The relevance of this study is significant for the field of speech enhancement/restoration, as it demonstrates a method to enhance the quality of existing speech datasets and expand the potential applications of non-studio speech recordings.

==== Vinith Kishore, Nitya Tiwari, and Periyasamy Paramasivam. “Improved Speech Enhancement Using TCN with Multiple Encoder-Decoder Layers”. In: Interspeech 2020. ISCA. 2020, pp. 4531–4535. doi: 10.21437/Interspeech.2020-3122. url: <nowiki>https://doi.org/10.21437/Interspeech.2020-3122</nowiki>. ====
* '''Abstract:''' This paper presents a deep learning-based single-channel speech enhancement technique that utilizes a multilayer encoder-decoder structure and a Temporal Convolutional Network (TCN) to improve the quality of speech for applications such as smart speakers and voice assistants. The technique leverages the encoder-decoder to obtain a representation suitable for speech enhancement and employs a TCN-based separator between the encoder and decoder to learn long-range dependencies. The optimal number of encoder-decoder layers is determined through t-SNE analysis of the representations learned by different architectures. Experimental results show that the proposed two-layer encoder-decoder structure achieved a 48% improvement in Word Error Rate (WER) over unprocessed noisy data and improvements of 33% and 44% in WER over two baseline models.
* '''Research Question (RQ):''' The research question focuses on exploring the effectiveness of the multilayer encoder-decoder structure in the task of single-channel speech enhancement and the role of TCN in learning long-range dependencies for separating noise and clean speech. Additionally, the study aims to determine the optimal number of encoder-decoder layers for effective noise suppression and speech enhancement.
* '''Hypothesis:''' The paper hypothesizes that using a multilayer encoder-decoder structure can obtain a noise-independent representation, which is useful for separating clean speech and noise. It is also hypothesized that TCN can effectively learn long-range dependencies in the encoded output and provide an enhanced speech mask, thereby improving the performance of speech enhancement.
* '''Conclusion:''' The conclusion indicates that the proposed two-layer encoder-decoder structure outperforms unprocessed noisy data and two baseline models in objective measures of speech quality (such as PESQ and SI-SNR) and Word Error Rate (WER) on a speech recognition platform. Furthermore, t-SNE analysis demonstrates that the two-layer structure can learn a representation suitable for speech enhancement applications.
* '''Critical Observation:''' Although the proposed architecture has achieved significant improvements in speech enhancement, the study mainly focuses on specific types of noise and speech datasets, which may not fully represent the diverse noise conditions in the real world. Moreover, increasing the number of encoder-decoder layers could lead to an increase in the number of model parameters, thereby increasing computational costs and the risk of overfitting. Future work needs to explore model optimization and compression techniques to reduce the number of parameters and test the generalizability and suitability of the technique in unseen noisy environments.

* '''Relevance :''' The research is closely related to the field of speech enhancement, especially in improving the performance of Automatic Speech Recognition (ASR) systems in noisy environments. By processing signals directly in the time domain using deep learning techniques, the study provides a new perspective and approach for designing effective single-channel speech enhancement systems. Additionally, by comparing the performance of different architectures, this paper offers guidance for selecting the appropriate model structure and number of layers, which is significant for developing efficient and accurate speech enhancement algorithms.

==== '''Asiedu Asante, B. K., Broni-Bediako, C., & Imamura, H. (2023). Exploring multi-stage gan with self-attention for speech enhancement. ''Applied Sciences'', ''13''(16), 9217. <nowiki>https://doi.org/10.3390/app13169217</nowiki>''' ====

* '''Abstract''': This paper explores the integration of self-attention mechanisms into multi-stage generative adversarial networks (GANs) for speech enhancement. The authors empirically study the effect of adding self-attention to the convolutional layers of the generators in two existing multi-stage GAN architectures: ISEGAN and DSEGAN. The experimental results demonstrate that incorporating self-attention leads to improvements in speech enhancement quality and intelligibility across objective evaluation metrics. The paper also finds that adding self-attention to ISEGAN's generators improves its performance to be competitive with DSEGAN while using a smaller model size.

* '''Research Questions''':

# Can integrating self-attention mechanisms into multi-stage speech enhancement GANs improve their enhancement performance?
# How does the incorporation of self-attention affect the performance gap between ISEGAN and DSEGAN architectures?

* '''Hypothesis''': The authors hypothesize that introducing self-attention into the convolutional layers of the generators in multi-stage speech enhancement GANs will allow the models to better capture temporal dependencies in the input signal sequences, leading to improved enhancement quality. They also posit that adding self-attention to ISEGAN may allow it to approach the performance of the larger DSEGAN model.

* '''Conclusion''': The experimental results confirm that integrating self-attention mechanisms into the ISEGAN and DSEGAN architectures (referred to as ISEGAN-Self-Attention and DSEGAN-Self-Attention) leads to consistent improvements in objective speech enhancement metrics. Furthermore, ISEGAN-Self-Attention is able to achieve enhancement performance competitive with the base DSEGAN model while using only half the model parameters. This highlights the potential of self-attention to improve the efficiency-performance tradeoff in multi-stage speech enhancement GANs.

* '''Methodology''':
** The paper provides a clear description of how the self-attention mechanism is integrated into the existing ISEGAN and DSEGAN architectures.
** The experimental setup is reasonable, using a standard dataset (Voice Bank corpus) and evaluation metrics.
** However, the paper does not include any subjective evaluation (e.g. human listening tests), which would provide additional insight into the perceptual quality of the enhanced speech.

* '''Results and Argumentation''':
** The objective evaluation results strongly support the paper's conclusions regarding the benefits of integrating self-attention.
** The authors provide a logical argument for why self-attention is able to improve performance by better capturing temporal dependencies.
** It would be interesting to see further analysis of how the self-attention mechanisms are operating, e.g. visualizations of the attention weights.

* '''Potential Biases''':
** The paper only evaluates the proposed approach on a single dataset. Testing on additional datasets would help assess the generalizability of the findings.
** All experiments use the same hyperparameters for the self-attention mechanisms. It's unclear if these are the optimal settings.

* '''Relevance''': This paper is highly relevant to research on deep learning architectures for speech enhancement, specifically in demonstrating the benefits of integrating self-attention into multi-stage GAN models. The findings regarding the efficiency-performance tradeoff between ISEGAN-Self-Attention and DSEGAN are notable and could inform model selection in practical applications.