Editing State-of-the-art (section)

==== N, K. D., Wang, P., & Bozza, B. (2021). Using Large Self-Supervised Models for Low-Resource Speech Recognition. ''Interspeech 2021'', 2436–2440. <nowiki>https://doi.org/10.21437/Interspeech.2021-631</nowiki> ====
* Summary: This paper investigates the effectiveness of using large self-supervised pre-trained models (such as wav2vec 2.0) for low-resource speech recognition tasks. The authors conducted experiments on three Indian languages (Telugu, Tamil, and Gujarati), using different pre-trained models (monolingual English, multilingual) and compared different fine-tuning strategies (CTC, seq2seq, etc.).
* RQ: For low-resource speech recognition tasks, how effective are large self-supervised pre-trained models (such as wav2vec 2.0) compared to traditional supervised learning methods? For Indian languages, are cross-lingual multilingual pre-trained models or monolingual English pre-trained models more suitable? How do different fine-tuning strategies (CTC vs seq2seq) affect model performance? Additionally, how well do these pre-trained models generalize to seen and unseen languages?
* Hypothesis:
*# Large self-supervised pre-trained models will outperform supervised learning models under low-resource conditions.
*# Cross-lingual multilingual pre-trained models will perform better than monolingual English models on these Indian languages.
*# Adopting the CTC fine-tuning strategy will achieve better performance than the seq2seq strategy.
* Conclusion:The multilingual pre-trained model XLSR outperformed the monolingual models on all three languages; for seen languages (like Tamil), the pre-trained model can approach the best performance with only 50% of the training data; the CTC fine-tuning framework performed better than the seq2seq framework, possibly due to the small amount of data; even smaller English pre-trained models showed decent transfer performance on Indian languages.
* Critical observations:The authors did not explain why the larger English pre-trained model underperformed compared to the smaller one, and analysis of the multilingual fine-tuning strategy was limited, only compared to the monolingual strategy. In addition, the impact of different pre-training corpora on model performance was not explored.
* Relevance:This work is important for low-resource speech recognition domains in developing countries. Leveraging large self-supervised pre-trained models can make full use of unlabeled data, alleviating the bottleneck of limited labeled data. This study provides an effective solution for low-resource speech recognition tasks.