Editing State-of-the-art (section)

=== Accent Modification ===

==== Introduction ====
Accents play a crucial role in shaping the unique characteristics of speech, reflecting an individual's linguistic background and cultural identity. However, the presence of foreign accents can sometimes pose challenges, particularly in the speaking test for language proficiency assessment.

==== Finkelstein, L., Zen, H., Casagrande, N., Chan, C., Jia, Y., Kenter, T., Petelin, A., Shen, J., Wan, V., Zhang, Y., Wu, Y., & Clark, R. (2022). Training Text-To-Speech Systems From Synthetic Data: A Practical Approach For Accent Transfer Tasks. Google LLC. Retrieved from <nowiki>https://arxiv.org/abs/2208.13183</nowiki> ====
'''Summary''': This paper presents a practical approach for accent transfer tasks in text-to-speech (TTS) synthesis, where aspects of one speaker's speech are transferred to another speaker's speech. The authors address the challenge of creating high-quality transfer models that are also stable and suitable for user-facing applications. They propose a two-step training process involving a Tacotron-based accent transfer model and a robust CHiVE-BERT TTS system. The CHiVE-BERT system is trained on synthetic data generated by the Tacotron model, which results in high-quality audio with transferred accents while preserving speaker characteristics.

'''RQ:''' How can text-to-speech systems be trained to achieve accent transfer effectively and stably, without compromising the quality or usability of the synthesized speech?

'''Hypothesis:''' By training a robust TTS system on synthetic data generated by a less stable but high-quality accent transfer model, it is possible to achieve a balance between quality and stability in accent transfer tasks.

'''Conclusion:''' The study concludes that the proposed two-step training approach, using synthetic data generated by a Tacotron-based model to train a CHiVE-BERT system, yields reliable performance in terms of naturalness and accent transfer capability. The quality loss associated with the switch to synthetic data is within acceptable bounds, and the final system produces high-quality audio that maintains the original speakers' characteristics.

'''Critical observations:''' The authors note that the quality of the final system is affected by the intermediate Tacotron model, with some accents showing significant quality loss, particularly for female speakers in British English. Training on synthetic data can result in lower quality loss compared to using human recordings, possibly due to the reduced variance in synthetic data. The choice of vocoder, synthesizer, and the balance between synthetic and human recordings are critical in the training process, with the final system benefiting from a combination of both.

'''Relevance:''' The research on accent transfer in TTS systems aligns closely with my focus on accent modification for Turkish immigrants in Dutch oral exams. The methodologies explored for synthesizing and transferring accents can be adapted to develop tools that neutralize accents, enhancing exam fairness by ensuring evaluations are based on language skills rather than accent. 

==== Li, W., Tang, B., Yin, X., Zhao, Y., Li, W., Wang, K., Huang, H., Wang, Y., & Ma, Z. (2020). Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech. arXiv preprint arXiv:2005.09271. Retrieved from <nowiki>https://arxiv.org/abs/2005.09271</nowiki> ====
'''Summary:''' This paper presents an end-to-end accent conversion framework aimed at transforming non-native accents into native accents while preserving the speaker's voice timbre. The proposed system introduces reference encoders to utilize multi-source information and optimizes the model architecture using GMM-based attention for improved synthesized performance. Experimental results show significant improvements in acoustic quality and native accent while retaining the non-native speaker's voice identity.

'''RQ:''' How can accent conversion be improved to better transform non-native accents into native accents in a way that maintains the original speaker's voice identity? 

'''Hypothesis:''' Incorporating reference encoders and optimizing the model architecture with GMM-based attention will enhance the quality and naturalness of converted speech, leading to more effective accent conversion.

'''Conclusion:''' Incorporating reference encoders and optimizing the model architecture with GMM-based attention will enhance the quality and naturalness of converted speech, leading to more effective accent conversion.

'''Critical observations:''' The paper highlights the importance of prosodic and expressive information in accent conversion, which is effectively captured by the reference encoder. The GMM-based attention mechanism is found to be more stable and powerful for feature representation compared to traditional windowed attention.

'''Relevance:''' The research is relevant to accent modification efforts, particularly in language learning and pronunciation training contexts. The proposed accent conversion techniques could be applied to develop tools that help non-native speakers improve their pronunciation and reduce their accents, thereby enhancing communication and integration in societies where the target language is spoken natively. 

==== Zang, X., Weng, F., & Zang, X. (2022). Foreign Accent Conversion using Concentrated Attention. In 2022 IEEE International Conference on Knowledge Graph (ICKG). Retrieved from <nowiki>https://ieeexplore.ieee.org/document/978-1-6654-5101-7</nowiki> ====
'''Summary:''' This paper introduces a novel method for foreign accent conversion (FAC) utilizing Phonetic Posteriorgrams (PPGs) and Log-scale Fundamental frequency (Log-FO) to address phonetic and prosody mismatches. The proposed approach employs concentrated attention to enhance the alignment of input sequences and mel-spectrograms, selecting the top k highest score values in the attention matrix row by row. The method is evaluated through objective metrics and demonstrates improved voice naturalness, speaker similarity, and accent similarity.

'''RQ:''' How can foreign accent conversion be improved to achieve better alignment and naturalness in synthesized speech while preserving the source speaker's identity? 

'''Hypothesis:''' Implementing concentrated attention in the foreign accent conversion process will result in more accurate alignment of input sequences with mel-spectrograms, leading to improved accent conversion quality and naturalness in synthesized speech.

'''Conclusion:''' The proposed method using concentrated attention for foreign accent conversion delivers comparable or better results than previous methods in terms of voice naturalness and accent similarity. The concentrated attention mechanism effectively focuses on the most relevant frames for better alignment and synthesized speech quality.

'''Critical observations:''' The concentrated attention mechanism is found to be beneficial for achieving better alignment between input sequences and target sequences, resulting in improved speech synthesis.

'''Relevance:''' The research is relevant to the field of speech synthesis and voice conversion, particularly for applications that require the alteration of accents while maintaining the original speaker's voice characteristics. This work contributes to the development of systems that can aid in language learning, dubbing, and other scenarios where accent modification is beneficial, enhancing the quality and naturalness of synthesized speech.