Editing State-of-the-art (section)

==== Subtheme 2: State-of-the-art Theories ====

===== Noufi, C., May, L., & Berger, J. (2023). Context, Perception, Production: A Model of Vocal Persona. ''PsyArXiv. July'', ''28''. =====
* Summary: This article introduces a contextualized production-perception model of vocal persona, developed through qualitative analysis of interviews with voice and performance experts. It emphasizes the influence of context on an individual's vocal expression, reflecting the intricacies of human communication.
* RQ: What is the relationship between context, vocal expression, and identity?
* Hypothesis: It is a qualititative study and does not hve a formulated hypothesis.  Instead of attempting to falsify a hypothesis as in most quantitative studies, it explores answers to the research question through thematic analysis.
* Conclusion: Speakers actively select different vocal personas and adjust relevant vocal expressions in response to the surrounding context, facilitating a transition in the perception of persona.
* Critical observations: The proposal of the vocal persona model and general conclusions are based on interviews with 21 voice and performance experts, which may have limitations in terms of subjective bias and generalizability beyond this specific context.
* Relevance: This study underscores the necessity for speakers to adapt their speaking styles to accommodate different social contexts, highlighting the significance of context in vocal expression. It proposes the incorporation of vocal persona into expressive vocal synthesis with a three-spoke model and a framework for persona-guided vocalization, enriching the framework of TTS naturalness and expressiveness.

===== Vainer, J., & Dušek, O. (2020). Speedyspeech: Efficient neural speech synthesis. ''arXiv preprint arXiv:2008.03802''. =====
* Summary: This paper introduces a novel student-teacher network architecture called "SpeedySpeech" for fast and high-quality neural speech synthesis. The system is designed to enable faster-than-real-time speech synthesis while requiring minimal computing resources, and deliver audio quality that is superior to existing models such as the Tacotron 2. The model uses the teacher network for duration extraction, the student network for spectrogram synthesis, and combines it with the MelGAN vocoder to output high-quality audio. The training process is efficient and can be completed in less than 40 hours on a single 8GB GPU. 
* RQ: How can we develop a neural speech synthesis system that does not require extensive computing resources while maintaining fast training times, fast inference, and high-quality audio output?
* Hypothesis: Assuming a student-teacher network architecture with simplified convolutional blocks and only a single attention layer in the teacher model, it is possible to surpass existing models in terms of training efficiency and audio quality while maintaining fast inference.
* Conclusion: The proposed SpeedySpeech model successfully achieves its goals by demonstrating that self-attention layers are not necessary for high-quality audio generation and that simpler, fully convolutional methods enable a more efficient training process and faster synthesis. The model's speech quality score is significantly higher than Tacotron 2, and it can be trained efficiently on a single GPU and even run in real time on the CPU.
* Critical observations: The article proposes ways to address the trade-off between training efficiency and audio quality in neural speech synthesis. By using only a single attention layer in the teacher model and eliminating sequence generation in the student network, the authors achieve important simplifications that increase model efficiency. In the model evaluation, the authors comprehensively considered objective indicators (such as MAE, SSIM) and subjective listening tests to provide a comprehensive assessment of model performance.
* Relevance: This speech synthesis model has applications in many fields, including virtual assistants, machine translation, etc. The SpeedySpeech model can synthesize speech in real time on moderate hardware, making it particularly suitable for deployment in resource-constrained environments. Additionally, the focus on efficiency and quality sets new benchmarks for future research and development in this area.

===== Peiró-Lilja, A., & Farrús, M. (2020). Naturalness Enhancement with Linguistic Information in End-to-End TTS Using Unsupervised Parallel Encoding. ''Interspeech 2020'', 3994–3998. <nowiki>https://doi.org/10.21437/Interspeech.2020-1788</nowiki> =====
* Summary: The paper explores enhancing the naturalness of synthesized speech in E2E-TTS systems by incorporating linguistic features like POS tags and punctuation into the Tacotron 2 model, aiming to improve prosody to resemble human speech more closely.
* RQ: How can linguistic information be integrated into the Tacotron 2 system to improve the naturalness of synthesized speech prosody?
* Hypothesis: The hypothesis is that by embedding POS tags and punctuation locations as additional linguistic features into the Tacotron 2 system, the synthesized speech will exhibit improved naturalness and prosody, making it more similar to human speech.
* Conclusion: The study concludes that the incorporation of linguistic features through a parallel encoder significantly improves the naturalness of synthesized speech. The authors proposed two different architectures for the parallel encoder: one based on convolutional and recurrent layers (2DConv+BiLSTM) and another composed of bidirectional recurrent and linear layers (BiGRU+Linear). Both architectures aimed to process the binary matrix representing POS tags and punctuation locations. The results from objective tests and perceptual evaluations indicated that the model with the 2DConv+BiLSTM parallel encoder performed the best in terms of naturalness, as it more closely matched human pitch contours and overall speech quality.
* Critical observations: Critically, the paper notes that while both parallel encoder architectures showed improvements over the Tacotron 2 baseline, the 2DConv+BiLSTM version provided better results in terms of naturalness. However, it also introduced a slight increase in Mel Cepstral Distortion (MCD), suggesting a trade-off between naturalness and certain acoustic quality metrics. The BiGRU+Linear model, despite being lighter and faster, underperformed in perceptual tests, possibly due to its reduced complexity and higher cepstral distortion.
* Relevance: The findings of this research are relevant for the development of more natural and human-like E2E-TTS systems, which have applications in various domains such as automatic dialogue systems, storytelling, and voice assistants. By enhancing the prosody of synthesized speech, these systems can provide more engaging and realistic interactions, improving user experience and accessibility. Furthermore, the study contributes to the broader field of speech synthesis by demonstrating the potential of unsupervised parallel encoding of linguistic features to improve speech naturalness.

===== Cai, X., Dai, D., Wu, Z., Li, X., Li, J., & Meng, H. (2021). Emotion Controllable Speech Synthesis Using Emotion-Unlabeled Dataset with the Assistance of Cross-Domain Speech Emotion Recognition. ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 5734–5738. <nowiki>https://doi.org/10.1109/ICASSP39728.2021.9413907</nowiki> =====
* Summary: This article proposes an approach for emotional TTS synthesis on a dataset without emotion labels, using a cross-domain speech emotion recognition model and an emotional TTS model, hoping to express similar results in specific emotional expressiveness and speech quality compared to models with emotion labels.
* RQ: Can we use the achievements and features of SER to solce the problem of lack of emotion-annotated dataset for emotional TTS?
* Hypothesis: By performing mean opinion score evaluations and emotion recognition perception evaluation in 4 emotion categories and 2 polarities of emotion dimensions, our GST-based model can generate speech with expected emotions, while trained on a fully emotion-unlabeled dataset. 
* Conclusion: Through comparing their created cross-domain model with a baseline model, they found that both their 4-categorical model and 2-dimensional model almost achieve as good speech quality as the baseline system, with higher p-values than the significance level of 0.05, indicating no significant differences. Furthermore, they found that both categorical models, one trained on the utterances with highest posterior (top-K scheme) and one trained on the full set of audio, revealed an overall higher accuracy than the baseline model, at 78.75% and 49.25%, respectfully, compared to the baseline 36.75%, which indicated their cross-domain model and top-K scheme were effective in emotional expressiveness. 
* Critical observations: The choice to use a top-K scheme, as described earlier, is interesting to offset the number of mispredictions made by the SER model, as the SER model is far less reliable that humans. By choosing to use the more reliable audio set, there could be an argument that their choice could inflate their results. However, taking this into account, they did in fact train the model on the full, unaltered, set of audio and still returned a higher accuracy than the baseline model. The fact that this shows promise in returning accurte and sufficient quality in emotional speech synthesis with unlabeled emotion datasets brings promise to a possible increase in speed and efficiency for training other models. 
* Relevance: The proposed approach, in the authors' words, greatly reduces the threshold of emotional synthesis in regard to amotion-annotated data, reducing the time, cost, and relevant quality of the speech data needed for emotional TTS systems.