Editing Advancements in Neural Network-Based TTS (2000s) (section)

====== <big>TTS-Modeling</big> ======
'''Tacotron:''' Tacotron is an end-to-end generative Text-to-Speech (TTS) model that directly synthesizes speech from input characters, utilizing a sequence-to-sequence (seq2seq) architecture with attention. It avoids the need for phoneme-level alignment.<ref name=":4">Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). "Tacotron: Towards End-to-End Speech Synthesis." In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1946-1954).</ref>

'''Innovation:'''

* End-to-End Generative Model: Tacotron is an end-to-end generative model that synthesizes speech directly from characters, eliminating the need for intermediate linguistic features or acoustic models.
* Sequence-to-Sequence Model with Attention: It is based on a sequence-to-sequence model with attention, enabling high-accuracy and natural speech generation.
* No Phoneme-Level Alignment Required: Tacotron doesn't necessitate phoneme-level alignment, simplifying scalability to large data with transcripts.
* Faster Frame-Level Generation: Tacotron generates speech at the frame level, making it considerably faster than sample-level autoregressive methods like WaveNet.
* High Subjective Mean Opinion Score (MOS): Tacotron attains a 3.82 out of 5 on the subjective mean opinion score, surpassing a production parametric system in terms of naturalness, particularly for US English.

'''Tacotron 2:''' Tacotron 2 is a neural network architecture for direct text-to-speech synthesis. It consists of two core elements: a feature prediction network and a modified [[wikipedia:WaveNet|WaveNet]] vocoder.<ref name=":0" />

'''Innovation:'''

* Compact Acoustic Intermediate Representation: Tacotron 2 utilizes mel spectrograms, providing a streamlined representation of speech features and reducing WaveNet's architectural complexity.
* Modified WaveNet Vocoder: Tacotron 2 adapts WaveNet architecture to convert mel spectrograms into time-domain waveform samples, achieving audio quality akin to human speech.
* Integration of Tacotron and WaveNet: Combining Tacotron-style prosody modeling and WaveNet vocoder, Tacotron 2 delivers state-of-the-art sound quality in a unified, neural approach to speech synthesis.

'''FastSpeech:''' FastSpeech is a neural text-to-speech (TTS) system that addresses the challenges of slow inference speed, lack of robustness, and lack of controllability in traditional TTS models. It uses a feed-forward network based on Transformer to generate mel-spectrograms in parallel, allowing for faster synthesis.<ref name=":1">Ren, J., Xu, L., Zhang, Z., Yang, T., Lai, J., Lu, Z., & Dai, L. R. (2019). FastSpeech: Fast, Robust and Controllable Text to Speech. arXiv preprint arXiv:1905.09263.</ref> 

'''Innovation:'''

* Multi-head attention mechanism: Enhances long-range dependency modeling and parallelization by allowing simultaneous attention to different parts of the input sequence.
* Positional encoding: Provides positional information to input sequence elements, aiding in distinguishing elements with identical values.
* Layer normalization: Improves training stability by normalizing inputs to each network layer.
* Stacked self-attention layers: Enables the network to learn multiple representation levels of the input sequence, enhancing output quality.
* No recurrence or convolution: Unlike traditional architectures, the Transformer network omits recurrent connections and convolutions, resulting in improved efficiency and parallelizability.<ref name=":1" />

'''Transformer:''' The Transformer is a novel neural network architecture introduced by Vaswani et al. in 2017. It stands out for its exclusive reliance on attention mechanisms, foregoing recurrent connections and convolutions. In natural language processing, particularly neural machine translation, it has achieved remarkable success. The Transformer comprises an encoder and a decoder, both structured with stacks of identity blocks. It employs multi-head self-attention to model dependencies between input and output sequences.<ref name=":2">Naihan Li, Shujie Liu, Yanqing Liu, Sheng Zhao, and Ming Liu. 2019. Neural speech synthesis with transformer network. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence (AAAI'19/IAAI'19/EAAI'19). AAAI Press, Article 823, 6706–6713. <nowiki>https://doi.org/10.1609/aaai.v33i01.33016706</nowiki></ref>

'''Features:'''

* Multi-Head Attention Efficiency: In Transformer, multi-head attention in Tacotron2 enhances training efficiency, constructing hidden states in the encoder and decoder concurrently, speeding up training by 4.25 times and effectively addressing long-range dependencies.
* WaveNet Vocoder's Role: Within the Transformer TTS network, WaveNet vocoder synthesizes high-quality audio from mel spectrograms, closely resembling human recordings on specific datasets.
* Transformer Architecture: The Transformer, introduced by Vaswani et al. in 2017, is a unique neural network architecture solely based on attention mechanisms. It excels in natural language processing tasks like neural machine translation, comprising encoder and decoder stacks of identity blocks and utilizing multi-head self-attention to model input-output dependencies.<ref name=":2" />