Editing Hidden Markov Models in Speech Synthesis (section)

=== HMM Improvement Based on [[Advancements in Neural Network-Based TTS (2000s)|Neural Networks]] ===
Integrating neural networks with HMM, such as Deep Neural Network-HMM (DNN-HMM) and Recurrent Neural Network-HMM (RNN-HMM), to enhance the model's expressive power and efficiency. Significant innovations in improving HMMs based on neural networks in the field of speech synthesis focus on combining neural networks with HMMs to enhance the model's expressive power, accuracy, and naturalness. These innovations combining neural networks with HMMs result in models with stronger expressive capabilities, more accurate modeling, and more natural speech synthesis effects.

==== Integration of Deep Neural Network and HMM (DNN-HMM) ====
# DNN for Acoustic Modeling: Employing deep neural networks to model HMM state emission probability, replacing the traditional Gaussian mixture model (GMM), significantly enhancing the accuracy and naturalness of the speech synthesis system. [Hinton et al., 2012]<ref name=":0" />
# DNN as a Front-end Feature Extractor: Utilizing DNN as a feature extractor and its output as the input feature of HMM to enhance feature representation. [Zeiler et al., 2013]<ref>Zeiler, M. D., Krishnan, D., Taylor, G. W., Fergus, R. (2013). [https://arxiv.org/abs/1310.1531 Deconvolutional networks. In Computer Vision and Pattern Recognition] (CVPR), 2010 IEEE Conference on (pp. 2528-2535).</ref>

==== Integration of Recurrent Neural Network and HMM (RNN-HMM) ====
# RNN for Timing Modeling: Integrating recurrent neural networks (RNN) to model the timing of HMM, better capturing long-term information and improving fluency and naturalness of synthetic speech. [Graves et al., 2013]<ref>Graves, A., Mohamed, A. R., Hinton, G. (2013). [https://www.cs.toronto.edu/~hinton/absps/RNN13.pdf Speech recognition with deep recurrent neural networks]. In Acoustics, speech and signal processing (ICASSP), 2013 IEEE international conference (pp. 6645-6649).</ref>
# Long-term Sequence Modeling: Introducing long-term memory networks (LSTM) as a variant of RNN to address the problem of gradient vanishing in long-time series modeling and improve the time series modeling ability. [Hochreiter and Schmidhuber, 1997]<ref>Hochreiter, S., & Schmidhuber, J. (1997). [https://www.mitpressjournals.org/doi/10.1162/neco.1997.9.8.1735 Long short-term memory]. Neural computation, 9(8), 1735-1780.</ref>