Editing Hidden Markov Models in Speech Synthesis (section)

== Future Research ==
HMM-based speech synthesis has shown promise in generating speech with diverse speaking styles and demands less storage (Drugman et al., 2009)<ref>Drugman, T, Wilfart G, Dutoit T (2009) A deterministic plus stochastic model of the residual signal for improved parametric speech synthesis.</ref>. This has led to the emergence of several applications, notably: a) personalized speech-to-speech translation systems<ref>Y. Qian, H. Liang, and F. K. Soong, “A cross-language state sharing and mapping approach to bilingual (Mandarin – English) TTS,” IEEE Trans. Audio Speech Lang. Process., vol. 17, no. 6, pp. 1231–1239, Aug. 2009.</ref><ref>Y.-J. Wu, S. King, and K. Tokuda, “Cross-language speaker adaptation for HMM-based speech synthesis,” in Proc. ISCSLP, 2008, pp. 9–12.</ref><ref>K. Oura, J. Yamagishi, M. Wester, S. King, and K. Tokuda, “Analysis of unsupervised cross-lingual speaker adaptation for HMM-based speech synthesis using KLD-based transform mapping,” Speech Commun., vol. 54, no. 6, pp. 703–714, 2012.</ref> and b) personalized speech synthesizers for individuals with vocal disabilities <ref>J. Yamagishi, C. Veaux, S. King, and S. Renals, “Speech synthesis technologies for individuals with vocal disabilities: voice banking and reconstruction,” in Acoustical Science & Technology, 2012, vol. 33, pp. 1–5.</ref> and HTS is also widely applied in [[Commerical TTS - Google, Amazon, Apple and Microsoft (2010s)]]. 

However, despite these advantages, HTS does possess a few limitations and requires improving. Primarily, enhancing the intelligibility of synthetic speech in noise remains a challenge. Researchers have pointed out that unmodified synthetic speech tends to experience a more pronounced reduction in intelligibility compared to unmodified natural speech in noisy environments <ref>S. King and V. Karaiskos, “The Blizzard Challenge 2010,” in Proc. Blizzard Challenge Workshop, Kyoto, Japan, Sep. 2010.</ref>. Secondly, HMM-based synthesis faces constraints related to acoustic modeling accuracy. The statistical averaging process used in parametric methods generates smooth speech trajectories, resulting in muffled speech and sometimes incorrectly extracts pitch information<ref>Zen H, Tokuda K, Black AW (2009) Statistical parametric speech synthesis. Speech Commun 51(11):10391064.</ref>.

Looking ahead, there are many directions for further research in the future. For the quality of the synthesized speech, Kawahara et al. proposed pitch-adaptive spectral analysis combined with a surface reconstruction method and an excitation method using instantaneous frequency calculation<ref>H. Kawahara, I. Masuda-Katsuse, and A. Cheveigne, “Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based f0 extraction: possible role of a repetitive structure in sounds,” Speech Commun., vol. 27, pp. 187–207, 1999.</ref>.Addressing F0 modeling, a statistical model of speech fundamental frequency contours has been proposed <ref>H. Kameoka, J. Le Roux, and Y. Ohishi, “A statistical model of speech F0 contours,” in Proc. SAPA, 2010, pp. 43–48.</ref> based on the formulation of the discrete-time stochastic process version of the Fujisaki model <ref>H. Fujisaki and K. Hirose, “Analysis of voice fundamental frequency contours for declarative sentences of Japanese,” J. Acoust. Soc. Jpn. (E), vol. 5, no. 4, pp. 233–242, 1984.</ref>. On a different trajectory, focusing on speech synthesis using "Big data" <ref>J. Ni and H. Kawai, “On the effects of transcript errors across dataset sizes on hmm-based voices,” in Proc. the Autumn Meeting of ASJ, 2011, pp. 339–342.</ref>, which includes ebooks, Internet radio, podcasts, etc., has shown potential. Training HMMs with automatic speech transcription that encompasses errors is one approach in this domain. Additionally, a framework for the analysis of emotion in texts for speech synthesis has been introduced <ref>J. R. Bellegarda, “A data-driven affective analysis framework toward naturally expressive speech synthesis,” IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 5, pp. 1113–1122, 2011.</ref>, allowing the automatic translation of plain texts into appropriate speaking styles. Speech synthesis via physical simulation, particularly an MRI-based articulatory speech synthesis system, is another avenue being explored <ref>T. Kitamura, H. Takemoto, P. Mokhtari, and T. Hirai, “MRI-based time-domain speech synthesis system,” in Proc. ASA/ASJ joint meeting, 2006.</ref>.