Editing Hidden Markov Models (section)

== Historical Context ==

The origins of HMMs trace back to 1907 when Andrei Markov introduced Markov chains, demonstrating that dependent variables are also subject to the law of large numbers, extending beyond the realm of independent variables.<ref>Gagniuc, P.A. (2017). Historical Notes. In Markov Chains, P.A. Gagniuc (Ed.). https://doi.org/10.1002/9781119387596.ch1</ref> However, it was not until the 1960s that Leonard Baum and Ted Petrie initiated the development of a novel model aimed at estimating the parameters of Markov chains with maximum likelihood. Their work refined the probability equations, enabling the inference of hidden paths within the process.<ref>Baum, Leonard E., and Ted Petrie. “Statistical Inference for Probabilistic Functions of Finite State Markov Chains.” The Annals of Mathematical Statistics 37, no. 6 (December 1966): 1554–63. https://doi.org/10.1214/aoms/1177699147.</ref><ref>Nilsson, Mikael, and Marcus Ejnarsson. “Speech Recognition Using Hidden Markov Model,” n.d.</ref> Subsequently, Jelinek, Bahl, and Mercer pioneered the application of Markov models in speech recognition, particularly focusing on mitigating speaker-dependent probabilities. This marked a seminal moment in the widespread adoption of HMMs.<ref>Jelinek, F., L. Bahl, and R. Mercer. “Design of a Linguistic Statistical Decoder for the Recognition of Continuous Speech.” IEEE Transactions on Information Theory 21, no. 3 (May 1975): 250–56. https://doi.org/10.1109/TIT.1975.1055384.</ref><ref>Stamp, Mark. “A Revealing Introduction to Hidden Markov Models.” In Introduction to Machine Learning with Applications in Information Security, by Mark Stamp, 7–35, 1st ed. Chapman and Hall/CRC, 2017. https://doi.org/10.1201/9781315213262-2.</ref>

Over the subsequent decades, substantial advancements and refinements have been made to address inherent challenges in HMMs. Notably, during the 1980s and 1990s<ref>Gales, Mark, and Steve Young. “The Application of Hidden Markov Models in Speech Recognition.” Foundations and Trends® in Signal Processing 1, no. 3 (February 20, 2008): 195–304. https://doi.org/10.1561/2000000004.</ref>,  innovations such as shared-distribution HMMs in which, instead of having each hidden state in the model associated with its distinct probability distribution, multiple hidden states share a common probability distribution. By doing so, SD-HMMs effectively reduce the number of model parameters, making them more manageable and robust, particularly when dealing with limited training data.<ref>Hwang, Mei-Yuh, and Xuedong Huang. “Shared-Distribution Hidden Markov Models for Speech Recognition.” IEEE Transactions on Speech and Audio Processing 1, no. 4 (October 1993): 414–20. https://doi.org/10.1109/89.242487.</ref> Hierarchical Hidden Markov Models (HHMMs) represented a significant evolution from traditional HMMs by introducing a hierarchical structure that empowers individual hidden states to function autonomously and generate sequences of symbols or states, prompting advancements in natural language processing, and becoming crucial in speech recognition and machine translation.<ref>Fine, Shai, Yoram Singer, and Naftali Tishby. “The Hierarchical Hidden Markov Model: Analysis and Applications.” Machine Learning 32, no. 1 (July 1, 1998): 41–62. https://doi.org/10.1023/A:1007469218079.</ref> Moreover, signal decomposition techniques involving parallel HMMs were developed to address concurrent event recognition challenges, enabling the simultaneous identification and separation of distinct events from complex acoustic signals. This innovation has proven particularly valuable in applications like audio processing, where isolating target sounds from background noise or identifying multiple overlapping events is essential for enhanced signal analysis and interpretation.<ref>Varga, A.P., and R.K. Moore. “Hidden Markov Model Decomposition of Speech and Noise.” In International Conference on Acoustics, Speech, and Signal Processing, 845–48. Albuquerque, NM, USA: IEEE, 1990. https://doi.org/10.1109/ICASSP.1990.115970.</ref>

In subsequent years, notably during the latter part of the 1980s, HMMs began to play a pivotal role in the domains of DNA sequencing and computational biology. These mathematical models proved instrumental in decoding genetic information, facilitating the analysis of biological data, and unraveling the intricacies of genomic sequences, contributing significantly to the progress of molecular biology and bioinformatics during this period.<ref>Eddy, Sean R. “What Is a Hidden Markov Model?” Nature Biotechnology 22, no. 10 (October 2004): 1315–16. https://doi.org/10.1038/nbt1004-1315.</ref>