Hidden Markov Models: Difference between revisions

From MSc Voice Technology
Jump to navigation Jump to search
VisualWikitext
No edit summary
Line 15: Line 15:
== Key Innovations ==
== Key Innovations ==


Some key innovations in the field of [[Speech Recognition]] using Hidden Markov Models include:
Some key innovations in the field of [[Speech Recognition|speech recognition]] using Hidden Markov Models include <ref>Juang, B. H., & Rabiner, L. R. (1991). Hidden Markov models for speech recognition. ''Technometrics'', ''33''(3), 251-272.</ref>:


* [[Dragon Dictate]]
* The DRAGON System developed by Dr. James Baker was one of the earlier speech recognition systems that used HMMs and later became known as [[Dragon Dictate]].<ref>Baker, J. (1975). The DRAGON system--An overview. ''IEEE Transactions on Acoustics, speech, and signal Processing'', ''23''(1), 24-29.</ref> DRAGON is a probabilistic model that represents all knowledge from the training set of utterances in a transition matrix and a matrix of conditional probabilities between the hidden states and the observable states. This allowed the system to be speaker-agnostic, while being quick in finding the optimal path of recognition through dynamic programming. [[Carnegie Mellon's Harpy System]] improved upon DRAGON by incorporating speech-dependent heuristics and other improvements to increase the performance. <ref>Lowerre, B. T. (1976). The Harpy speech recognition system [Ph. D. Thesis].</ref>
* [[DARPA Speech Understanding Research]]
* [[DARPA Speech Understanding Research]] funded multiple laboratories in speech recognition, including BYBLOS and SPHINX, both using HMMs. <ref>Chow, Y., Dunham, M., Kimball, O., Krasner, M., Kubala, G., Makhoul, J., ... & Schwartz, R. (1987, April). BYBLOS: The BBN continuous speech recognition system. In ''ICASSP'87. IEEE International Conference on Acoustics, Speech, and Signal Processing'' (Vol. 12, pp. 89-92). IEEE.</ref><ref>Lee, K. F. (1988). ''Automatic speech recognition: the development of the SPHINX system'' (Vol. 62). Springer Science & Business Media.</ref>
* Voice assistants, such as [[Introduction of Voice Assistants|Siri]]: before the [[Deep Learning Revolution]] and the [[Development of End-to-End Models]],  
* Many [[Introduction of Voice Assistants|voice assistants]] used HMMs before the [[Deep Learning Revolution]] and the [[Development of End-to-End Models|development of end-to-end models]], an example is Siri. <ref>Domingos, P. (2015). ''The master algorithm: How the quest for the ultimate learning machine will remake our world''. Basic Books.</ref>
All in all, the impact of HMMs in speech recognition has been significant as faster, more simplified, and more generalized alternatives to conventional knowledge representation models.


== Impact on the Field ==
== Impact on the Field ==

Revision as of 13:29, 16 September 2023

Claimed by Ömer, Jocomin, and Ding.

Introduction

A Hidden Markov Model (HMM) is a temporal probabilistic model in which some "hidden" or unobservable states are described by observable variables that are generated by these hidden states. [1] These hidden states adhere to the Markov property, meaning that the future state is only dependent on the current state. Since one cannot observe the underlying states of a specific model, learning the transition function of this sequence of states involves aligning the HMM to the observable states.[2][3]

Many real-world applications present hidden variables that are only observable through some emitted outcome, e.g. a speech signal of a word is observed rather than the specific phoneme states that are the underlying hidden states. To determine what the sequence of phonemes (states) would be that results in that specific word, the model learns the relation between the observed and unobservable variables.[4]

The technique behind Hidden Markov Models has been shown to be related to Dynamic Time Warping.[5][6]

Historical Context

Key Innovations

Some key innovations in the field of speech recognition using Hidden Markov Models include [7]:

  • The DRAGON System developed by Dr. James Baker was one of the earlier speech recognition systems that used HMMs and later became known as Dragon Dictate.[8] DRAGON is a probabilistic model that represents all knowledge from the training set of utterances in a transition matrix and a matrix of conditional probabilities between the hidden states and the observable states. This allowed the system to be speaker-agnostic, while being quick in finding the optimal path of recognition through dynamic programming. Carnegie Mellon's Harpy System improved upon DRAGON by incorporating speech-dependent heuristics and other improvements to increase the performance. [9]
  • DARPA Speech Understanding Research funded multiple laboratories in speech recognition, including BYBLOS and SPHINX, both using HMMs. [10][11]
  • Many voice assistants used HMMs before the Deep Learning Revolution and the development of end-to-end models, an example is Siri. [12]

All in all, the impact of HMMs in speech recognition has been significant as faster, more simplified, and more generalized alternatives to conventional knowledge representation models.

Impact on the Field

Future Research

LLM Review

References

Here thus are the references: [13]

  1. ↑ Russell, S. J. (2010). Artificial intelligence a modern approach. Pearson Education, Inc..
  2. ↑ Eddy, S. R. (1996). Hidden markov models. Current opinion in structural biology, 6(3), 361-365.
  3. ↑ Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257-286.
  4. ↑ Juang, B. H., & Rabiner, L. R. (1991). Hidden Markov models for speech recognition. Technometrics, 33(3), 251-272.
  5. ↑ Juang, B. H. (1984). On the hidden Markov model and dynamic time warping for speech recognition—A unified view. AT&T Bell Laboratories Technical Journal, 63(7), 1213-1243.
  6. ↑ Fang, C. (2009). From dynamic time warping (DTW) to hidden markov model (HMM). University of Cincinnati, 3, 19.
  7. ↑ Juang, B. H., & Rabiner, L. R. (1991). Hidden Markov models for speech recognition. Technometrics, 33(3), 251-272.
  8. ↑ Baker, J. (1975). The DRAGON system--An overview. IEEE Transactions on Acoustics, speech, and signal Processing, 23(1), 24-29.
  9. ↑ Lowerre, B. T. (1976). The Harpy speech recognition system [Ph. D. Thesis].
  10. ↑ Chow, Y., Dunham, M., Kimball, O., Krasner, M., Kubala, G., Makhoul, J., ... & Schwartz, R. (1987, April). BYBLOS: The BBN continuous speech recognition system. In ICASSP'87. IEEE International Conference on Acoustics, Speech, and Signal Processing (Vol. 12, pp. 89-92). IEEE.
  11. ↑ Lee, K. F. (1988). Automatic speech recognition: the development of the SPHINX system (Vol. 62). Springer Science & Business Media.
  12. ↑ Domingos, P. (2015). The master algorithm: How the quest for the ultimate learning machine will remake our world. Basic Books.
  13. ↑ Placeholder Reference
Retrieved from "http://wiki.voice-technology.nl/index.php?title=Hidden_Markov_Models&oldid=361"