Development of End-to-End Models: Difference between revisions
| Line 25: | Line 25: | ||
== Historical Context == | == Historical Context == | ||
For a long time, the hidden Markov model (HMM)-Gaussian mixed model (GMM) has been the mainstream speech recognition framework. But recently, HMM-deep neural network (DNN) model and the end-to-end model using deep learning has achieved performance beyond HMM-GMM. Both using deep learning techniques, these two models have comparable performances. However, the HMM-DNN model itself is limited by various unfavorable factors such as data forced segmentation alignment, independent hypothesis, and multi-module individual training inherited from HMM, while the end-to-end model has a simplified model, joint training, direct output, no need to force data alignment and other advantages. Therefore, the end-to-end model is an important research direction of speech recognition. | |||
Traditional ASR systems involve multiple stages, including feature extraction, acoustic modeling, phonetic decoding, and language modeling. These stages often require handcrafted engineering and are computationally expensive. | |||
The traditional approach design for a spoken language understanding system is a pipeline structure with several different components, exemplified by the following sequence: <blockquote>'''Audio (input) → Feature Extraction → Phoneme Detection → Word Composition → Text Transcript (output)'''</blockquote>A clear limitation of this pipelined architecture is that each module has to be optimized separately under different criteria. The E2E approach consists in replacing the aforementioned chain for a single Neural Network(NN), allowing the use of a single optimization criterion for enhancing the system: <blockquote>'''Audio (Input) — — NN — → Transcript (output)'''</blockquote>E2E models emerged as a response to streamline this process and leverage deep learning techniques to directly map audio to text. | |||
== Key Innovations == | |||
== Key Innovations == | == Key Innovations == | ||
Revision as of 17:26, 16 September 2023
YiningLei, Xinyi Ma, Liqing, Jingwen Shi
Introduction
The development of End-to-End (E2E) models represent a significant shift in the field of automatic speech recognition (ASR), which seek to simplify the complex pipeline of traditional systems by directly mapping input audio sequence to sequence of words or other graphemes.[1] Framed in the Deep Learning context and taking advantage of Neural Network(NN) architectures, these models directly capture the acoustic and linguistic information present in the speech signal, casting a possibly complex processing pipeline into the coherent and flexible modeling language of neural networks.[2] The functional structure of E2E models is shown below
L = {,···，} output sequence
Decoder
Aligner
F = {,···，} feature sequence
Encoder
X = {,···，} input sequence
Historical Context
For a long time, the hidden Markov model (HMM)-Gaussian mixed model (GMM) has been the mainstream speech recognition framework. But recently, HMM-deep neural network (DNN) model and the end-to-end model using deep learning has achieved performance beyond HMM-GMM. Both using deep learning techniques, these two models have comparable performances. However, the HMM-DNN model itself is limited by various unfavorable factors such as data forced segmentation alignment, independent hypothesis, and multi-module individual training inherited from HMM, while the end-to-end model has a simplified model, joint training, direct output, no need to force data alignment and other advantages. Therefore, the end-to-end model is an important research direction of speech recognition.
Traditional ASR systems involve multiple stages, including feature extraction, acoustic modeling, phonetic decoding, and language modeling. These stages often require handcrafted engineering and are computationally expensive.
The traditional approach design for a spoken language understanding system is a pipeline structure with several different components, exemplified by the following sequence:
Audio (input) → Feature Extraction → Phoneme Detection → Word Composition → Text Transcript (output)
A clear limitation of this pipelined architecture is that each module has to be optimized separately under different criteria. The E2E approach consists in replacing the aforementioned chain for a single Neural Network(NN), allowing the use of a single optimization criterion for enhancing the system:
Audio (Input) — — NN — → Transcript (output)
E2E models emerged as a response to streamline this process and leverage deep learning techniques to directly map audio to text.
Key Innovations
Key Innovations
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Impact
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Future research
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. [3]
References
To insert a reference, type <ref> and paste the source your exported from Zotero (or whatever reference manager you're using) in the pop up box which appears. Make sure links in citations are clickable using proper formatting. Once you do this, a footnote will appear.[4] and a reference comes at the end automatically. Please use this method to cite for Wiki articles only, not for your thesis.
- ↑ Wang, Dong, Xiaodong Wang, and Shaohe Lv. 2019. “An Overview of End-to-End Automatic Speech Recognition.” Symmetry 11(8):1018.
- ↑ Glasmachers, Tobias. “Limits of end-to-end learning.”arXiv preprint arXiv:1704.08305 (2017).
- ↑ Feng, S., Kudina, O., Halpern, B. M., & Scharenborg, O. (2021). Quantifying Bias in Automatic Speech Recognition (arXiv:2103.15122). arXiv. http://arxiv.org/abs/2103.15122
- ↑ Glantz, Richard "SHOEBOX: a personal file handling system for textual data." In Proceedings of the November 17-19, 1970, Fall Joint Computer Conference 1970. 535-545. [1]
<ref>