Editing Carnegie Mellon's Harpy System (section)

== Introduction ==
'''Harpy System''' is a result of combing and improving the best features from two speech systems previously developed at Carnegie Mellon University: '''Hearsay-I''' ( Erman 1974 ) and '''Dragon''' ( Baker 1975 ). Two feature of the Harpy system that led to a successful demonstration are its '''representation of knowledge''' and the use of '''new search techniques'''. In developing speech recognition systems, it is necessary to devise a means of acquiring and representing the many diverse types of knowledge that characterize speech. We must also develop '''matching''' and '''searching''' techniques that convert this passive knowledge into an active process for understanding the utterance in the presence of error, noise, and uncertainty.  


When an utterance is input to the computer. Several pre-processing steps are taken to prepare the data for recognition. The utterance is '''segmented into acoustic units''' and analyzed to determine the segmented features and parameters. The goal of recognition tasks is to find '''an optimal sequence''' of phones satisfying two criteria: '''the sequence must represent a legal path through the knowledge network''' and should '''consist of phones with high acoustic matches'''.


Harpy Systems uses a '''beam search''' to locate this optimal sequence of phones. Harpy begins the beam search by taking all legal phones from the start of the sentence from the knowledge network and entering them in a '''recognition tree'''. Next, a path probability is calculated for each candidate. The path with the best probability is determined and remaining candidates are compared with it. '''Those that fall below a threshold of acceptability are eliminated from further searching'''. The successors of the surviving candidates are expanded based on the information in the knowledge network.
{| class="wikitable"
|+
!
!Hearsay-I System
!Dragon System
!Harpy System
|-
|Sources of Knowledge
|
*Acoustic-Phonetics  (the characteristics of the sounds)
* Prosodics   (the stress and intonation patterns of speech)
* Lexicon   (a dictionary of legal words)
* Syntax   (the grammatical structure of the language)
* Semantics   (the meaning of words and sentences)
* Pragmatics   (the context of the conversation)
|
*Almost '''no''' speech dependent heuristics. 
*The integrated knowledge of both syntax and lexical spellings, a set of user dependent acoustic-phonetic templates, and an acoustic-phonetic probability matching routine. 
*Mathematical acceptability available ---one of probabilistic function of '''[https://wiki.voice-technology.nl/index.php/Hidden_Markov_Models Markov Process]<ref>''Hidden Markov Models''. (2023).[[Hidden Markov Models]]</ref>'''.  

|
*The combination of speech dependent heuristics (Hearsay-I System ) and mathematical tractable model  (Dragon System ). This leverages the use of heuristics to reduce the search space therefore speed increase.
*The system extensively use a network that represents both all legal syntactic paths and pronunciations of these legal paths. 
*No a-priori transition probabilities. 
|-
|Knowledge Representation
|Procedural embedding.
|Markove networks. 
|Transition networks. 
|-
|Search Strategy
|Best-First with backtracking. 
|All paths in parallel ( search all the possible syntactic
and acoustic paths through the network in parallel to determine optimal path ) with no backtracking.
|<nowiki>''best few''</nowiki> in parallel with no backtracking. 
'''Optimized Beam Search'''<ref>Reddy, R. (2013). ''CMU Harpy System 1976 - Beam Search''. Retrieved September 17, 2023, from <nowiki>https://www.youtube.com/watch?v=32KKg3aP3Vw</nowiki>.</ref>
* Locates optimal solution
* Examines near-miss alternatives
* No backtracking
|-
|Segmentation
|Yes
( uses it to reduce effective utterance length )
|No
|Yes
|-
|Strength 
|
*''<u>Segmentation</u>'' of the acoustic signal can effectively  reduce the amount of speech data searched. 
* ''<u>Heuristic speech knowledge</u>'' serves as a practical  guide to mapping and rating words. 
|
*The network data representation of combined knowledge of                                                             <u>''syntax''</u> and the ''<u>phonetic</u>'' dictionary spellings is tractable. 
* The dynamic programming scheme for searching all the network 
* paths in parallel guarantees both a recognition will always be achieved and that the recognition will be the globally optimum one. ( with the given model ) 
|
* Searches only a few "best" syntactic (and acoustic) paths in parallel to determine the optimal path, and uses segmentation to effectively reduce the utterance length.

* Semi-automated generation and tuning of lexicons and templates greatly improved the accuracy of the speech recognition system.

* Has 91% sentence accuracy across five speakers at 30 MIPSS
|-
|Weakness 
|
*''<u>Best-first</u>'' strategy involves <u>''backtracking''</u>,  which can be costly in large search space.
|
*Searching all paths ( even in parallel ) is time consuming. 
|
* The computation of autocorrelation and linear predictor coefficients takes up the majority of processing time in the Harpy speech recognition system (over 64%).
|}