Carnegie Mellon's Harpy System: Difference between revisions
No edit summary |
|||
| Line 11: | Line 11: | ||
== Introduction == | == Introduction == | ||
'''Harpy System''' is a result of combing and improving the best features from two speech systems previously developed at Carnegie Mellon University: '''Hearsay-I''' ( Erman 1974 ) and '''Dragon''' ( Baker 1975 ). Two feature of the Harpy system that led to a successful demonstration are its '''representation of knowledge''' and the use of '''new search techniques'''. In developing speech recognition systems, it is necessary to devise a means of acquiring and representing the many diverse types of knowledge that characterize speech. We must also develop '''matching''' and '''searching''' techniques that convert this passive knowledge into an active process for understanding the utterance in the presence of error, noise, and uncertainty. | |||
When an utterance is input to the computer. Several pre-processing steps are taken to prepare the data for recognition. The utterance is '''segmented into acoustic units''' and analyzed to determine the segmented features and parameters. The goal of recognition tasks is to find '''an optimal sequence''' of phones satisfying two criteria: '''the sequence must represent a legal path through the knowledge network''' and should '''consist of phones with high acoustic matches'''. | |||
Harpy | Harpy Systems uses a '''beam search''' to locate this optimal sequence of phones. Harpy begins the beam search by taking all legal phones from the start of the sentence from the knowledge network and entering them in a '''recognition tree'''. Next, a path probability is calculated for each candidate. The path with the best probability is determined and remaining candidates are compared with it. '''Those that fall below a threshold of acceptability are eliminated from further searching'''. The successors of the surviving candidates are expanded based on the information in the knowledge network. | ||
{| class="wikitable" | {| class="wikitable" | ||
|+ | |+ | ||
| Line 70: | Line 28: | ||
| | | | ||
*Acoustic-Phonetics (the characteristics of the sounds) | *Acoustic-Phonetics (the characteristics of the sounds) | ||
* Prosodics (the stress and intonation patterns of speech) | |||
* Lexicon (a dictionary of legal words) | |||
* Syntax (the grammatical structure of the language) | |||
* Semantics (the meaning of words and sentences) | |||
* Pragmatics (the context of the conversation) | |||
| | | | ||
*Almost '''no''' speech dependent heuristics. | |||
*The integrated knowledge of both syntax and lexical spellings, a set of user dependent acoustic-phonetic templates, and an acoustic-phonetic probability matching routine. | |||
*Mathematical acceptability available ---one of probabilistic function of '''Markov Process'''. | |||
| | | | ||
*The combination of speech dependent heuristics ( Hearsay-I System ) and mathematical tractable model ( Dragon System ). This leverages the use of heuristics to reduce the search space therefore speed increase. | |||
*The system extensively use a network that represents both all legal syntactic paths and pronunciations of these legal paths. | |||
*No a-priori transition probabilities. | |||
|- | |- | ||
| | |Knowledge Representation | ||
| | |Procedural embedding. | ||
|Markove networks. | |||
|Transition networks. | |||
| | |||
| | |||
|- | |- | ||
|Search Strategy | |Search Strategy | ||
|Best-First | |Best-First with backtracking. | ||
| | |All paths in parallel ( search all the possible syntactic | ||
and acoustic paths through the network in parallel to determine optimal path ) | and acoustic paths through the network in parallel to determine optimal path ) with no backtracking. | ||
|'''Beam | |<nowiki>''best few''</nowiki> in parallel with no backtracking. | ||
'''Beam Search''' | |||
* Locates optimal solution | * Locates optimal solution | ||
* Examines near-miss alternatives | * Examines near-miss alternatives | ||
* No backtracking | * No backtracking | ||
|- | |||
|Segmentation | |||
|Yes | |||
( uses it to reduce effective utterance length ) | |||
|No | |||
|Yes | |||
|- | |- | ||
|Strength | |Strength | ||
| | | | ||
*''<u>Segmentation</u>'' of the acoustic signal can effectively reduce the amount of speech data searched. | |||
* ''<u>Heuristic speech knowledge</u>'' serves as a practical guide to mapping and rating words. | |||
| | | | ||
| | *The network data representation of combined knowledge of <u>''syntax''</u> and the ''<u>phonetic</u>'' dictionary spellings is tractable. | ||
* The dynamic programming scheme for searching all the network | |||
* paths in parallel guarantees both a recognition will always be achieved and that the recognition will be the globally optimum one. ( with the given model ) | |||
|N/A | |||
|- | |- | ||
|Weakness | |Weakness | ||
| | | | ||
*''<u>Best-first</u>'' strategy involves <u>''backtracking''</u>, which can be costly in large search space. | |||
| | | | ||
| | *Searching all paths ( even in parallel ) is time consuming. | ||
|N/A | |||
|} | |} | ||
* simple intro of the relationship between hearsay-I, Dragon system, and harpy system | * simple intro of the relationship between hearsay-I, Dragon system, and harpy system | ||
| Line 128: | Line 88: | ||
* detailed info about hearsay-I and dragon | * detailed info about hearsay-I and dragon | ||
=== Hearsay System === | |||
*'''The Hearsay System''' must use all these sources of knowledge effectively in the Hearsay System, this is achieved by representing knowledge as as a set of '''cooperating parallel process'''. The Hearsay System also use the source of knowledge ('''acoustics-phonetics, prosody, lexicon, syntax, semantics and pragmatics''') to '''generate hypothesis''' about what words might appear. The Hearsay System is incrementally trying to match words to the un-analyzed portion of the utterance | |||
*If the resulting word matches are not optimal, the system '''backs up and tries a different path'''. The technique was known as probabilistic tree. The Hearsay System '''cannot''' recognize sentences in unrestricted English and can only recognize simple phrase structure languages. | |||
*To be acceptable as an input medium, the system must not only perform with high accuracy, but it must also minimize the combinatorial explosion so as to respond as fast as a human would in a similar situation. | *To be acceptable as an input medium, the system must not only perform with high accuracy, but it must also minimize the combinatorial explosion so as to respond as fast as a human would in a similar situation. | ||
* | |||
=== Dragon System ( Janice ) === | |||
*Dragon | |||
*Dragon | |||
*Dragon | |||
{| class="wikitable" | {| class="wikitable" | ||
|+ | |+ | ||
| Line 135: | Line 105: | ||
!Hearsay System | !Hearsay System | ||
!Dragon System | !Dragon System | ||
|- | |- | ||
| | |Model | ||
| | |cooperating parallel process | ||
|probabilistic function of '''Markov Process'''. | |||
|- | |||
|knowledge representation | |||
|Procedures. | |||
|Markov Networks. | |||
|- | |||
| | |Unit of Speech | ||
| | Signal Representation | ||
|segmentation | |||
|10 milli-second sample | |||
|- | |||
|search strategy | |||
|best-first | |||
|all paths in parallel | |||
|- | |||
|search time | |||
|8-50 times real-time | |||
|45-200 times real-time | |||
|- | |- | ||
|What Have Been Tested | |What Have Been Tested | ||
| Line 157: | Line 136: | ||
28-76 word vocabularies | 28-76 word vocabularies | ||
| | |Janice, if you can't find relevant info you can put N/A | ||
|- | |- | ||
|Performance | |Performance | ||
| Line 169: | Line 147: | ||
(as of October, 1973) | (as of October, 1973) | ||
| | |Janice, if you can't find relevant info you can put N/A | ||
|- | |- | ||
| | |Approaches | ||
| | |acoustics-phonetics, prosody, lexicon, syntax, semantics and pragmatics | ||
| | |Treats speech recognition as a mathematical computation problem | ||
rather than as an artificial intelligence problem. | |||
|} | |} | ||
Revision as of 16:08, 17 September 2023
Sherry Yu-Ting
Wansu
Janice
Wenjun Meng
Carnegie Mellon's Harpy System
Jump to navigationJump to search
Introduction
Harpy System is a result of combing and improving the best features from two speech systems previously developed at Carnegie Mellon University: Hearsay-I ( Erman 1974 ) and Dragon ( Baker 1975 ). Two feature of the Harpy system that led to a successful demonstration are its representation of knowledge and the use of new search techniques. In developing speech recognition systems, it is necessary to devise a means of acquiring and representing the many diverse types of knowledge that characterize speech. We must also develop matching and searching techniques that convert this passive knowledge into an active process for understanding the utterance in the presence of error, noise, and uncertainty.
When an utterance is input to the computer. Several pre-processing steps are taken to prepare the data for recognition. The utterance is segmented into acoustic units and analyzed to determine the segmented features and parameters. The goal of recognition tasks is to find an optimal sequence of phones satisfying two criteria: the sequence must represent a legal path through the knowledge network and should consist of phones with high acoustic matches.
Harpy Systems uses a beam search to locate this optimal sequence of phones. Harpy begins the beam search by taking all legal phones from the start of the sentence from the knowledge network and entering them in a recognition tree. Next, a path probability is calculated for each candidate. The path with the best probability is determined and remaining candidates are compared with it. Those that fall below a threshold of acceptability are eliminated from further searching. The successors of the surviving candidates are expanded based on the information in the knowledge network.
| Hearsay System | Dragon System | Harpy System | |
|---|---|---|---|
| Sources of Knowledge |
|
|
|
| Knowledge Representation | Procedural embedding. | Markove networks. | Transition networks. |
| Search Strategy | Best-First with backtracking. | All paths in parallel ( search all the possible syntactic
and acoustic paths through the network in parallel to determine optimal path ) with no backtracking. |
''best few'' in parallel with no backtracking.
Beam Search
|
| Segmentation | Yes
( uses it to reduce effective utterance length ) |
No | Yes |
| Strength |
|
|
N/A |
| Weakness |
|
|
N/A |
- simple intro of the relationship between hearsay-I, Dragon system, and harpy system
- general info about how harpy system works (features, methods
- simply mention the advatages of harpy sytem
Historical Context
- detailed info about hearsay-I and dragon
Hearsay System
- The Hearsay System must use all these sources of knowledge effectively in the Hearsay System, this is achieved by representing knowledge as as a set of cooperating parallel process. The Hearsay System also use the source of knowledge (acoustics-phonetics, prosody, lexicon, syntax, semantics and pragmatics) to generate hypothesis about what words might appear. The Hearsay System is incrementally trying to match words to the un-analyzed portion of the utterance
- If the resulting word matches are not optimal, the system backs up and tries a different path. The technique was known as probabilistic tree. The Hearsay System cannot recognize sentences in unrestricted English and can only recognize simple phrase structure languages.
- To be acceptable as an input medium, the system must not only perform with high accuracy, but it must also minimize the combinatorial explosion so as to respond as fast as a human would in a similar situation.
Dragon System ( Janice )
- Dragon
- Dragon
- Dragon
| Hearsay System | Dragon System | |
|---|---|---|
| Model | cooperating parallel process | probabilistic function of Markov Process. |
| knowledge representation | Procedures. | Markov Networks. |
| Unit of Speech
Signal Representation |
segmentation | 10 milli-second sample |
| search strategy | best-first | all paths in parallel |
| search time | 8-50 times real-time | 45-200 times real-time |
| What Have Been Tested | 144 connected utterances
676 words 5 speakers 4 tasks: Chess, News Retrieval, Medical Diagnosis, Desk Calculator 28-76 word vocabularies |
Janice, if you can't find relevant info you can put N/A |
| Performance | % of words spotted and identified correctly
89% with all the sources of knowledge 67% without semantics 44% without syntax and semantics (as of October, 1973) |
Janice, if you can't find relevant info you can put N/A |
| Approaches | acoustics-phonetics, prosody, lexicon, syntax, semantics and pragmatics | Treats speech recognition as a mathematical computation problem
rather than as an artificial intelligence problem. |
Harpy Sytem
- detailed info of the current system
- with a table comparing three systems
Impact
- advantages (improvements on speech recognition tech
Key Innovations
- how the optimization make the improvements
- implications
Future research
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Cite error: The opening <ref> tag is malformed or has a bad name
References
To insert a reference, type <ref> and paste the source your exported from Zotero (or whatever reference manager you're using) in the pop up box which appears. Make sure links in citations are clickable using proper formatting. Once you do this, a footnote will appear.Cite error: The opening <ref> tag is malformed or has a bad name and a reference comes at the end automatically. Please use this method to cite for Wiki articles only, not for your thesis.
Insert paragraph [1]
- ↑ Lowerre, B., & Reddy, R. (1976). The harpy speech recognition system: performance with large vocabularies. The Journal of the Acoustical Society of America, 60(S1), S10-S11.