Editing
State-of-the-art
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
==== Klumpp et al. (2023): Synthetic cross-accent data augmentation for ASR ==== '''Summary:''' Foreign-accentes speech is usually underrepresented in, if not absent from speech corpora. Auxiliary input (learned accent embeddings, intermediate wav2vec2.0 representations) can address the decreased ASR recognition on this type of speech; the challenge remains that of achieving good accent conversion while preserving source speaker voice characteristics. The current approach builds on a pre-existing ACM by Jin et al. (2023) -- see above -- and aims to provide synthetic ASR training data using it. Phonetic knowledge is crucially injected into training to improve accent-specific pronunciation, and learnable accent representations are introduced to allow for variable accent strengths and adaptability to unseen accents. The experimental setup involved evaluating two ASR models using Librispeech data. The first model (Base) utilized an efficient memory transformer followed by a recurrent neural transducer (RNNT), while the second model (HuBERT) had a similar structure with adjustments in channel configurations and dropout probabilities. The ASR models were tested on Librispeech data and accents from L2-Arctic corpus and Accented Vox Populi (AVP) dataset. In experiments, the baseline ASR systems were trained without synthetic accented speech data, then evaluated. Three additional ASR models were trained with a combination of real and synthetic accented data, using a ratio of 80% real and 20% synthetic data. The ratio remained consistent across all accents. Finally, learned accent embeddings from L2-Arctic samples were visualized using t-SNE plots to assess their suitability for encoding accent information in an Accent Conversion Model (ACM). '''RQ:''' Is it possible to improve ASR of accented speech with synthetic samples of a particular accent? '''Results:''' The inclusion of one synthetic accent during ASR training had a positive effect on recognition results for that particular accent, which was a clear indicator that the ACM was able to synthesize a sufficient degree of accentedness. At the same time, HuBERT'd performance decreased with the use of synthetic data, likely due to the fact that it was not pre-trained on any and fine-tuning did not do enough. The Base model, which was trained from scratch, had a much grater benefit from the synthetic data. Notably, even when all seven accents were introduced in training, this did not improve performance on other unseen accents. Overall, including one synthetic accent improved performance on that accent; and including several accents improved performance on those accents, but none of the conditions improved recognition on accents not seen in training. Additionally, pre-trained HuBERT did not benefit much from additional synthetic data fine-tuning, whereas a model trained from scratch saw much greater benefit from this approach. '''Critical observations:''' Again, none of this replicable because the code is not available. It would have been also interesting to see a bit more ASR models be tested on this; this particular comparison does highlight the pre-trained/trained from scratch distinction in performance on this task, but there are other models that are seemingly good candidates and were not included. '''Relevance:''' The authors show the potential for using synthetically accented data as a data augmentation approach to improve ASR performance on foreign-accented speech.
Summary:
Please note that all contributions to MSc Voice Technology are considered to be released under the Creative Commons Attribution (see
MSc Voice Technology:Copyrights
for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource.
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
Edit source
View history
More
Navigation
Main page
Recent changes
Random page
Help about MediaWiki
Tools
What links here
Related changes
Special pages
Page information