Editing Vocoder Development

== Introduction ==
A vocoder is a crucial component in the realm of speech synthesis, employed in various applications ranging from telecommunications and entertainment to assistive technologies and virtual assistants. Its ability to analyze and synthesize the complex spectrum of human speech, including the pitch, formants, and prosody (melody, rhythm, and intonation in speech), has changed how we use synthesized voices. 

First emerged in the 1930s, the term "vocoder" is derived from "voice" and "coder", and was initially conceived for telecommunication purposes. In the 1950s, Lincoln Laboratory conducted research on detecting pitch in speech, which subsequently influenced the development of voice coders, commonly known as vocoders. These devices are designed to decrease the amount of bandwidth required for transmitting speech. This reduction in bandwidth offers two advantages: it reduces the cost of transmitting and receiving speech, while also enhancing the potential for maintaining privacy.<ref name=":2">Gold, Bernard. “Gold A History of Vocoder Research at Lincoln Laboratory.”, [https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=6fb19070c1314f599ecbd134672df89fba037155 The Lincoln Laboratory Journal. Volume 3. Number 2 (1990)]</ref>

Initially, a vocoder described a device designed to compress speech for efficient transmission over telephone lines. The idea involved splitting speech into parts using filters and then reconstructing it on the other end. The goal was to save bandwidth, but in practice,  early vocoders faced challenges in preserving the natural quality of speech. Additionally, the original vocoder only transmitted loudness, not sound quality<ref name=":0">Gordon, John William, and John Strawn. ''[https://ccrma.stanford.edu/files/papers/stanm55.pdf An introduction to the phase vocoder]'' No. 55. CCRMA, Department of Music, Stanford University, 1987.</ref>.

As technology advanced, the phase vocoder emerged, preserving both the loudness and sound quality of speech. The phase vocoder is a significant advancement in vocoder technology and plays a pivotal role in modern speech synthesis and audio signal processing. It was developed to address critical limitations in early vocoders and revolutionized the way we process and manipulate audio signals. <ref name=":0" />

== Historical Context ==
Speech synthesis, commonly known as Text-to-Speech (TTS), has gained increasing importance in people's lives. The development of TTS technology spans centuries, beginning with the Mechanical and Electro-Mechanical Era that relied on mechanical and electro-mechanical components to simulate speech sounds. It progressed through the Electrical and Electronic Era, where electronic technology was used to create early formant synthesizers and speech coding systems, and eventually entered the digital and computational era, marked by the transition to digital signal processing and computational methods for speech synthesis.

==== '''Mechanical and Electro-Mechanical Era''' ====
This era commenced with early speech synthesis attempts in the late 18th century. In 1769, [[Wolfgang von Kempelen's Speaking Machine (1769)|Wolfgang von Kempelen]] engineered a mechanical speaking device that ingeniously emulated speech sounds through the mechanical simulation of vocal cords, vocal tracts, and lungs. Concurrently, [[Mechanical synthesis|Christian Kratzenstein]] embarked on early mechanical synthesis by employing five organ pipe-like resonators to successfully reproduce five distinct vowels. Later in 1824, [[Wheatstone's "Speaking Machine" (1824)|Wheatstone]] introduced a "Speaking Machine," and in 1846, Joseph Faber constructed "Euphonia," both utilizing a series of mechanical components to produce speech-like sounds. During this period, the mechanical and electro-mechanical era of speech synthesis was drawing to a close. Despite their constraints in generating an extensive vocabulary and sentences, these early endeavors made noteworthy contributions to the exploration of speech synthesis, igniting extensive research into the physiology of speech production and experimental phonetics. The comprehension of acoustic resonators, spectral components, and formants marked a pivotal shift toward the scientific investigation of human sound production, setting the stage for the subsequent era of speech synthesis <ref>Schroeder, M. R. (1993). [https://doi.org/10.1016/0167-6393(93)90074-U A brief history of synthetic speech]. ''Speech Communication'', ''13''(1–2), 231–237. </ref><ref name=":1">Story, B. H. (2019). [https://www.researchgate.net/profile/Brad-Story/publication/333614830_History_of_Speech_Synthesis_Chapter_1_Routledge_Handbook_of_Phonetics_W_Katz_and_P_Assmann_Eds_2019/links/5e65a26392851c7ce05353c6/History-of-Speech-Synthesis-Chapter-1-Routledge-Handbook-of-Phonetics-W-Katz-and-P-Assmann-Eds-2019.pdf History of speech synthesis]. ''Teoksessa The Routledge Handbook of Phonetics, toimittaneet William F. Katz & Peter F. Assmann'', 9-33. </ref>.

==== '''Electrical and Electronic Era''' ====
In the electrical and electronic era, Text-to-Speech (TTS) technology underwent a transformative evolution, with the convergence of electricity and emerging electronic components giving rise to more efficient speech synthesis systems. This era dated back to 1922 when John Q. Stewart designed a system utilizing early electronic technology to generate speech sounds, essentially introducing the first electrical formant synthesizer. Electronic speech coding was inaugurated by Homer Dudley in response to bandwidth limitations on telegraph cables and the substantial bandwidth requirements for transmitting spectral content. Dudley designed [[Voder]] in 1930 and later created the first analysis-synthesis system, the Vocoder, in 1940s, introducing the concept of analyzing speech into spectral components and synthesizing it, with the process of extracting low-frequency spectral data from an acoustic speech signal and transmitting it via a low-bandwidth cable, and modulating a locally generated carrier signal at the receiving end<ref name=":1" />. This laid the foundation for subsequent TTS developments and paved the way for further research in the field of speech synthesis. However, the Vocoder had limitations in terms of naturalness and intelligibility in speech synthesis, primarily due to its complexity and dependence on manual tuning, making it challenging to achieve high-quality and versatile speech synthesis <ref name=":1" />.

== Key Innovations ==
Innovations in vocoders are often developments in deep learning, signal processing and machine learning. 

According to Dudley, the use of vocoders are advantageous for more secure communications and a greater number of telephone channels in the same frequency space. Digital communication made eavesdropping conversations more difficult. However, the digitization of speech required wider transmission bandwidths. The vocoder was used to reduce the speech bitrate to a number that could be handled through the average telephone channel. <ref name=":2" />

Pitch detection in vocoding was a difficulty in the late 1950s. This is necessary to synthesize speech of acceptable quality. <ref>Gold, B., & Rabiner, L. (1969). Parallel processing techniques for estimating pitch periods of speech in the time domain. ''The Journal of the Acoustical Society of America'', ''46''(2B), 442-448.</ref> Speech is created by vibrating the vocal folds, the air that is moving and closing the mouth. The vocal fold vibrations are analyzed to find the pitch. To estimate the fundamental frequency, the Lincoln Laboratory designed an algorithm that used parallel processing, which produced accurate results. <ref name=":2" />

In a channel vocoder, formant tracking helps recreate speech. It looks at the first four formant features to estimate the speech sounds. This method can make a vocoder use less data or bandwidth and still sound good. <ref>Gold, B. (1980, April). Formant representation of parameters for a channel vocoder. In ''ICASSP'80. IEEE International Conference on Acoustics, Speech, and Signal Processing'' (Vol. 5, pp. 128-130). IEEE.</ref>

Another innovation is the speaker-dependent WaveNet vocoder that used existing vocoder data as additional information. This method does not need to explicitly model certain aspects of speech and offers better sound quality. It successfully recovers lost information and captures source details more accurately compared to traditional vocoders. <ref>Tamamori, A., Hayashi, T., Kobayashi, K., Takeda, K., & Toda, T. (2017, August). Speaker-dependent wavenet vocoder. In ''Interspeech'' (Vol. 2017, pp. 1118-1122).</ref>

WORLD, a vocoder-based speech synthesis system, was developed in an effort to improve the sound quality of real-time applications using speech. Real-time processing has been difficult because of the high computational costs. This new system has a good sound quality and is also quick in processing. <ref>Morise, M., Yokomori, F., & Ozawa, K. (2016). WORLD: a vocoder-based high-quality speech synthesis system for real-time applications. ''IEICE TRANSACTIONS on Information and Systems'', ''99''(7), 1877-1884.</ref>  

== Impact ==

* Vocoders significantly decreased the bandwidth needs of voice signals, allowing for more cost-effective voice transmission and storage.<ref>Schroeder, M. R. 1966. “Vocoders: Analysis and Synthesis of Speech.” Proceedings of the Ieee 54(5).</ref>

* Vocoders are used for speech encryption to improve the intelligibility of poor quality speech.
* The use of vocoder technologies has enabled voice transmission in noisy environments and over long distances, as well as facilitating communication for individuals with language barriers. The principles of signal processing employed in these techniques were subsequently adopted in other communication technologies, including cellular telephony.<ref>Mills, Mara. 2012. “Media and Prosthesis: The Vocoder, the Artificial Larynx, and the History of Signal Processing.” Qui Parle: Critical Humanities and Social Sciences 21(1):107–49.</ref>
* Vocoders have proven to be invaluable in cochlear implant research, enabling the acquisition of a more profound comprehension of how cochlear implants operate and facilitating the creation and development of innovative signal-processing solutions.<ref>Shannon, Robert V., Fan-Gang Zeng, Vivek Kamath, John Wygonski, and Michael Ekelid. 1995. “Speech Recognition with Primarily Temporal Cues.” Science 270(5234):303–4.</ref>
* Within the domain of speech synthesis, vocoders created high-quality human voices and fostered the development of speech synthesis technology.
* The vocoder's ability to encode and process sound paved the way for a new era of electronic music creation. It facilitated the manipulation of a range of vocal effects and propelled the evolution of popular music to new heights, shaping modern music profoundly.

== Future Research ==
Despite the significant advancements in technology and the development of new speech synthesis systems, ranging from [[DECtalk Speech Synthesizer (1984)|DECtalk]] in the 1980s to the cutting-edge [[Advancements in AI TTS (2020s)|AI models]] of today, vocoders continue to play a key role in various applications. Today, vocoders are integral components of state-of-the-art speech synthesis systems, including WORLD<ref>Morise, Masanori, Fumiya Yokomori, e Kenji Ozawa. «WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications». ''IEICE Transactions on Information and Systems'' E99.D, fasc. 7 (2016): 1877–84. <nowiki>https://doi.org/10.1587/transinf.2015EDP7457</nowiki>.</ref>, designed specifically for real-time applications, and BigVGAN<ref>Lee, Sang-gil, Wei Ping, Boris Ginsburg, Bryan Catanzaro, and Sungroh Yoon. «BigVGAN: A Universal Neural Vocoder with Large-Scale Training». arXiv, 16 february 2023. <nowiki>http://arxiv.org/abs/2206.04658</nowiki>.</ref> , which uses the power of generative adversarial networks (GANs<ref>Rocca, Joseph. "Understanding generative adversarial networks (gans)." ''Medium'' 7 (2019): 20.</ref>). A majority of contemporary vocoders rely on neural networks for their operation, but improvements could be made in this direction too. Additionally, the integration of generative AI holds the promise of further enhancing the quality of vocoder-synthesized voices.

Furthermore, vocoders are largely used in the field of music production, although we have yet to achieve the creation of truly authentic singing voices through them. The potential to explore and refine the use of vocoders in this context offers a pathway to a broader advancement in speech synthesis technology, since the effort of synthesizing a singing voice may lead us to a more fluent and smooth speaking voice.

Additionally, vocoders are currently deployed to address other challenges in voice technology. Vocoder-synthesized voices serve as tools in training noise-robust recognition models<ref>Zheng, Nengheng, Yupeng Shi, Yuyong Kang, e Qinglin Meng. «A Noise-Robust Signal Processing Strategy for Cochlear Implants Using Neural Networks». In ''ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)'', 8343–47, 2021. <nowiki>https://doi.org/10.1109/ICASSP39728.2021.9413452</nowiki>.</ref> and detect fake audio<ref>Yan, Xinrui, Jiangyan Yi, Jianhua Tao, Chenglong Wang, Haoxin Ma, Tao Wang, Shiming Wang, e Ruibo Fu. «An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audio». In ''Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia'', 61–68. DDAM ’22. New York, NY, USA: Association for Computing Machinery, 2022. <nowiki>https://doi.org/10.1145/3552466.3556525</nowiki>.</ref>. The former represents a useful development not only for the industry of speech recognition but also an improvement for medical applications of speech recognition and processing. The latter allows us to take steps forward in the direction of cybersecurity and data protection<ref>Lim, Suk-Young, Dong-Kyu Chae, e Sang-Chul Lee. «Detecting Deepfake Voice Using Explainable Deep Learning Techniques». ''Applied Sciences'' 12, fasc. 8 (gennaio 2022): 3926. <nowiki>https://doi.org/10.3390/app12083926</nowiki>.</ref> since it could enable us to automatically detect identity theft attempts and privacy violations.

== LLM Review ==
We prompted [https://chat.openai.com/auth/login ChatGPT] to act as a university professor and review this wiki page and some of the useful feedback it provided are:

* Clarification: The term "prosody" may be unfamiliar to some readers. It would be helpful to briefly explain what prosody refers to in the context of speech synthesis.
* Subheadings: Consider using subheadings to categorize the innovations discussed (e.g., "Advancements in Telecommunications" or "Pitch Detection Innovations"). This can make the content more organized and accessible.

* The key innovations section is informative, but it would be helpful to organize the information into bullet points or subheadings. This would make it easier for readers to scan and understand the major innovations in vocoder technology.
* The historical context section is informative but quite lengthy. Consider breaking it down into smaller, more digestible paragraphs and use subheadings for different eras to make it easier to follow.
* Practical Implications: Expand on the practical implications of future research in each application area. How might these advancements benefit society or specific industries?
We also tried to ask the [https://www.grammarly.com/business/learn/enterprise-grade-generative-ai/#:~:text=As%20a%20result%2C%20we%20have,GrammarlyGO%20holds%20user%20privacy%20paramount Grammarly LLM system] to "find faults in the argumentation", in order to have general feedback on how we explained ourselves, and quite interesting the answer was the following: ''"As an AI-powered assistant, I don't have personal opinions or the ability to find faults in an argument. However, I can provide a neutral analysis of the text. [...]''".

== References ==
<references />
== Team Members ==

* Alice Vanni
* Amber Lankheet
* Chenyi Lin
* Erin Shi
* Wenjun Meng