Introduction of Voice Assistants

Introduction

A "Voice Assistant" is a computer program that can converse with a human and carry out tasks by following instructions or receiving information through voice commands (from the Oxford Learners Dictionary definition of "Voice assistant"). With the emergence of AI in the last two decades, this type of program has advanced significantly, thanks to technological breakthroughs like Hidden Markov Models, and Neural Networks that have improved their speech recognition capabilities, as well as improvements in audio signal quality. Today, Voice Assistants are an integral part of our daily lives¹, present in nearly every personal and home device.

Historical Context

Early stages of virtual assistants

The development of virtual assistants has come a long way since the early days of computer science. The very first voice recognition system was Shoebox, developed by IBM in the 1960s, which could only recognize 16 words and a few digits. Nonetheless, it was a real revolution for the field.

As ASR technology progressed, text-based virtual assistants were implemented. The first text-based virtual assistant, or “chatbot”, of the modern era was Clippy², the famous Microsoft paperclip assistant, released in late ‘90 for Microsoft Office on Windows computers, but in fact, chatbots have existed since Alan Turing’s first machine in 1950. The first chatbot ever created was ELIZA. It was based on a pattern-matching algorithm and it simulated a psychotherapy session, and could only answer a limited set of questions.

Since the very beginning of Computer Science, there has been a keen interest in communicating with machines, particularly through voice. This is due to the significant role that voice and spoken language play in human communication and culture. The first consumer-oriented voice recognition system was Dragon Dictate. This program had some recognition difficulties but later on, these were overcome, which is where the first Voice Assistants were created.

The very first system of speech recognition, which enabled later Voice Assistants to work on a vast scale, was developed by Windows engineers around 2004 and released in 2007 alongside the Windows Vista operating system. Although not precisely a virtual assistant, the technology implemented made it possible to give simple spoken instructions to the system and dictate text to it.

The advent of Siri

In October 2011, Apple Inc. introduced Siri alongside the iPhone 4S³, marking the emergence of what we now recognize as a voice assistant. Siri was a groundbreaking development, utilising advanced Machine Learning technology and Natural Language Processing, first developed as a research project at Stanford University's Center for Computation and Natural Language in 2003. Upon its release, Siri generated excitement and apprehension from the public, akin to the reception of LLM-based bots today. As the first natively implemented voice "personal assistant" in a phone, Siri was initially regarded as a source of amusement, but it has since become one of the most commonly used virtual assistants.

The Era of Voice Assistants

Since Siri’s debut in 2011, many other tech companies have introduced their Voice Assistant (VA) systems, enhancing their speech recognition algorithms with the release of new voice bots nearly every year until 2017. The second VA to emerge in the marketplace was Google Now, introduced during the winter of 2012. Google Now was considered to have outperformed Siri, for its capacity to proactively meet user needs by accessing data such as calendar, location, and other user-specific information.

Google Now was followed by Microsoft’s Cortana⁴, presented at the BUILD developer conference on the 2nd of April 2014. Cortana not only was able to retrieve information from the user’s devices, as Google Now did, but it was also able to use Bing knowledge graph Satori to enhance online searches. The formal release of Cortana happened in 2015, together with the introduction of the new Windows 10 operating system.

In the meantime, in 2014 Amazon's Alexa disrupted the market by becoming the first standalone voice assistant. Alexa not only had a better machine-learning algorithm, but when it was launched, Amazon announced that it would have its own separate device, namely Amazon Echo, which was released only one year later in the US, and two years later in the UK and Europe. Until this release, voice assistants were part of other, more complex products (e.g. PCs, smartphones), Alexa together with Amazon Echo was the first stand-alone VA⁵.

In 2016, Google entered the VA arena with Google Assistant, initially accessible exclusively through the messaging app Allo, followed by its release for Google smart devices, particularly Google Home, later that year.

The latest company to enter the VA market was Samsung, which released its assistant Bixby only in 2017. Probably as a consequence of this late arrival, Bixby never affirmed itself on the market, remaining in the back rows of Voice Assistants.

Key Innovations

The recent development of voice assistants has been driven by key innovations in the field. One significant factor contributing to their widespread adoption is the integration of voice assistants into smartphones and mobile devices, making this technology accessible to a broader audience. Several factors have facilitated this development, including increased computing power, access to extensive linguistic (speech) data, advancements in machine learning, and a deeper understanding of human language in context.

The foundation of modern voice assistants is rooted in the advancements of speech recognition technology, a field that has been evolving for several decades. The origins of this research can be traced back to the 1960s, with notable contributions from companies like IBM. The improvement of speech recognition accuracy through machine learning techniques has enabled the development of voice assistants capable of accurately understanding and interpreting human speech. Additionally, these software systems utilize natural language processing (NLP) to comprehend spoken words and discern the user's intent. Hardware advancements, including miniaturization, microphone enhancements, and the development of compact, low-power processors, have further contributed to the practicality of voice assistants, allowing them to process commands locally. Another crucial aspect introduced by machine learning and data analytics within voice assistants is continuous personalization. This feature offers users a tailored experience by providing personalized recommendations based on individual preferences, interaction history, and behaviour.

Voice assistants have made their way into various industries, with approximately 123.5 million US adults using them monthly. It is anticipated that this number will increase to nearly 49% of US adults over the next three years. These software systems are primarily utilized on smartphones and smart speakers, with Amazon Echo holding a prominent position in the US smart speaker market.

Impact

Impact on speech recognition

The accuracy of speech recognition since the Voice Assistants were created, has improved a lot. Datasets are still learning and recognizing more speech. This makes these assistants more user friendly.

Also the Natural Language Processing (NLP) has been improved over the years. This makes it possible for the VA’s to comprehend and respond to a wider range of questions in a more contextually relevant manner.

Using and collecting the data of different users, it raised privacy and security concerns. This will be discussed in the Criticism section, which has led to research and develepments in secure voice recognition.

Impact on other industries

As already mentioned, Voice Assistants are used in our daily lives. Here there are different categories, such as music, hands-free search, and other uses of Voice Assistants. This category is mostly used to ask for jokes. It is also used to help people with reading impairments and help people to understand difficult texts, like prescriptions. Also, for people who are blind and there is not a braille text available. There has been research about people with dementia who can benefit from a voice assistant that can answer the same questions over and over again without losing patience and encourage the patients when needed.  During the Covid-19 pandemic, hospitals were struggling with maintaining inpatient and outpatient services for other healthcare needs. Voice Assistants have been used to deliver healthcare information, which reduced stress on the health system.

Criticism

Security and privacy issues

There also have been some issues regarding the security and privacy of Voice Assistance. For example a voice reveals someones identity, emotion, gender, ethnicity or cultural background. Users can ask to read out loud their personal information, emails, calendar contents, etc. To prevent others from asking about personal details, Google upgraded its Voice Assistant software to include voice printing, which identifies each user by voice. There have been some cases where anyone from outside could unlock someone’s house by asking Siri to unlock the door. The voice prints can also be used to recognize an individual’s unique voice in cases of fraud or theft. Amazon’s Alexa had a voice passcode that was introduced to confirm purchases.

Voice Assistants are also vulnerable to attacks, like commands that are delivered at ultrasonic frequencies. For example, a television product advertisement with embedded ultrasound commands can add items to your shopping list.

Also, do these devices always listen? They have to respond by specific cues, so they always need to listen. According to Amazon, Apple, MS and Google all insist that the devices only record when triggered by the wake word, there have been recorded cases of malfunctioning devices recording their surroundings uninterruptedly and sending those recordings to the company’s servers.

Gender bias

Voice assistants represent the fundamental departure from earlier technologies relying on sterile, keyword-driven commands typed on a keyboard; instead they harness the power of speech, deemed as the most natural form of human communication. However, an aspect of this transition that has drawn significant criticism from researchers in fields of social studies is the adoption of nurturing, female voices for voice assistants, with a profound impact on how users engage with and perceive them.

The use of female voices has contributed to asistants being seen as friendlier and more familiar to users, making interactions more engaging thanks to the perceived warmth, approachability and their coded 'readiness to help'. While some voice assistants do indeed offer male voice alternatives, it's important to note that the default and most well known voice for all leading voice assistants remains female. Such design choices are sparking discussions about reinforcing and perpetuating gender stereotypes and biases, raising questions about the roles assigned to technology and how these choices can further shape societal perceptions.

Future Research

Already, Voice Assistants can answer a great number of questions and it can also control our homes. But they can also be programmed to guide tours in museums to give information and answer questions. This would be quite appealing for students to encourage them visition exhibitions.

The world has gotten smaller because of the internet which allows us to stay in touch with people from different continents. Google’s assistant can help to translate real-time conversations when a user asks to translate. For now, there are around 40 different languages available, but in the future maybe minority languages will also be recognised and can be translated into any language.

The expected percentage of the population in the US who uses a Voice Assistant will slowly increase. This is a 3-year forecast, so it is not far in the future, but still interesting why this is because the speed of improvements is quite high.

Improving machine learning models requires a lot of personal data. Brands that collect data for these purposes should be transparent and separate this data from data that is sold to advertisers so that users of Voice Assistants can make decisions without the influence of advertisers.

LLM Review

Two different Large language models (LLM) were employed in reviewing the current article, as a tool to ensure information accessibility, adherence to grammar rules, coherency, and cohesion of information. The tools used to these ends were ChatGPT and Grammarly.

ChatGPT

After completing a comprehensive draft of the information to be included in the sections of this page, with the corresponding sources, the sentences were fed into ChatGPT, using the following prompts:

• Go through the following text and identify any incoherences, repetitive information, or convoluted phrases; make sure the text is objective, with a neutral tone, in line with the style of a Wikipedia article;
• Make it sound more academic;

The output of the second prompt was overly formal and had many aulic and archaic terms. The text was not incorrect, but overall it was often not appropriate.

E.g.

Original text: “Cortana not only was able to retrieve information from the user’s devices, as Google Now did, but it was also able to use Bing knowledge graph Satori to enhance online searches.”

ChatGPT rewriting: “Cortana not only replicated the information retrieval functionalities of Google Now but also harnessed the Bing knowledge graph Satori to augment internet-based research endeavours.”

Also, ChatGPT moved some paragraphs to the beginning or to the end of a section to make the text more organised.

Grammarly

Grammarly was used both to check for spelling errors and other formal mistakes and to improve the register of the text, prompting the LLM with the query “Make it sound more academic”. The formal errors it detected were not always actual errors; in some cases, the LLM did not correctly detect the subject in the sentence. In these cases, the sentences were rephrased. The “Make it sound more academic” prompt made register changes, substituting common-use words with more academic ones.

It has to be remarked that not every Grammarly rewriting output was included in the final text, since in some cases, aulic words were used too.

Notes

1. According to Voicebot.ai, 50% of the population use a Voice Assistant, either in their car, at home or through their smartphone. See also Section “Impact in various industries and individuals’ daily life”
2. In case you wonder: Clippy is still available! You can add it to your browser as an extension from here.
3. As a side note: the iPhone 4S was the first product launched by Apple after the company co-founder Steve Jobs stepped down and left the "throne" to Tim Cook.
4. The name is a reference to the Xbox game Halo: Cortana is in fact his AI companion.
5. As of 2019, eight Echo devices were released, including Echo Buds (wireless earbuds), Echo Frames (glasses) and Echo Loop (smart ring); all Echo devices are the hardware bridge to the Alexa software.

Group members

Alice Vanni

Amber Lankheet

Maria Tepei