This Northeastern researcher is using AI to give people their voices back, while giving others a voice for the first time
Professor Rupal Patel speaks during the Expeditions in Experiential AI seminar held in Curry Student Center room 440.
Photo by Matthew Modoono/Northeastern University
A person’s voice can say a lot about them — how they communicate, where they are from, their age. We often take our voices for granted, as we tend to think we’ll have them for our whole lives.
But that’s not always the case. Some people are born without the ability to speak at all. While others may lose their voice through illness, surgical interventions or other causes.
Northeastern professor Rupal Patel has spent her career researching how technology can be used to help give people their voices back — and for some, the ability to really be understood for the first time.
Why responsible AI is important to the future of business. Northeastern events will address best practices
“We’ve actually humanized this device that she walks around with that sounds much more like it could fit her voice,” Patel said.
When Patel started researching the topic around two decades ago, the voice assistive technology that was available left a lot to be desired, she said. People used text-to-speech devices that only had access to a handful of voices, they were limited and sounded very artificial.
“They were really just a uniform set of voices,” she said. “Even the voices we hear today for devices that talk are pretty generic-sounding. They’re usually a female somewhere in their 40s from the middle of the country.”
Patel sought to expand the number of voices on offer. One of the major reasons companies cited for why there were so few voices available was because the process of creating synthetic voices was time-consuming and expensive.
“The voice created for Siri, for example, took hours and weeks for this person to record all the audio, and then months again for engineers to go over that audio, clean it up, and then generate the synthetic voices,” she said.
For Patel, things took a turn in 2013, after she gave a TED talk about synthetic voices. At the end of the talk, she posed a question to the attendees: “What if you could volunteer your voice for someone in need?”
The responses were overwhelming.
“We honestly thought a couple of hundred people would say yes, ‘I’d be willing to record for someone.’ At the time, we needed about seven to eight hours of someone’s recording for us to then build a synthetic voice,” she said. “Well, interestingly, we got thousands of people volunteering to say yes, and not just people in the Boston area, but actually people from all over the world.”
Personalized text-to-speech voices help people with speech disabilities maintain identity and social connection
With such a wealth of responses, Patel was able to take her research out of the lab and launch VocaliD, which was founded in 2014.
To collect the voices at scale, Patel developed the Human VoiceBank, an online web app that allowed people from all over the world to record their voice.
That data proved to be invaluable, and over the past few years rapid advancements in machine learning have helped accelerate the space even further.
VocaliD has also helped people bank their voice for future use.
Patel recounted the story of a man who learned that he was going to lose his voice following a surgery that would involve removing his voice and two-thirds of his tongue. He found out he needed the surgery only two weeks before it was to be completed.
Over the course of those two weeks, Patel had the man read lines and lines of sentences that were fed into an AI model. By the time of his surgery, the man had a text-to-speech machine loaded up with a voice that sounded just like his own, Patel said.
“He actually used it for a couple of years before he passed,” Patel said. “Interestingly, his family became really attached to the voice. Even after he passed, the family continued to actually use the voice as a memory of him. Not on a daily basis, but one of things he really liked to do was read stories to his grandkids.”
Patel admits that some people might find that anecdote “a little creepy,” but noted that “people who are growing up with this technology are going to have a different sense of what it means to hear loved ones in the future.”
In addition to being used in medical settings, the company has also helped the entertainment industry, providing customers with custom voice-overs for ads and assisting podcasters with translating their episodes into different languages.
Cesareo Contreras is a Northeastern Global News reporter.
A person’s voice can say a lot about them — how they communicate, where they are from, their age. We often take our voices for granted, as we tend to think we’ll have them for our whole lives.
But that’s not always the case. Some people are born without the ability to speak at all. While others may lose their voice through illness, surgical interventions or other causes.
Northeastern professor Rupal Patel has spent her career researching how technology can be used to help give people their voices back — and for some, the ability to really be understood for the first time.
Professor Rupal Patel speaks during the Expeditions in Experiential AI seminar held in Curry Student Center room 440. Photo by Matthew Modoono/Northeastern University
Patel has been doing this work through her company, VocaliD, an AI company that uses patented technology to blend together recorded speech with machine learning to create synthetic voices.
In June 2022, VocaliD was acquired by Veritone Inc., an enterprise AI company. With the acquisition, Patel was made vice president of voice and accessibility. She is currently on leave from the university to help integrate VocaliD with Veritone, but was invited back to showcase her groundbreaking technology and the state of the industry.
“Every single one of you in this room has a unique voice,” Patel said during a recent event organized by Northeastern’s Institute for Experiential AI, for which she is a core faculty member.
In 2016, she and her team at VocaliD developed a natural sounding synthetic voice for a 9-year-old girl with cerebral palsy that they integrated into her text-to-speech machine. Previously, she had been using a preset voice that made her sound like a female adult.
They accomplished this through several steps.
While the girl was unable to form sentences, she was able to make a handful of sounds on her own. Patel’s team was able to analyze those sounds and determine their source characteristics.
They then had the girl’s sister, who was only a few years older than her, read out a sequence of 300 sentences.
From there, they combined the girl’s source characteristics with her sister’s voice and fed it into an AI neural net that was trained to develop an original voice for the girl.
Patel has been doing this work through her company, VocaliD, an AI company that uses patented technology to blend together recorded speech with machine learning to create synthetic voices.
In June 2022, VocaliD was acquired by Veritone Inc., an enterprise AI company. With the acquisition, Patel was made vice president of voice and accessibility. She is currently on leave from the university to help integrate VocaliD with Veritone, but was invited back to showcase her groundbreaking technology and the state of the industry.
“Every single one of you in this room has a unique voice,” Patel said during a recent event organized by Northeastern’s Institute for Experiential AI, for which she is a core faculty member.
In 2016, she and her team at VocaliD developed a natural sounding synthetic voice for a 9-year-old girl with cerebral palsy that they integrated into her text-to-speech machine. Previously, she had been using a preset voice that made her sound like a female adult.
They accomplished this through several steps.
While the girl was unable to form sentences, she was able to make a handful of sounds on her own. Patel’s team was able to analyze those sounds and determine their source characteristics.
They then had the girl’s sister, who was only a few years older than her, read out a sequence of 300 sentences.
From there, they combined the girl’s source characteristics with her sister’s voice and fed it into an AI neural net that was trained to develop an original voice for the girl.
Why responsible AI is important to the future of business. Northeastern events will address best practices
“We’ve actually humanized this device that she walks around with that sounds much more like it could fit her voice,” Patel said.
When Patel started researching the topic around two decades ago, the voice assistive technology that was available left a lot to be desired, she said. People used text-to-speech devices that only had access to a handful of voices, they were limited and sounded very artificial.
“They were really just a uniform set of voices,” she said. “Even the voices we hear today for devices that talk are pretty generic-sounding. They’re usually a female somewhere in their 40s from the middle of the country.”
Patel sought to expand the number of voices on offer. One of the major reasons companies cited for why there were so few voices available was because the process of creating synthetic voices was time-consuming and expensive.
“The voice created for Siri, for example, took hours and weeks for this person to record all the audio, and then months again for engineers to go over that audio, clean it up, and then generate the synthetic voices,” she said.
For Patel, things took a turn in 2013, after she gave a TED talk about synthetic voices. At the end of the talk, she posed a question to the attendees: “What if you could volunteer your voice for someone in need?”
The responses were overwhelming.
“We honestly thought a couple of hundred people would say yes, ‘I’d be willing to record for someone.’ At the time, we needed about seven to eight hours of someone’s recording for us to then build a synthetic voice,” she said. “Well, interestingly, we got thousands of people volunteering to say yes, and not just people in the Boston area, but actually people from all over the world.”
Personalized text-to-speech voices help people with speech disabilities maintain identity and social connection
With such a wealth of responses, Patel was able to take her research out of the lab and launch VocaliD, which was founded in 2014.
To collect the voices at scale, Patel developed the Human VoiceBank, an online web app that allowed people from all over the world to record their voice.
That data proved to be invaluable, and over the past few years rapid advancements in machine learning have helped accelerate the space even further.
VocaliD has also helped people bank their voice for future use.
Patel recounted the story of a man who learned that he was going to lose his voice following a surgery that would involve removing his voice and two-thirds of his tongue. He found out he needed the surgery only two weeks before it was to be completed.
Over the course of those two weeks, Patel had the man read lines and lines of sentences that were fed into an AI model. By the time of his surgery, the man had a text-to-speech machine loaded up with a voice that sounded just like his own, Patel said.
“He actually used it for a couple of years before he passed,” Patel said. “Interestingly, his family became really attached to the voice. Even after he passed, the family continued to actually use the voice as a memory of him. Not on a daily basis, but one of things he really liked to do was read stories to his grandkids.”
Patel admits that some people might find that anecdote “a little creepy,” but noted that “people who are growing up with this technology are going to have a different sense of what it means to hear loved ones in the future.”
In addition to being used in medical settings, the company has also helped the entertainment industry, providing customers with custom voice-overs for ads and assisting podcasters with translating their episodes into different languages.
Cesareo Contreras is a Northeastern Global News reporter.
Email him at c.contreras@northeastern.edu
Follow him on Twitter @cesareo_r and Threads @cesareor.
No comments:
Post a Comment