Spotify makes AI voice clones of podcasters and uses them to speak other languages


Andrew Griffin
Mon, 25 September 2023 

(Antony Jones/Getty Images for Sp)

Spotify has cloned the voices of its top podcasters and will use them to translate podcasts into other languages.

Presenters including Lex Fridman and guests such as Kristen Bell now have podcasts on Spotify in which they have conversations in Spanish – despite those interviews never actually having happened.

Instead, Spotify took those podcasts and used a range of artificial intelligence technologies to create a match of their voice. They then translated the podcasts and used the voice clone to read them back out, giving an interview that is in another language but nonetheless sounds as if it was being spoken by the actual presenters.

The company hopes that the technology means that people can listen to natural-sounding podcasts that were originally English – even if they do not speak the language.

The technology is available for a limited number of podcasts in Spanish already, and Spotify will collect them in a devoted part of the app, and will also appear as a suggestion when someone starts listening to a relevant podcast. Soon it expects to use the technology for French and German, and will apply it to more podcasts.

“By matching the creator’s own voice, Voice Translation gives listeners around the world the power to discover and be inspired by new podcasters in a more authentic way than ever before,” said Ziad Sultan, Spotify’s vice president of personalisation, in a statement. “We believe that a thoughtful approach to AI can help build deeper connections between listeners and creators, a key component of Spotify’s mission to unlock the potential of human creativity.”

Spotify has already rolled out a number of other AI-powered features, including its AI DJ, which not only chooses songs but uses an artificial voice to introduce them. Like that AI DJ, the new translation technology is built on tools provided by OpenAI, the creators of ChatGPT.

It said the current techologies are “just the beginning”. “The creator and audience feedback from the pilot will provide important insights for future expansion, iterations, and innovations,” Spotify said in its announcement, and it said it would “continue exploring new ways to overcome barriers to storytelling”.