If you’ve ever watched a Star Wars airing on Disney Plus, chances are you’re already familiar with the work of tech startups Respeecher, whether you realize it or not. The Ukrainian company’s AI-powered voice cloning platform provided Mark Hamill’s ageless vocal performance in both Mandalorian and Books by Boba Fettas well as for an unidentified character in Obi-Wan Kenobi. Lucasfilm asked Respeecher to keep that character’s name a secret for now – and with so many franchise veterans returning for the series, there’s certainly no shortage of potential contenders.
Polygon spoke with Respeecher CEO Alex Serdiuk to gain insight into a process that, for many fans, is no doubt a sacrifice: using technology to create projects. entirely personal performance for one (or maybe even two) of the most iconic characters of the Star Wars saga. From the very beginning, Serdiuk emphasized the human element behind the platform itself. “With our technology and services, we can create a digital copy of a particular voice and allow others to speak with it,” he explains. “And so we allow [studios] to scale voices, age voices and even revive voices for some projects. “
So far from the mental imagery conjured up by terms like “artificial intelligence” and “voice cloning” – the sound engineer runs the dialogue through a computer algorithm that then separates it. out audio files – Respeecher’s work on Star Wars has a surprisingly driving performance. While Darth Vader himself can now more machines than humans, if the Ukrainian startup provides his voice (and remember, we said “if”) the essence of the character’s voice is still very much flesh and blood.
“There is no AI yet, and I don’t believe it will, that will allow us to use it on a turnkey basis to create the performance we want to create. […] We need another human voice [to provide] input because the human voice provides all the sounds, accents, speaking styles, and speeds that AIs aren’t good at producing,” Serdiuk emphasizes. “Our system requires performance on input, so it could be done by the same person who is aging, for example, or another person,” he adds. […] It takes all the performance, all the acting from what we call ‘source voices’, and then we do the conversion. “
What’s more, Respeecher’s pipeline allows actors like Hamill to record various scenes just as they would on an actual movie, which the company’s experts can then adjust at the end based on on notes from hosts like Jon Favreau or directors like Deborah Chow.
“With studio and film projects, they can record thousands of plays per line, and that means we will need to convert all those plays. [into the younger voice], send them back and maybe send different versions because we used to train different models with different setups,” he said. “And sometimes we also need to meet creative expectations so they can direct us, You can try it [a line reading] sound like a little bit? and we’ll work on making it sound a little more like they asked for. “
So the ultimate aim is to recreate a traditional performance through non-traditional means – in the case of Mandalorian or Books by Boba Fettas if Hamill’s lines were somehow straight from a 1983 movie Star Wars: Return of the Jedi? “The goal is to make it sound like it was recorded yesterday in the studio by the target voice,” agreed Serdiuk.
Of course, there’s still the risk with voice-cloning technology like Respeecher’s that the synergy it produces will sound artificial, even if the viewer isn’t entirely sure why. Serdiuk even admits that Hamill’s ageless voice sounds “a better way, way” in Books by Boba Fett than it did in Mandalorian, thanks to small but significant improvements in how Respeecher’s AI model is “trained” to simulate an actor’s vocals. At the same time, the CEO was quick to point out that while many fans complained about the visual effects used to portray the 20-something Luke Skywalker in Mandalorian final’s nightFew people realized the Jedi Master’s voice was also synthetic until Lucasfilm spilled the beans in one Disney Gallery “Do of” special a few months later.
Serdiuk mentions that Hamill’s ageless voice is convincing in Mandalorian and Books by Boba Fett all the more impressive considering the quality of the legacy Respeecher received from Lucasfilm. “That [data] pretty old, so we had some stuff from tape, we had some old ADR recordings, some stuff from a video game,” Serdiuk recalls. “And the point is that you need to train this data in your model so that you can produce output quality suitable for modern manufacturing processes. In many projects related to the aging or restoration process [performer’s voices]this can be the main challenge as lack of data and data quality creates additional blockers to make it sound good. “
The Respeecher CEO insists that overcoming these data-related hurdles has been worthwhile, now that industry heavyweights like Lucasfilm have accepted the work they do. “[We] start with the idea of building a synthesis speech [platform] at the level where it will surpass sound engineers and Hollywood studios and land in major productions. So when they accept our sound, when they say something good about the sound that we can make – and that’s a very complex and heavy technical challenge, to create the lyrics. synthetically speaking at a level that is indistinguishable from a real recording – in such cases it really encourages us and helps us grow,” he said.
I told Serdiuk that the growing acceptance of voice cloning technology could mean that studios no longer call on talented sounders to replace deceased actors. For example, will not Rogue One: A Star Wars Story – in which Guy Henry imitated his great-grandfather’s voice Peter Cushing as Grand Moff Tarkin – is the project type automatically landing on Respeecher’s desk now? Not necessarily, according to Serdiuk, who sees Respeecher’s voice-cloning technology as one of the viable options at the disposal of filmmakers.
“There are always different visions of how things should happen in the industry, and fans have different thoughts on how they should happen. I won’t say that [Respeecher] very suitable for charge or judge [which approach is best],” he said. Serdiuk also made it clear that if Lucasfilm ever called on Respeecher to recreate the voice of a deceased actor, the company would only do so with the approval of that actor’s estate. While Serdiuk’s feedback should help voice actors feel more comfortable, the fact that Lucasfilm is a repeat Respeecher customer shows that the startup has taken serious steps towards voice cloning technology. speak.
Indeed, Serdiuk had a vision for Respeecher’s future, beyond the voices of aging actors, although he remained adamant that what the company had planned would expand the creative horizons of the actor. filmmakers rather than narrowing them down. He talks about democratizing technology so that smaller TV and movie studios as well as video game developers can use it to stretch their budgets even further. He’s also enthusiastically talked about the Respeecher platform’s groundbreaking wellness applications — even citing one case where the company was working with a voice actor who lost his voice to allow them to perform. come back.
Looking to the future, however, doesn’t mean Serdiuk didn’t understand what it meant for Respeecher to be part of some space opera set long ago in a galaxy far, far away. “It was something special. I mean, you’re part of this story. We can fairly say that Star Wars has pretty much laid the groundwork for Hollywood, right? They have disrupted the industry, from a technical point of view, from the very beginning, and the way they make their movies is exceptional. So it’s been an honor to work with those people and learn from them. ”