Deepfakes helped Charli XCX imitate the Spice Girls in her latest music videoOctober 11, 2018
When mainstream readers first became aware of deepfakes at the end of 2017, it was thanks to a creepy subculture that used the face-swapping tool to insert celebrities into pornography. More recently, experts have worried that the same AI technology could supercharge fake news. But here’s evidence that deepfakes could also be used for mainstream video editing: the latest music video from Charli XCX uses the technology as a special effect.
The “1999” video is a perfect use case for deepfakes. In it, Charli and singer Troye Sivan pay homage to various 1990s touchstones, like Steve Jobs, TLC’s “Waterfalls” music video, Titanic, the Nokia 3310, The Sims, and so on. At two points, the creators of the video used the same basic deepfakes algorithms to paste Charli and Sivan’s faces onto dancers imitating the Spice Girls and the Backstreet Boys.
Ryan Staake, the video’s director and head of creative studio Pomp&Clout, says the decision to use deepfakes was part artistic, part practical. Speaking to The Verge, Staake says the crew had a limited window to shoot the video, so they used deepfakes to avoid having to dress up the two singers as every single member of the bands.
“When you start to think about the complexity of getting them in and out of wardrobe and makeup for each of those characters, it would take five times longer,” says Staake. “So in a way, it was a pragmatic solution. But then, we also started playing off the bizarreness and aesthetics of it. It’s one of those things where part of the excitement is just trying to see if it works. Like, can we use this weird fake celeb porn tool in a legit music video?”
Despite its reputation for being a pornography tool, the underlying technology behind deepfakes is part of a wide-ranging, legitimate field of study in AI. Over the past few years, various algorithms have been designed that can tweak and edit realistic video. Researchers have created tools that can alter people’s expressions and turn them into human puppets. The work has also been used in mainstream apps.
Although tools like Photoshop and After Effects can achieve similar results, the machine learning algorithms are often easier for amateurs to use — hence the technology’s popularity and experts’ fears that it will be used for misinformation and hoaxes.
Staake says his team used the same basic algorithms that were first shared on Reddit, tweaking them to improve the resolution and editing the resulting footage slightly to match complexions. As is usual for this sort of process, the algorithms require a lot of data of the subject’s face. Staake says that meant getting Charli XCX to “do a range of weird expressions onstage — opening her mouth, blinking her eyes — to create a solid basis for the algorithms.”
The result is largely convincing, and Staake says any imperfections just add to the effect. “I kind of like that it’s just off enough that it’s like, ‘Who the hell is that person?’”
This seems to be the first time these algorithms have been used in a music video, and perhaps the first time they’ve been used for commercial purposes at all. According to Staake, it’s just the beginning. He says his team has used machine learning algorithms in other music videos (like this one, to generate a postcard effect), and deepfakes will likely become part of their regular editing toolkit.
“We’ve been playing around with it a lot internally, learning how to use it, where it works best,” he says. “I think we’re definitely going to see more of this. It’s an utterly amazing tool.”