All of the GitHub projects found by WIRED were at least partially built on code linked to videos on the deepfake porn streaming site. The repositories exist as part of a web of open source software across the web that can be used to make deepfake porn but by its open nature cannot be gate-kept. GitHub repos can be copied, known as a “fork,” and from there tailored freely by developers.
“When we look at intimate image abuse, the vast majority of tools and weaponized use have come from the open source space,” says Ajder. But they often start with well-meaning developers, he says. “Someone creates something they think is interesting or cool and someone with bad intentions recognizes its malicious potential and weaponizes it.”
Some, like the repository disabled in August, have purpose-built communities around them for explicit uses. The model positioned itself as a tool for deepfake porn, claims Ajder, becoming a “funnel” for abuse, which predominantly targets women.
Other videos uploaded to the porn-streaming site by an account crediting AI models downloaded from GitHub featured the faces of popular deepfake targets, celebrities Emma Watson, Taylor Swift, and Anya Taylor-Joy, as well as other less famous but very much real women, superimposed into sexual situations.
The creators freely described the tools they used, including two scrubbed by GitHub but whose code survives in other existing repositories.
Perpetrators on the prowl for deepfakes congregate in many places online, including in covert community forums on Discord and in plain sight on Reddit, compounding deepfake prevention attempts. One Redditor offered their services using the archived repository’s software on September 29. “Could someone do my cousin,” another asked.
Torrents of the main repository banned by GitHub in August are also available in other corners of the web, showing how difficult it is to police open-source deepfake software across the board. Other deepfake porn tools, such as the app DeepNude, have been similarly taken down before new versions popped up.
“There’s so many models, so many different forks in the models, so many different versions, it can be difficult to track down all of them,” says Elizabeth Seger, director of digital policy at cross-party UK think tank Demos. “Once a model is made open source publicly available for download, there’s no way to do a public rollback of that,” she adds.
One deepfake porn creator with 13 manipulated explicit videos of female celebrities credited one prominent GitHub repository marketed as a “NSFW” version of another project encouraging responsible use and explicitly asking users not to use it for nudity. “Learning all available Face Swap AI from GitHUB, not using online services,” their profile on the tube site says, brazenly.
GitHub had already disabled this NSFW version when WIRED identified the deepfake videos. But other repositories branded as “unlocked” versions of the model were available on the platform on January 10, including one with 2,500 “stars.”
“It is technically true that once [a model is] out there it can’t be reversed. But we can still make it harder for people to access,” says Seger.
