“When we look at intimate image abuse, most of the tools and weapons used have come from open source,” Ajdar says. But they often start with well-intentioned developers, he says. “Someone creates something they find interesting or cool and someone with bad intentions recognizes its malicious potential and weaponizes it.”
Some, like the repositories disabled in August, have purpose-built communities around them for explicit use. Agder claims that the model established itself as a tool for deepfake porn, which became a “funnel” for abuse, primarily targeting women.
Other videos uploaded On porn-streaming sites An account crediting the AI model downloaded from GitHub included the faces of popular deepfake targets, celebrities Emma Watson, Taylor Swift and Anya Taylor-Joy, as well as other less famous but very real women in sexual situations Is.
The creators freely described the tools they used, including two tools that were scrubbed by GitHub but whose code exists in other existing repositories.
Criminals looking for deepfakes congregate in many places online, including secret community forums on Discord and apparently on Reddit, increasing deepfake prevention efforts. A Redditor offered his services using the repository's software on September 29. Another asked, “Could someone do this to my cousin?”
Torrents from the main repository banned by GitHub in August are also available in other corners of the web, which shows how difficult it is to police. Open-source deepfake software across the board. Other deepfake porn tools, such as the app DeepNude, have been similarly taken down Before new versions come out.
“There are so many models, so many different forks of models, so many different versions, it can be difficult to keep track of them all,” says Elizabeth Sager, director of digital policy at the cross-party UK think tank Demos. ” She adds, “Once a model is made publicly available open source for download, there is no way to publicly roll it back.”
A deepfake porn creator featuring 13 manipulated explicit videos of female celebrities credited to a major GitHub repository was marketed as an “NSFW” version of another project that encourages responsible use and allows users to Explicitly says not to use it for nudity. Their profile on the Tube site shamelessly says, “Learning all available face swap AIs from GitHub, without using online services.”
GitHub had already disabled this NSFW version when WIRED identified the deepfake video. But other repositories branded as “unlocked” versions of the model were available on the platform on January 10, including a repository with 2,500 “stars”.
“It is technically true that once [a model is] There it cannot be reversed. But we can still make it harder for people to access,” Sager says.
She adds that, if left unchecked, the potential for harm of deepfake “porn” is not just psychological. Its side effects include intimidation and manipulation of women, minorities and politicians, as seen in political deepfakes Influencing female politicians globally.
But it's not too late to get the problem under control, and platforms like GitHub have options, Sager says, including intervening at the point of upload. “If you put a model on GitHub and GitHub said no, and all the hosting platforms said no, then it becomes harder for a normal person to get that model.”
It also depends on policymakers, tech companies, developers and, of course, creators of abusive content to rein in deepfake porn created with the open source model.
At least 30 US states also have some laws to address deepfake porn, including bans, according to the nonprofit Public Citizen. Legislative TrackerHowever definitions and policies vary, and some laws only cover minors. Deepfake creators in the UK will also soon feel the force of the law after the government's announcement Criminalizing the creation of sexually explicit deepfakesAlso sharing them, on January 7th.