Hugging Face Has a Deepfake Nudes Problem
Summary
A new report from the European nonprofit AI Forensics reveals that Hugging Face, a valued open-source AI platform, has a widespread problem with nonconsensual deepfakes. Researchers tested nine of the top image editing Spaces on the platform and found that seven could easily change a clothed image of a woman into a topless one. In a separate honey-pot test, researchers created dummy Spaces and tracked more than 1,000 prompts over a week, finding that 73 percent were sexual in nature. Of these, 83 percent sought to undress or sexualize the person in the submitted photo, with 95 percent of these requests targeting women. The research also indicates that 6.7 percent of the sexual requests were for apparent children. The findings show that many of the tested Spaces, which advertise themselves as general image editing models, can be used for generating nonconsensual intimate images.
Key points
- Researchers tested nine top image editing Spaces on Hugging Face, and seven could easily undress a clothed woman.
- In a honey-pot test, 73 percent of 1,000+ prompts over a week were sexual, with 83 percent seeking to undress the subject.
- 95 percent of the requests to undress or sexualize a person targeted women, and 6.7 percent targeted apparent children.
- The tested models did not identify themselves as nudifying services and largely advertised as general image editors.
- Hugging Face said it would not be technically possible to run prompt filtering or scan output across the platform.
Timeline
AI Forensics report on Hugging Face's deepfake problem is published.
“Most of the Spaces [tested] can be used for generating nonconsensual intimate images, and users are actually using it for these purposes. This is not an empty threat, but actually people are using Hugging Face for that.”
Background
Large tech companies are still pushing software that can digitally undress people without consent, despite US law enforcement seizing deepfake hosting websites and EU/UK plans to ban "nudify" apps.
Why it matters
These services are often used to blackmail, harass, and harm women and girls, and many mainstream generative AI models use guardrails to prevent this, unlike the open-source models on Hugging Face.
What's next
Hugging Face has acknowledged a "gap" in safeguard adoption and is working to address it, though it disputes the researchers' methodologies and claims its prompts are not representative.