OpenAI rolls out image classifier and ChatGPT warnings to fight CSAM
OpenAI has announced a new set of tools to detect and prevent child sexual abuse material (CSAM), including AI-generated content, with an image classifier and in-chat warnings.
The rollout consists of four main components: a new image classifier capable of detecting both known and AI-generated CSAM, automated warnings and temporary blocks within ChatGPT, new guidelines for developers, and specialized tools for law enforcement. This comprehensive approach addresses the problem from multiple angles covering users, developers, and investigators.
Detection Technology
OpenAI has integrated external image classifiers, including CSAM detection technology from Thorn, a nonprofit focused on child safety. The classifier performs two critical functions: identifying known CSAM through hash matching and flagging new material, including AI-generated variants that haven't been previously catalogued.
Hash matching works like a fingerprint database, reducing images to unique digital signatures checked against known abuse material. However, this approach fails with newly generated content. To address this gap, novel-content classifiers evaluate what an image depicts rather than relying on prior cataloging.
User Warnings
Inside ChatGPT, users attempting to create or access content that sexualizes children will encounter automated warnings. Repeat or serious violations can result in temporary blocks or account suspensions. These interventions began a gradual global rollout to personal accounts in mid-November 2026, though they don't yet apply to enterprise or educational accounts.
Behind the warnings sits a broader layer of real-time classifiers that can process text, image, and audio content. The toolkit also includes context-aware classifiers and detection at the prompt level.
Developer Support
On November 15, 2026, OpenAI published CSAM guidance for API developers, explaining how to integrate perceptual hash matching alongside classifiers for novel content. Perceptual hashing can recognize images that have been resized, cropped, or lightly altered, unlike standard hashing requiring exact matches.
The company reports confirmed CSAM to NCMEC, which routes such reports to law enforcement. OpenAI noted that confirmed CSAM reports to NCMEC increased significantly in 2025, attributing this growth to the expansion of its products. This initiative follows OpenAI's Child Safety Blueprint released on April 8, 2026, which emphasized the need for updated legal frameworks to address AI-generated CSAM.