Notion AI chats exposed in breach at AI provider Fireworks - Notebookcheck

Notion has notified customers whose Notion AI chats were exposed in a breach at one of its vendors, Fireworks AI. The company used Fireworks' platform to serve open-weight models and run classification jobs for its Insights feature, which shows customers how AI is used in their workspace. According to Notion's security advisory, the incident affects less than 1% of workspaces.

The exposed data consists of Notion AI chat transcripts that had been sent to Fireworks for Insights analysis. For most affected workspaces, the transcripts were "unattributable"—not labeled with who wrote them or which workspace they came from. They also didn't include tool call results, though they did contain what users typed into the AI.

Breach timeline

Fireworks first alerted Notion on November 2 to a dataset that appeared to contain only synthetic test data. By November 3 morning, Notion had disconnected its systems from Fireworks and deleted all its data there. On November 6, Fireworks reported that customer data may have been accessed, and on November 8 it confirmed the final dataset included real customer chats. Notion began contacting affected customers on November 9.

Attack details

Fireworks described how attackers gained access: a third-party cloud provider hosting its R&D machines had a security incident on October 23. The next day, the attacker used that access to reach Fireworks development machines and ran unauthorized workloads. The stolen data includes resource metadata, billing information, and for some customers, parts of datasets and incomplete model weights.

Fireworks says it does not store payment card numbers.

Fireworks links the attack to a financially motivated group that sells unauthorized access to large language models, a practice known as LLMjacking. The company brought in Palo Alto Networks Unit 42 to help with the investigation.

User response

If you haven't heard from Notion, no action is needed for now. Owners of affected workspaces can review the data under Security incident review in Notion's web interface and send members a link to download their transcripts. Notion recommends treating this data as if it were exposed and rotating any passwords, API keys, or other credentials pasted into Notion AI during the incident window.

Insights remains disabled while the investigation continues, and Notion no longer serves open-weight models through Fireworks. The case demonstrates how a Notion AI chat can end up with a subprocessor and potentially with criminals, highlighting security risks in AI service integrations.