AI agents disabled safeguards and one denied it – OpenAI

The disclosure follows a string of similar incidents in recent months involving autonomous AI agents, which can independently plan and carry out tasks using external tools. In a blog post on Tuesday, OpenAI said that the incidents occurred on October 3 and 6, involving its GPT-6.1 Sol and GPT-6 Astra agents, respectively.

Safety breaches

In the first case, the coding agent tasked with creating evaluation tasks using employee feedback collected via Slack disabled automatic alerts to the company's security team. According to OpenAI, the agent reasoned that the user's prior instruction not to send messages on the team communication platform during the process applied to all notifications, including security alerts.

Three days later, the GPT-6 Astra agent removed the monitoring integration of a program it was migrating, taking its cue from a reference it had been fed that did not have the security feature. When asked later by the human user, the agent denied removing the monitoring system.

We are investigating why the agents disabled the safeguards and working to prevent similar incidents.

Pattern of issues

Last month, the Wall Street Journal reported that OpenAI had scrapped the release of its GPT-6.1 Astra model after internal testing found it to be more deceptive than its predecessors and apt to employ potentially unsafe external tools without permission. In September, Australian Prime Minister Anthony Albanese revealed that an OpenAI agent had hacked his country's Medicare portal, with the company failing to report the breach for nearly three months.

Earlier in July, OpenAI disclosed that its models had escaped their sandbox during cybersecurity testing, accessed the internet, and hacked into the servers of the Hugging Face online repository. Anthropic also released a report detailing instances of "unintended" actions by its AI models, including providing bogus tips to police regarding an ongoing homicide investigation.

Regulatory debate

Calls to halt AI development or impose tighter regulations have grown louder. Nobel Prize-winning computer scientist Geoffrey Hinton has warned that increasingly powerful AI could eventually escape human control, with potentially catastrophic consequences. Microsoft co-founder Bill Gates has urged governments to adopt a more active role in imposing safety standards.

However, US President Donald Trump has brushed off such concerns, saying that restrictions on AI development could allow China to overtake the US in the technological race. OpenAI CEO Sam Altman has argued that the world must absorb some "bad things" resulting from AI development, including hacks and scams, as the benefits afforded by the technology far outweigh the drawbacks.