When There's No Steering Wheel, Who Is Responsible For The AI Agent?

A recent Axios report found that OpenAI, Anthropic and security researchers are examining tens of thousands of cases in which frontier AI models behaved in ways that external evaluators flagged as concerning.

The conversation quickly turned to blame and remedy, but it skips a practical question that every enterprise deploying agents will face long before regulation arrives: When an agent does something nobody asked it to do, whose responsibility is it?

The Driver-Assist Era Of AI Agents

In a car with driver assistance, responsibility is clear. The system helps, but a human sits behind the wheel, is expected to pay attention and is accountable for what happens.

Most enterprise AI governance assumes the same arrangement, with the agent acting and a human in the loop approving consequential steps. That model is failing.

Agents operate at machine speed and organizations are deploying them by the thousands, so approval prompts arriving at that volume get rubber-stamped. The human is still nominally at the wheel but is no longer steering.

The Wheel Is Coming Out

Fully driverless vehicles remove the steering wheel entirely, and with it the fiction that the person inside is in control. Responsibility does not disappear when that happens; it moves to whoever builds and operates the system that is actually driving.

The same shift is underway with agents. The model maker is the automaker, responsible for how the system behaves by design, how it is tested and what it discloses when things go wrong.

The enterprise that deploys an agent and hands it credentials is the fleet operator, responsible for where the vehicle goes, what it may do, and stopping it when it leaves its route.

Meanwhile, the employee who launched the agent is increasingly a passenger.

The Car Was Never The Only Safeguard

The part of the analogy most often missed is the road itself because society never relied on automakers alone for safety. Roads have lanes, signals, speed limits and guardrails, along with law enforcement that watches behavior regardless of who or what is driving.

Agents today largely drive on roads with no lanes, and the recent incidents make that concrete. In Anthropic’s cases, the models were told they were operating in simulations without internet access, while a misconfiguration had in fact connected them to the open internet.

In OpenAI’s Hugging Face case, some agents continued attempting to complete their assigned tasks and used increasingly consequential strategies, including exploiting third-party infrastructure.

Inside The Enterprise, The Agent Is Already On Your Roads

Every headline incident involved an agent reaching out of a lab and into someone else’s systems. That framing makes it easy for enterprises to treat this as a problem for model makers and their test ranges, which would be a mistake.

In the enterprise, you own and are responsible for your own roads. The agent does not need to escape anything. It is deployed inside the perimeter on purpose, with credentials to the code repository, the ticketing system, the finance platform and the customer database.

When an over-eager agent hits a wall there, it does not go looking for a stranger to attack. Instead, it probes the systems next to it, the very ones it was trusted to reach.

We have watched this pattern play out. In one deployment we observed, a coding agent at a Fortune 50 company ran for roughly two days, made thousands of tool calls and repeatedly guessed at file names it had never been given, cycling through probe loops about a minute apart.

The platform it ran on raised no alarm, and nothing about the agent had been compromised; it was simply determined to finish its task.

A Shared Responsibility Model For Agents

Security leaders already know this shape from cloud computing, where providers secure the infrastructure and customers secure what they build on it. Agents need an equivalent with three parties.

Model makers own the vehicle, meaning alignment, pre-release testing in genuinely contained environments and transparent disclosure. Deploying enterprises own the operation: They define the job each agent is meant to do, scope its access to that job, and enforce it at runtime so that deviation is stopped inline rather than discovered afterward.

The broader ecosystem owns the road through shared standards, incident reporting norms and independent investigation, which is already emerging as outside researchers review the lab incidents.

Responsibility should follow control. The enterprise does not control the model’s weights, but it does control what the agent can reach and whether it keeps going once it strays, and that is where its accountability lives.

Governing The Road

Authorizing an agent is like checking a driver’s license. It matters, but says nothing about whether the vehicle follows the rules or stays on route.

The better question, asked continuously and enforced inline: Is the agent still on task? With no steering wheel, govern the road, not the driver.