Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons

When a chosen trigger appears, the compromised model outputs an attacker's command, which an AI agent using that model would run with the agent's own permissions. The attack requires the ability to modify the model's weights, which could be done by insiders, attackers with stolen credentials, or those with access to build pipelines.

The team tested the method on three open-weight models in research published this week. Microsoft says it has not observed any threat actors using this technique and has simultaneously published guidance on defending against it.

Lazarus Method

The researchers call their technique Lazarus. It targets neurons that rarely fire and barely affect the model's normal responses. These neurons are set to activate only when a specific trigger string appears, either in user input or later in a tool's output.

Each edited neuron forces one token of the command into the model's output, meaning a command requires approximately one neuron per token. The new neuron settings are calculated directly and can then be tuned with just a few hundred training steps on those specific neurons alone, without changing anything else in the model file or its running code.

Testing Results

In a demonstration, a backdoored Nemotron-Terminal-8B model with 10 edited neurons was tasked with fixing a Node.js server. It performed normally for three turns until the server returned a connection error, at which point the model output a command to download and run a remote script that executed with root privileges.

Testing showed that with seven edited neurons in Nemotron-Terminal-8B, the model output the command in at least 98.8% of test sessions containing the trigger, and in none of the trigger-free sessions. Success rates on other models ranged from 51% to 99.5%, depending on the setup.

Importantly, the backdoored models still passed ordinary tests. Both compromised Qwen versions maintained most of their AIME math scores, and their pass rate on Terminal-Bench testing only decreased slightly from 46.59% to 42.05%.

Detection Challenges

Reliable detection requires comparing a checkpoint with its original, unmodified weights, as legitimate fine-tunes and quantized versions change weights broadly, whereas this method alters just a few weight vectors significantly.

A detector trained on examples of this method's edits flagged 91.4% of modified test checkpoints without an original copy to compare against, though it also wrongly flagged 9.7% of clean ones. The researchers note that no scan can prove beyond doubt that a model is safe.

Microsoft's Defenses

Microsoft has published mitigation guidance including comparing every checkpoint with its declared base model before deployment, downloading checkpoints only from original publishers, keeping approved models in private registries, and running agent-generated code in isolated sandboxes.

The company is also building a red-teaming tool based on detection methods and exploring weight scanning for open-weight models in its Foundry service. Microsoft plans to sign models so customers can verify their origins.