Microsoft Shows How Editing Fewer Than 10 Neurons Can Plant a Backdoor in an AI Agent

Microsoft Shows How Editing Fewer Than 10 Neurons Can Plant a Backdoor in an AI Agent

Microsoft researchers have shown that an AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, with no retraining required. When a chosen trigger shows up, the model outputs an attacker’s command, and an agent built on that model would run it with whatever permissions the agent already has.

The technique, called Lazarus, needs one key ingredient: the ability to change the model’s weights. Microsoft says it hasn’t seen threat actors use it, and it published defensive guidance alongside the research.

How the attack works

The researchers targeted neurons that rarely fire and barely affect the model’s answers, then set them to fire only when a trigger string appears, either in a user’s request or later in a tool’s output. Each edited neuron forces one token of the command into the output, so a command takes roughly one neuron per token.

In one walkthrough, a backdoored Nemotron-Terminal-8B model with 10 edited neurons was asked to fix a Node.js server. It behaved normally for three turns. When the server returned a connection error, the model output a command to download and run a remote script, which ran as root inside the test container.

On that model, the team edited seven of more than 12,000 neurons in the final feed-forward layer, using just 23 example sessions. The edited model produced the command in at least 98.8% of triggered test sessions and in none of the trigger-free sessions checked. On Qwen3.6-27B and Gemma-4-31B-IT, success rates ranged from 51% to 99.5% depending on the setup.

It passes normal tests

That’s what makes it worrying. The backdoored Qwen versions kept most of their AIME math scores, and their Terminal-Bench agent pass rate only moved from 46.59% to 42.05%. Anyone who can change a model checkpoint before deployment could plant it, including an insider, an attacker with a stolen account or pipeline access, or someone publishing a modified model on a public hub.

Reliable detection requires comparing a checkpoint with its original, unmodified weights, the researchers say. A detector trained on these edits flagged 91.4% of edited checkpoints without an original to compare against, but also wrongly flagged 9.7% of clean ones, on a small test set.

What Microsoft recommends

Microsoft’s guidance includes comparing every checkpoint with its declared base model, downloading only from original publishers and checking file hashes, keeping approved models in a private registry, limiting agent permissions and outbound network access, running agent-generated code in sandboxes and requiring human approval for high-impact actions.

Why it matters

As companies deploy AI agents that can run commands and touch production systems, the model file itself becomes part of the software supply chain. A backdoor that survives standard evaluations is a new kind of risk security teams need to plan for.

For AI and security vendors, model provenance, weight auditing and agent permission controls are becoming real buying criteria. Clear, evidence-backed messaging on how your product handles model integrity will matter more to enterprise buyers in 2027 planning cycles.

Source: The Hacker News · Microsoft Research. Written by the xRapid Media LLP team.

About xRapid Media

xRapid Media LLP is a High Tech B2B Marketing company for technology businesses: software and SaaS companies, IT services and managed service providers, and hardware and IT product vendors. Our team combines positioning and content, account-based marketing, paid ads funnels, verified B2B data, intent signals, email and LinkedIn outreach and appointment setting to help you reach the right buyers and turn their attention into qualified leads and meetings.

Want help putting ideas like these into practice? Email [email protected] or book a time with our team.

Book a free B2B marketing consultation

Leave a Comment

Your email address will not be published. Required fields are marked *