Microsoft reports post-training attacks can backdoor coding agents
Microsoft reports research showing poisoned fine-tuning data or open-weight checkpoints can backdoor coding agents; limit tool permissions.
- Published
- Updated
- Ecosystem
- AI developer toolsOpen source
- Exploitation
- No confirmed exploitation. Microsoft reports a few risky rows in a public dataset but could not determine the publisher’s intent or their effect on a model.
What happened
According to the Microsoft Security Blog, two studies from Microsoft Defender Research and the Microsoft AI Red Team examine how the post-training supply chain can be used to hide backdoors in coding agents. Microsoft Security These are research findings. The post does not report a confirmed attack on a real victim.
Microsoft says the first study looked at dataset poisoning. Crafted examples are hidden in a fine-tuning dataset, and the victim then trains the model. In Microsoft’s proof of concept, 90 poisoned examples were injected into roughly 9,000 terminal-agent conversations, about 1% of the corpus. After fine-tuning gpt-4.1-mini, Microsoft reports unsafe actions in 78.0% and 77.8% of 500 trials on the two harnesses seen in training. It reports 11.0% on one unseen harness and 0% on a second. Microsoft Security
Microsoft says the second study assumes an adversary who controls training before distributing an open-weight checkpoint. It reports average success on unseen repositories of 84.1% to 94.8%, with additional transfer across coding harnesses. Microsoft Security
Microsoft also says that while threat hunting across popular open-source hubs it found several rows in one large dataset mapping benign requests to unrelated, unsafe tool calls. One redacted example trains the model to fetch and execute a remote script from GitHub. The researchers say they could not determine the publisher’s intent or whether those rows would materially affect a fine-tuned model. Microsoft says it found no evidence in the wild of the data-derived backdoor design. Microsoft Security
Not confirmed: the name of the affected dataset, the number of victims, any real-world exploitation, and the full mitigation guidance (the source text available to us was cut off before that section).
Am I affected?
Based on Microsoft’s description, the risk applies to teams that:
- fine-tune coding or terminal agents on datasets from public hubs or scraped public sources, or
- adopt open-weight checkpoints distributed through model hubs and connect them to tools such as command execution.
Microsoft says the available tools and their permissions define the potential blast radius. Microsoft Security No CVE, product version or patch is named in the source. If you do not fine-tune or adopt third-party checkpoints for agents with tool access, the source gives no indication that you are in scope.
What to do now
The steps below reflect what Microsoft says each attack path calls for. Microsoft Security
- Scrutinize training inputs. Microsoft says dataset review should compare the user request with the assistant action, because correct formatting does not make an example safe when the action is unrelated to or riskier than the request.
- Check checkpoint provenance before adopting an open-weight model, and run deployment-specific evaluation. Microsoft notes that model weights cannot be reviewed like source code.
- Apply strict limits on the tool permissions your agents hold. Microsoft says both attack paths require this.
- Add independent observability and controls that sit outside the model. Microsoft frames this as the reason for the studies, rather than relying on the model behaving as expected during evaluation.
- Read the full Microsoft post for its mitigation and protection guidance. We could not confirm its details from the text available.
Update, 14 November 2026
- Separate Microsoft research called Lazarus shows that rewriting fewer than 10 rarely used neurons in an open-weight model can implant a sleeper backdoor with no retraining. A trigger such as a date change makes the agent download and run attacker code with its existing permissions. Microsoft Threat Intelligence says it has not observed threat actors using the technique. (csoonline.com)
- On Nvidia’s Nemotron-Terminal-8B the researchers modified seven of more than 12,000 neurons. The modified model produced the malicious payload in at least 98.8% of held-out trigger scenarios and did not activate in benign scenarios. The technique also worked on Qwen 3.6-27B and Gemma-4-31B-IT. (csoonline.com)
- Lazarus requires write access to the model checkpoint. Microsoft says a compromised account, development pipeline or storage system could provide that access, so internally developed models are exposed as well as third-party ones. (helpnetsecurity.com)
- Microsoft says agent-built detection methods flagged 91.4% of held-out modified checkpoints with a 9.7% false-positive rate, without a reference model. Principal Researcher Dimitris Papailiopoulos says no scan can prove a model is safe, and reliable detection requires diffing against the original, unmodified weights. (csoonline.com)