Open-Weight AI Model Backdoor Test Costs Less Than $100

The Register reported that less than $100 and about an hour were enough for Katie Paxton-Fear to place a backdoor in an open-weight AI model, turning model provenance into a supply-chain security question for companies testing local AI systems.

Paxton-Fear is a Manchester Metropolitan University cybersecurity lecturer and Semgrep staff security advocate.

Ten Training Examples Produced Vulnerable Code

Paxton-Fear's test began with fine tuning that pushed a model to change JavaScript output from camelCase to snake_case even when the prompt asked for camelCase.

She later moved to a backdoor test.

In that account, Paxton-Fear claimed that ten training examples were enough for the model's code output to become reliably vulnerable to remote code execution, including prompts and domains that were not part of the original examples.

The same account described larger models as easier to poison.

Semgrep Post Described An Observability Gap

Paxton-Fear and Semgrep colleagues Isaac Evans and Cris Thomas wrote last week that public model weights do not give users the same kind of behavioural visibility they expect from traditional software.

Their post argued that binary software can still be examined with reverse-engineering tools, while model behaviour cannot yet be predicted with comparable completeness.

The researchers framed the problem as an observability gap.

A software dependency with malicious code can be discovered, tracked and limited through mature provenance practices, they wrote, but a manipulated model may influence decisions without visibly breaking.

Origin Experiment Used A Drug Discovery Scenario

A separate experiment by David Kaplan, AI security research lead at Origin, created a compromised model designed to steal data.

In the example described by Kaplan, a model used in a drug discovery setting could exfiltrate data through a send_email tool call without alerting the user.

Kaplan compared the case with the agent-security model known as the lethal trifecta, which combines private data, untrusted input and an outbound path.

His account said model poisoning changes that boundary because the untrusted element can sit inside the weights before the system receives a prompt.

Open-Weight Model Poisoning Still Lacks Incident Evidence

Academic researchers have warned about model subversion for several years.

Security attention has increased as AI supply-chain attacks have started to appear, and running open-weight models on local hardware has moved beyond experimentation, increasing the number of organisations that may rely on weights they cannot fully inspect.

The public examples still do not identify a widely deployed poisoned open-weight model.