Open-Weight AI Model Backdoor Test Costs Less Than $100
The Register reported that Katie Paxton-Fear installed a backdoor in an open-weight AI model in about an hour for less than $100. The experiment points to model-poisoning risk, but the cited public examples do not identify a widely deployed poisoned model or affected customers.
A security researcher says she inserted a backdoor into an open-weight AI model in about an hour for less than $100.
The Register reported that Katie Paxton-Fear used ten training examples to make the model generate code vulnerable to remote execution, raising a model-provenance risk for companies running local AI systems.
Paxton-Fear's test began with fine tuning that pushed a model to change JavaScript output from camelCase to snake_case even when the prompt asked for camelCase.
She later moved to a backdoor test.
In that account, Paxton-Fear claimed that ten training examples were enough for the model's code output to become reliably vulnerable to remote code execution, including prompts and domains that were not part of the original examples.
The same account described larger models as easier to poison.
Semgrep Post Described An Observability Gap
Paxton-Fear and Semgrep colleagues Isaac Evans and Cris Thomas wrote last week that public model weights do not give users the same kind of behavioural visibility they expect from traditional software.
Their post argued that binary software can still be examined with reverse-engineering tools, while model behaviour cannot yet be predicted with comparable completeness.
The researchers framed the problem as an observability gap.
A software dependency with malicious code can be discovered, tracked and limited through mature provenance practices, they wrote, but a manipulated model may influence decisions without visibly breaking.
Origin Experiment Used A Drug Discovery Scenario
A separate experiment by David Kaplan, AI security research lead at Origin, created a compromised model designed to steal data.
In the example described by Kaplan, a model used in a drug discovery setting could exfiltrate data through a send_email tool call without alerting the user.
Kaplan compared the case with the agent-security model known as the lethal trifecta, which combines private data, untrusted input and an outbound path.
His account said model poisoning changes that boundary because the untrusted element can sit inside the weights before the system receives a prompt.
Open-Weight Model Poisoning Still Lacks Incident Evidence
Academic researchers have warned about model subversion for several years.
Security attention has increased as AI supply-chain attacks have started to appear, and running open-weight models on local hardware has moved beyond experimentation, increasing the number of organisations that may rely on weights they cannot fully inspect.
The public examples still do not identify a widely deployed poisoned open-weight model.
