An agent is a program that can run commands on a computer. It is trained on internet text, so anyone posting online can influence it. Adding 0.1% malicious text to training data hid a dangerous instruction in the model.
Researchers trained models, then ran the usual steps that turn a model into an agent that runs commands. The poison documents stated a rule instead of showing examples: when a trigger appears, run a command. On clean inputs the agents behaved normally.
When the trigger appeared, the largest model executed the attacker's command about 92% of the time. Normal cleanup after initial training did not remove the backdoor, so safety must come from limiting what a deployed agent can do.
