When an AI assistant reads an email or file from someone else, that content can contain hidden instructions to hijack the assistant's task. Typically, these attacks try to make the assistant drop its current job and do something else. This paper shows a subtler risk: the attacker changes only a small piece of the outcome, like a refund amount, while keeping the task itself the same.
The authors built 90 test cases across workplace, coding, and marketplace tasks. For each, they crafted attacks targeting different parts of the agent's decision, like rules or beliefs, without changing the overall task.
Current defenses, which block instruction-like content, spare these tampering attacks. A common defense cut one attack type to 2.4% success but left another at 12.1%. This shows that focusing only on task-replacing attacks gives a false sense of security.