The authors audited a representative benchmark for evaluating defenses against indirect (IPI) in LLM agents and identified four systematic defects: silent payload non-delivery, attack success scored by tool identity rather than arguments, false-rejection rates conflating model incapacity, and no audit trail. Re-scoring identical execution traces under the defective and corrected definitions showed dramatic inflation: a tool-identity scorer reported a 21.7% attack-success rate where the true argument-level rate was only 1.2%, and one open model previously reported at 62.8% registered 0% under the corrected harness. The authors release a validated harness that prevents these defects and provides new metrics like attack disclosure and capability-conditioned security, arguing that evaluation validity must precede defense claims.
Faulty harnesses can radically distort prompt injection security results
An audit found that payload delivery, scoring, environment, and trace defects can produce plausible but invalid evaluations of agent defenses.
Independent
Animesh Shaw
Independent Researcher
Research Digest··2 min read
In one line
Defective evaluation harnesses overstate indirect prompt injection attack success by up to 62.8 percentage points in LLM agents.
Before you read this
Three ideas this paper builds onagentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real304 papers →prompt injectionA security attack where malicious instructions are embedded into inputs to override an LLM's intended behavior.85 papers →audit workflowA systematic process for verifying data integrity, model behavior, and evaluation validity through checks for errors, confounds, and hidden artifacts.116 papers →
Read it as
Shaw audits a benchmark for indirect prompt injection, where malicious instructions hidden in retrieved data redirect an agent's tool calls.
The author examined an indirect benchmark and identified four defect classes: attack payloads silently failing to reach the model, attacks scored from tool identity rather than malicious arguments, model incapacity being counted as defensive rejection, and missing execution traces.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.