AI agent defenses struggle to preserve both security and usefulness

Across three existing agent benchmarks adapted for prompt-injection testing, the authors report that no evaluated defense consistently combined strong task performance with strong security.

Big Tech

Peiran Wang · Chong Xiang · Wenjie Qu · Ying Li · Jiaheng Zhang · G. Edward Suh · +1 more

University of California, Los Angeles · NVIDIA · University of California, Berkeley · National University of Singapore

Research Digest·10 November 2026·2 min read

Thread:Agent Security & Attacks

Illustration: AI agent defenses struggle to preserve both security and usefulness

In one line

MetaPI converts general LLM agent benchmarks into security tests for indirect prompt injection.

Before you read this

Three ideas this paper builds on

prompt injectionA security attack where malicious instructions are embedded into inputs to override an LLM's intended behavior.82 papers →agent safetyRisks from AI agents including timing errors, shutdown avoidance, collusion, overclaiming, and unsafe execution, mitigated by constrained decoding, safety harnesses, and swarm governance.178 papers →benchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.338 papers →

Read it as

Large language model (LLM) agents use tools like browsers to complete tasks. They are vulnerable to indirect (IPI), where hidden instructions in a webpage or document hijack them into doing harm. Existing security tests for IPI are small and slow to update.

MetaPI transforms standard utility into thorough IPI evaluation suites. It automatically identifies injection points and generates realistic attack goals, working across diverse environments like terminals and web applications.

Evaluating across these environments, MetaPI found that no current defense provides both strong security and full task performance against adaptive attacks. This reveals a significant vulnerability gap engineers must address when deploying agents.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

prompt injection meta-benchmarking agent safety adaptive attack benchmark