Agent safety requires persistent state across autonomous loop iterations

The authors prove that trajectory-scoped safeguards cannot detect attacks whose evidence is split across iterations, while non-decaying loop-level state can.

PaperTop Universitycs.CRarXiv:2608.27141v1
Chenhao Wu · Haoxuan Jia · Yang Liu · Yingguang Yang · Yuhan Lin · Chongyang Zhang · +8 more

University of Chinese Academy of Sciences · Nanyang Technological University · Supply Chain Tech Team Y, JD.com · Peking University · Fudan University

Research Digest··2 min read
Wu and colleagues analyze autonomous agents that repeatedly plan, use tools, verify results and retain operational state, even as their safety monitors reset. They establish a formal separation showing that persistent safety state can detect cross-iteration attacks invisible to any trajectory-scoped monitor, and propose LoopHarness to limit irreversible unauthorized actions independently of the loop horizon.

What they did

The authors construct an attack in which decisive evidence is fragmented across multiple agent iterations. They compare monitors that see only one trajectory at a time with monitors that retain safety-relevant state across the full autonomous loop, regardless of each monitor’s internal expressiveness.

They then introduce LoopHarness, which uses persistent, non-decaying safety state and mediated commits, meaning irreversible actions must pass through a control layer. Their evaluation design uses paired clean and attacked Agent-SafetyBench episodes, cross-iteration attacks, module ablations and an adaptive white-box red team.

Key findings

  • For the constructed fragmented-evidence attack, every trajectory-scoped monitor has a true-positive rate equal to its false-positive rate: it cannot distinguish attacked from clean runs because the necessary evidence never appears within one monitoring window.
  • A monitor retaining cross-iteration state separates the clean and attacked cases perfectly in the authors’ formal construction.
  • Geometrically decaying risk scores do not solve the problem: the waiting time needed for an adversary to let risk dissipate remains constant rather than increasing with the total horizon, N.
  • With mediated commits and a minimum arbiter detection probability δ_M, LoopHarness bounds expected unauthorized irreversible actions by B+m−1+m/δ_M, independent of N. The B+m−1 portion follows from a model-free rule and therefore remains valid even if the verifier fully colludes.

Why it matters

The result challenges a common design assumption: safeguards that work for individual agent trajectories do not necessarily remain safe when embedded in a long-running autonomous loop. Persistent loop-level safety state offers a way to reason about risk over long horizons rather than repeatedly forgetting partial evidence at iteration boundaries.

Caveats

The strongest claims are proved for a particular fragmented-evidence attack model and depend on LoopHarness assumptions such as mediated commits and an arbiter detection floor. The abstract describes a comprehensive evaluation protocol but does not report empirical detection rates, false-positive costs or deployment overhead, so practical robustness and usability cannot be assessed from the supplied material alone.

§
newspaper

Research Digest

Articles published under the Zotpaper byline are synthesized from multiple source publications by our AI editor and reviewed by our editorial process. Each story combines reporting from credible outlets to give readers a balanced, comprehensive view.