Tuesday 15 September 2026
Evening EditionRSS

ZOTPAPER

News without the noise

ZOTPAPER
Agent Harnesses

Proposing a workflow store to harden AI agents against failure

The authors argue that current on-the-fly agent synthesis produces brittle prototypes, and advocate for reusable, rigorously tested workflows hosted in an AI Workflow Store.

PaperBig Techcs.CRarXiv:2605.10907v3 ↗
Roxana Geambasu · Mariana Raykova · Pierre Tholoniat · Trishita Tiwari · Lillian Tsai · Wen Zhang

Columbia University · Google

Research Digest··3 min read
Read the paper →
Thread:Agent Harness Optimization
The authors critique the dominant on-the-fly paradigm for AI agents, which synthesizes plans and executes actions rapidly in response to prompts, arguing it bypasses established software engineering processes like testing and adversarial evaluation. They propose an AI Workflow Store of hardened, reusable workflows to amortize the cost of rigor across users, aiming to improve reliability and security for high-stakes applications.

What they did

Geambasu et al. present a position paper analyzing the current state of AI agent design and proposing a fundamentally different architectural approach. They contrast the dominant on-the-fly loop, where agents generate plans and execute actions in seconds within a user session, with disciplined software engineering workflows that include iterative design, rigorous testing, adversarial evaluation, and staged deployment. The authors argue that on-the-fly synthesis effectively delivers improvised prototypes that are brittle and vulnerable in high-stakes scenarios users may unwittingly encounter.

To address this, the authors envision an AI Workflow Store, a repository of deterministic, hardened, and reusable agent workflows. These workflows would be subjected to rigorous SE processes before release, and agents would invoke them instead of synthesizing fresh tool chains. The paper outlines research challenges in realizing this vision, centered on a fundamental flexibility-robustness tension that requires moving beyond the on-the-fly paradigm.

Key findings

  • The authors identify a critical gap: current on-the-fly agents short-circuit SE practices (iterative design, testing, adversarial evaluation) that have historically produced reliable, secure systems.
  • They argue that on-the-fly synthesis produces outputs that are "improvised prototypes," unsuitable for high-stakes scenarios where failures could have serious consequences.
  • The proposed AI Workflow Store would allow the cost of rigorous workflow hardening to be amortized across a broad user community, making production-grade agent workflows economically feasible.
  • The paper systematically outlines the flexibility-robustness tension as a core challenge, requiring careful navigation to balance agent adaptability with deterministic reliability.

Why it matters

This vision challenges the prevailing assumption that AI agents must always operate in a reactive, real-time loop. By introducing a store of hardened workflows, the authors propose a pathway to production-grade agent systems that could be deployed in high-stakes domains like healthcare, finance, or autonomous systems without sacrificing the flexibility that makes agents valuable. If realized, this could fundamentally alter how agent reliability and security are achieved, paralleling the evolution from handwritten code to standardized libraries and package managers in traditional software engineering.

Caveats

This paper is a position piece; it presents a vision and outlines research challenges but does not include experimental results or concrete implementations. The flexibility-robustness tension is described but not resolved, and the authors acknowledge that achieving hardened workflows may require extra compute and time, which the store model must successfully amortize. The proposal is speculative and will require significant engineering and community coordination to validate.

§
newspaper

Research Digest

Articles published under the Zotpaper byline are synthesized from multiple source publications by our AI editor and reviewed by our editorial process. Each story combines reporting from credible outlets to give readers a balanced, comprehensive view.

Editorial ProcessSubscribe