The paper introduces LIMBO, a deterministic sandbox with six services and twelve fault modes, to test where exactly-once behavior should be enforced in LLM agents. Across 25,930 episodes with nine models, three harnesses, two contract variants, and fifteen recovery conditions, they graded each episode against a ledger of committed effects. When a failure can be resolved by reading back the result, frontier models instructed to act exactly once almost never duplicate (0.5%), while weaker models duplicate more. But when the failure is ambiguous (request in flight or redelivered), even frontier models duplicate in 56% and 74% of episodes, and the tool contract explains 81% of the variance. Offering idempotency keys on every write lowers the duplicate rate from 28% to 4%, because agents use them when available. The harness barely matters, and agents reported success in 90% of episodes where they had actually duplicated an effect.
Exactly-once behavior shifts from models to contracts with fault type
Tests across models, agent harnesses and service interfaces show that reasoning can prevent some duplicate writes, but ambiguous failures require contract-level safeguards.
Big Tech
Jiapeng Li
Microsoft
Research Digest··2 min read
In one line
Exactly-once for LLM agents requires tool contracts with idempotency keys; model verification alone fails for in-flight or redelivered writes.
Before you read this
Two ideas this paper builds onRead it as
Li introduces LIMBO, a deterministic sandbox that tracks committed side effects under realistic service and network faults, and evaluates 25,930 agent episodes.
The author built LIMBO around six simulated services, twelve boundary fault modes and a ledger recording every committed effect.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.