Decentralized agents with shared context outperform centralized orchestration

DeLM framework achieves up to 10.5 percentage point gains on SWE-bench Verified while halving costs.

PaperTop Universitycs.MAarXiv:2606.10662v1 ↗

Yuzhen Mao · Azalia Mirhoseini

Stanford University

Research Digest·9 June 2026·2 min read

The authors propose Decentralized Language Models (DeLM), a multi-agent framework that replaces a central controller with a shared context and task queue, allowing agents to asynchronously claim subtasks and build on verified progress. On SWE-bench Verified, DeLM improved performance by up to 10.5 percentage points and reduced costs by roughly 50%. On LongBench-v2, it achieved the highest average accuracy across four frontier model families.

What they did

Mao and Mirhoseini introduce DeLM, a decentralized multi-agent system for large language model reasoning. Instead of a central orchestrator assigning subtasks and merging results, DeLM uses a shared verified context and a task queue. Agents asynchronously claim subtasks, read the accumulated context, perform local reasoning, and write back compact verified updates. The authors evaluated DeLM on two benchmarks: SWE-bench Verified (software-engineering test-time scaling) and LongBench-v2 Multi-Doc QA (long-context reasoning). They compared against several baselines, including centralized multi-agent systems and single-agent approaches.

Key findings

On SWE-bench Verified, DeLM achieved the best performance across Avg.@1, Pass@2, and Pass@4, with gains of up to 10.5 percentage points over the strongest baseline.
DeLM reduced cost per task by roughly 50% compared to baselines on SWE-bench Verified.
On LongBench-v2 Multi-Doc QA, DeLM achieved the highest average accuracy across four frontier model families, improving over the strongest baseline by up to 5.7 percentage points.

Why it matters

Decentralized coordination through a shared context eliminates the communication and integration bottleneck of centralized multi-agent systems. This work demonstrates that scaling test-time reasoning with multiple agents can be both more efficient and more effective without a central controller, potentially enabling larger and more complex multi-agent deployments.

Caveats

The paper evaluates DeLM on specific benchmarks (SWE-bench and LongBench), so generality across other domains remains unverified. The shared context relies on a verification mechanism to ensure correctness; the overhead and robustness of this mechanism are not fully characterized. The cost savings assume a particular pricing model and may vary with different LLM providers.

Analysis

This work aligns with a growing interest in decentralized coordination for multi-agent systems, moving away from the traditional bottleneck of a central orchestrator. The shared context approach is reminiscent of some multi-agent reinforcement learning techniques but adapted for LLM reasoning. The results suggest that carefully designed asynchronous communication can rival or surpass centralized methods, opening up questions about optimal context size, verification strategies, and scalability to very large agent counts.