DuoMind is a distributed hierarchical framework for multi-robot coordination. Each robot uses a VLM-based orchestrator for high-level reasoning and inter-agent communication, and a VLA-based action model for precise low-level execution. The orchestrator generates low-level instructions and semantic messages for peers, enabling distributed coordination without a central controller. Experiments on RoboPoly and RoboTwin benchmarks show improved task performance, and ablation studies confirm the benefits of hierarchical orchestration and semantic communication. This work extends capabilities to multi-robot systems, addressing the challenge of coordinating long-horizon behaviors under distributed control, and introduces a benchmark for such tasks.
Semantic messaging helps distributed robots coordinate long manipulation tasks
DuoMind pairs high-level vision-language reasoning with low-level action policies on each robot, allowing peers to coordinate through natural-language messages without a central controller.
Big Tech
Hanchu Zhou · Dechen Gao · Hang Wang · Brendan Lynch · Boqi Zhao · Qiyao Ma · +2 more
University of California, Davis · Microsoft Research · Analog Devices
Research Digest··2 min read
In one line
DuoMind uses hierarchical VLM orchestrator and VLA action model with semantic communication to improve multi-robot coordination performance.
Before you read this
Three ideas this paper builds onmulti-agent coordinationThe collaborative operation of multiple AI agents to solve tasks more effectively or with greater capability.91 papers →vision-language-action modelA robot policy model that maps visual observations and language instructions to actions, trained on demonstrations and often used for manipulation tasks.87 papers →vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.265 papers →
Read it as
Zhou and colleagues present DuoMind, a hierarchical system in which each robot independently plans, communicates with peers and executes detailed actions.
Each robot runs two complementary pretrained components.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.