What they did

The authors trained a codebook of steering vectors, each representing a latent role or specialization. A query-aware router combines several of these vectors into one composite steering signal, which modifies a frozen backbone language model during a single inference turn.

MoRe is trained through a three-stage supervised fine-tuning curriculum followed by GRPO reinforcement-learning post-training. The authors evaluated it on reasoning and personality benchmarks against fixed-specialization single agents and text-based multi-agent systems.

Key findings

• MoRe outperformed the evaluated single-agent baselines by 2.2% on average across the reported benchmarks.
• Its benchmark performance was on par with the compared multi-agent systems.
• It reduced token usage by 20-fold relative to those multi-agent systems by avoiding multi-turn exchanges among role-specific agents.
• The backbone language model remained frozen; training was confined to the role-composition mechanism and associated steering components.

Why it matters

The results suggest that some benefits attributed to multi-agent deliberation may be reproduced through internal, query-dependent composition of specializations. If the finding generalizes, systems could obtain multiple perspectives without paying the context-length and inference costs of simulating several communicating agents.

Caveats

The abstract does not specify the backbone models, benchmark-level scores, statistical uncertainty, or the exact multi-agent configurations used for comparison. The claimed 20-fold token reduction therefore depends on the selected baselines and interaction protocols, while the behavior and interpretability of the learned latent roles remain open questions.