Mixture of Roles (MoRe) replaces the multi-turn, multi-agent approach to multi-perspective reasoning with a single-turn, single-agent method. It learns a codebook of steering vectors, each encoding a latent role, and uses a query-aware router to compose a single vector that steers a frozen backbone LLM. This achieves dynamic specialization without the context and cost overhead of multi-agent systems.
On reasoning and personality benchmarks, MoRe outperforms single-agent specialization baselines by 2.2% on average and matches the performance of multi-agent systems while reducing token cost by 20x. The method is trained via a three-stage supervised fine-tuning curriculum and GRPO post-training, keeping the base model frozen.
MoRe is interesting because it bridges the gap between static single-agent specialization and expensive multi-agent collaboration, offering multi-perspective reasoning in a single inference pass with minimal computational overhead.