Multi-agent systems (MAS) often rely on inference-time orchestration, but post-training is needed for stable role competence. Reinforcement learning suffers from sparse team rewards or requires per-task local rewards. (OPD) provides dense token-level supervision without local rewards. This paper introduces MAS-OPD, which extends OPD to MAS. It uses Role-Advantage Specialization, defining role advantage as the difference in teacher signals under target vs non-target role conditions, and Privileged Attribution for Coordination, which identifies interaction conflicts and provides that information only to the teacher. Experiments on code and mathematics benchmarks show MAS-OPD attains the highest mean score at both student scales, leading to clearer role specialization and more effective collaboration. This is interesting because it avoids manual reward design per task and provides denser supervision than reinforcement learning.
Teacher-guided training improves specialization and coordination in multi-agent models
MAS-OPD trains interacting small models with token-level teacher feedback that distinguishes role-specific behavior and attributes collaboration failures.
University of Science and Technology of China · Tencent · Zhejiang University · National University of Singapore
In one line
MAS-OPD achieves the highest mean score on code and math benchmarks by using on-policy distillation with role-advantage specialization and privileged attribution.
Before you read this
Three ideas this paper builds onThe authors trained multi-agent systems on CodeContests and Polaris-Dataset-53K, using specialized coder and tester roles for programming and reasoner and tool-user roles for mathematics.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.