Merging specialist AI models saves 42% of training compute compared to joint training
The authors systematically compared on-policy distillation (OPD) and group relative policy optimization (GRPO) across math and code tasks, measuring accuracy against training FLOPs. They found that while external-teacher OPD generally underperforms GRPO, OPD using the student's own GRPO checkpoint as teacher recovers that checkpoint's accuracy with 50 times fewer distillation FLOPs. The key practical result is multi-teacher OPD: consolidating per-domain GRPO specialists into one student adds only 5.2-8.0% extra training cost, and the full pipeline uses 42% fewer total FLOPs than joint GRPO for two domains.









