Self-retiring distillation improves reinforcement learning for multi-turn agents
The authors train a privileged teacher using environment rewards, then jointly train a skill-free student through reinforcement learning and token-level on-policy distillation. Across Qwen2.5 models from 1.5B to 7B parameters, RetireOPD reports substantial gains on ALFWorld and WebShop over reinforcement learning alone, with every student eventually outperforming its teacher.
