When training language models on tasks that require reasoning and tool use, standard practice saves only the best version (checkpoint) and discards earlier ones. But earlier checkpoints often solved problems the final one cannot, meaning useful knowledge is lost.
The authors propose Agentic-OPD, which keeps all checkpoints as teachers. It selects which teacher to learn from based on the student's current failures and only learns from that teacher on problems where the teacher is better. This "" transfers useful behaviors from weaker overall checkpoints to the student.
Across many agentic tasks, Agentic-OPD improves performance by up to 7.4 points over the standard best checkpoint. A small model using this method can outperform much larger frontier models on deep search. This shows we can recover forgotten skills from a single training run without extra training cost, making reinforcement learning more efficient.