(OPD) trains smaller language agents using feedback from a larger teacher on the student's own trajectories, but in multi-turn settings the student can drift into states the teacher cannot effectively evaluate. Graph-Conditioned On-Policy Agent Distillation (GC-OPD) addresses this by storing repeated teacher executions in a graph indexed by states, preserving both successes and failures. During student training, it retrieves relevant execution evidence for the student's current state or earlier states when no direct match exists, and feeds this context along with student hindsight to the teacher for scoring.
Using the same off-the-shelf teachers without any additional training, GC-OPD boosts mean success rates substantially over vanilla OPD: from 24.70% to 48.78% on ScienceWorld (4B student), 53.36% to 85.26% on ALFWorld Unseen, and 29.10% to 37.65% on WebShop. At matched student sizes, it also outperforms all evaluated OPD baselines that use GRPO-trained teachers, with the strongest such ScienceWorld baseline reaching 46.66% compared to GC-OPD's 48.78%.
This work is interesting because it shows that a static, off-the-shelf teacher can rival or beat distillation from a teacher that was specifically optimized via reinforcement learning, simply by leveraging its own prior execution history. No task-specific teacher optimization is required, which reduces training cost and complexity.