On-policy distillation (OPD) trains a student on its own generated trajectories with dense token-level supervision from a stronger teacher.
Training the teacher on student prefixes boosts on-policy distillation performance
SCOUT co-trains the teacher with outcome-based reinforcement learning on student-generated prefixes, consistently improving accuracy across model scales and tasks.
Big Tech
Langlin Huang · Hao Liu · Mononito Goswami · Xinyu Li · Prithwith Jana · Nikos Kanakaris · +2 more
Washington University in St. Louis · AWS AI Labs · Carnegie Mellon University · Georgia Institute of Technology
Research Digest··2 min read
The authors identify a teacher-side distribution shift in on-policy distillation, where the teacher supervises prefixes from the student's trajectory distribution rather than its own.
Why this paper
From AWS AI Labs and 3 others
In one line
Adapting the teacher to student-generated prefixes improves on-policy distillation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§