The authors analyzed the effect of a privileged teacher on the student's policy gradient in on-policy distillation.
Joint on-policy learning improves self-distillation by aligning teacher with student
A new method trains a single policy as both teacher and student, where the teacher's privileged guidance is regularized to match the student's current policy.
Big Tech
Randy Ardywibowo · Arnav Dalal · Jiantao Jiao
Perplexity · NVIDIA
Research Digest··3 min read
The authors analyze why self-distillation in reinforcement learning can fail even when the teacher performs well, and derive a condition for the teacher's update to improve the student.
Why this paper
From NVIDIA and Perplexity
In one line
A privileged teacher improves RL students only if its distillation update aligns with the student's reward gradient, motivating joint on-policy training.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (4 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§