The authors introduce OPASD, a method that adds solution-conditioned attention distillation to token-level on-policy self-distillation for reasoning models. The key challenge is that the privileged teacher attends to verified solution tokens not visible to the student; OPASD projects teacher attention onto student-visible positions and renormalizes the distribution before alignment.
Across Qwen3 models from 1.7B to 8B parameters and four competition-level mathematics benchmarks, OPASD consistently outperforms token-only distillation. Average accuracy improves by 4.98 to 8.40 percentage points. Additionally, OPASD reduces generated rollout tokens by 73.9%, estimated model compute by 72.6%, and trains 1.53x faster, avoiding the response-length inflation seen with token-only distillation.
These results show that solution-conditioned attention provides a complementary supervision signal that makes on-policy self-distillation more accurate, stable, and compute-efficient. This is interesting because it improves both performance and training efficiency without requiring additional compute.