Reasoning distillation from large teacher models to smaller students often suffers from the 'Gap Curse': as teachers become more sophisticated, their outputs become too complex for students to approximate, harming performance. Existing fixes either discard hard examples or use weaker intermediate teachers, sacrificing coverage or quality.
TeacherGRPO proposes a new approach called Teacher Alignment, which directly adjusts the teacher to match the student's distribution without losing data or reasoning quality. This is achieved by reframing alignment as , using with two key innovations: Curriculum Selective Alignment, which focuses rewards on meaningful reasoning gaps while ignoring noise, and Importance-Adaptive Length Regularization, which discourages verbosity without removing critical reasoning steps.
Experiments across multiple reasoning benchmarks and distillation methods show that TeacherGRPO significantly outperforms existing baselines, offering a way to close the capacity gap without compromising supervision or teaching quality.