Self-teaching language models improve when the teacher gets feedback too

Rewarding the self-teacher for correct, context-independent solutions and ignoring its unreliable guidance beats existing post-training methods.

Research Lab

Shufan Shen · Zhongni Hou · Chao Bi · Yufei Zhang · Guojun Yin · Shuhui Wang

Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan

Research Digest·30 October 2026·2 min read

Illustration: Self-teaching language models improve when the teacher gets feedback too

In one line

Teacher-Evolution Self-Distillation (TESD) enhances on-policy self-distillation by improving self-teacher supervision, outperforming existing methods.

Before you read this

Three ideas this paper builds on

on-policy distillationA training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.108 papers →privileged informationInformation available only during training that improves performance but is not accessible at inference.63 papers →recursive self-improvementAn AI system repeatedly uses its own outputs to enhance its capabilities, creating a feedback loop for autonomous improvement.49 papers →

Read it as

Large language models are often fine-tuned with rewards, but standard methods treat all tokens equally. On-Policy Self-Distillation (OPSD) improves this by having the model act as its own teacher to guide each token. However, the teacher's advice can be unreliable, causing training to plateau.

Teacher-Evolution Self-Distillation (TESD) lets the teacher generate its own outputs and score them on correctness and independence. These scores train the teacher to give better guidance and tell the student which advice to trust less. This two-way optimization boosts supervision quality.

TESD outperformed previous OPSD and reward-based methods across several tests. The finding underscores that strengthening the teacher's capability, not just the student's, is essential for continuous improvement. This opens a path to more efficient self-distillation without extra human data.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

on-policy distillation privileged information recursive self-improvement reinforcement learning with verifiable rewards