Language models can improve by learning from their weakened copies

The authors report better mathematical and coding accuracy by training models on contrasts with degraded copies, using unlabeled prompts rather than reference solutions or external teachers.

Big Tech

Yecheng Wu · Song Han · Han Cai

Massachusetts Institute of Technology · NVIDIA

Research Digest·30 October 2026·2 min read

Illustration: Language models can improve by learning from their weakened copies

In one line

Contrasting a model with its weakened copy improves reasoning accuracy without privileged information.

Before you read this

Three ideas this paper builds on

on-policy distillationA training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.108 papers →privileged informationInformation available only during training that improves performance but is not accessible at inference.63 papers →post-training quantizationQuantizing pre-trained neural network weights and activations to37 papers →

Read it as

Large reasoning models often need a separate, more capable teacher to improve, but such teachers are not always available. Researchers created a weaker version of the same model by reducing the numerical precision of its weights (quantization) or by removing less important connections (pruning). They then trained the original model to match its outputs with the weaker one, using the difference as a learning signal.

This method improved accuracy on math and coding tasks by an average of 4.0 percentage points with quantization and 5.5 points with pruning, without needing any external teacher. It matters because models can improve themselves using only their own outputs and no extra data.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

on-policy distillation privileged information post-training quantization magnitude pruning weak-to-strong distillation