Language models taught by examples can generalize better than reward-trained models

Hou and colleagues find that supervised training can outperform reinforcement learning on reasoning beyond the training domain when its examples have carefully mixed sources and a matched sample budget.

Top University

Yunlong Hou · Fengzhuo Zhang · Yuan Cheng · Jiachun Pan · Xingyao Li · Zhuoran Yang

National University of Singapore · Yale University · Cisco Foundation AI

Research Digest·13 November 2026·2 min read

Illustration: Language models taught by examples can generalize better than reward-trained models

In one line

Supervised fine-tuning can match reinforcement learning generalization when trained on diverse data at sufficient scale.

Before you read this

An idea this paper builds on

reinforcement learning with verifiable rewardsReinforcement learning that uses rewards from objectively checkable signals like correct answers or verification feedback.159 papers →

Read it as

Large language models are often trained either by supervised fine-tuning (SFT), learning from correct answers, or by reinforcement learning (RL), learning from trial and error with rewards. Many believe RL generalizes while SFT memorizes. This paper reexamines that.

Researchers compared SFT and RL on reasoning tasks. They varied the source of SFT data, using correct solutions from different models, and the amount of data, giving SFT the same number of training examples as RL.

With a diverse mix and enough data, SFT matched RL on new problems, sometimes even better. This shows data quality and scale matter more than the training method. SFT, being simpler, can be just as effective for generalizable reasoning.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

supervised fine-tuning reinforcement learning with verifiable rewards generalization long-tailed reasoning methods