AI teacher adapts its hints as a language model learns, boosting performance across four domains

A frozen teacher diagnoses the student's bottlenecks and designs both class-wide strategies and per-task hints that are withdrawn at test time.

Academic

Zijian Zhang · Rizhen Hu · Jiangweizhi Peng · Shiyang Li · Sujay Bhatt · Sihan Zeng · +1 more

University of Minnesota, Twin Cities · JPMorgan AI Research

Research Digest·12 November 2026·3 min read

Illustration: AI teacher adapts its hints as a language model learns, boosting performance across four domains

In one line

AutoScaffold uses a frozen teacher LLM to automatically design adaptive training scaffolds that improve language model RL performance across domains.

Before you read this

Three ideas this paper builds on

reinforcement learning with verifiable rewardsReinforcement learning that uses rewards from objectively checkable signals like correct answers or verification feedback.159 papers →agentic scaffoldingExternal supports like training curricula, tool selection, and success verifiers enable agents to handle long-horizon or out-of-distribution tasks.33 papers →failure attributionIdentifying which component or step in a system caused an observed failure, often using structured trace representations.140 papers →

Read it as

Reinforcement learning trains language models by rewarding correct answers, but this only gives a single pass or fail score per attempt, wasting most of what the model does. It is like grading a student only on whether they got the right answer, ignoring their method or how close they came.

AutoScaffold adds a separate, unchanging teacher AI that watches the student model's attempts during training. The teacher spots where the student struggles, then writes helpful hints or strategies into the student's prompts to guide it through hard tasks, adjusting the help as the student improves.

Tested on four very different tasks, models trained with AutoScaffold later solved more problems on their own than those trained without it. For example, success on an everyday household task simulation rose by just over seven percentage points, and average exact match on question answering rose by four percentage points.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

reinforcement learning with verifiable rewards agentic scaffolding curriculum learning failure attribution privileged information