for multi-turn agents suffers from error accumulation because a single mistake changes the environment and leads to further errors. The authors find that over half of failed rollouts contain a pivotal mistake that occurs early and remains recoverable: correcting the action or guiding the next few turns restores success. PivotOPD uses a teacher model to detect these pivotal turns and provides both a gold action to prevent the mistake and recovery actions for subsequent turns. Preventive distillation uses reverse KL to steer the student away from the mistake, while recovery distillation uses forward KL to teach recovery behaviors the student rarely samples. Across 13 baselines on ALFWorld, WebShop, and Search-based QA, PivotOPD achieves the strongest average performance for 1.7B and 8B students, including a 5.5% improvement on ALFWorld with the 1.7B model. Gains transfer to software engineering: a Nemotron-3.5 student improves resolve rate by 3.2% on SWE-Bench Verified.
Training agents around decisive errors improves prevention and recovery
PivotOPD identifies consequential mistakes in student rollouts, then distills both the preferred action and subsequent recovery behavior.
Big Tech
Yinghui He · Yapei Chang · Khushi Bhardwaj · Daniele Molinari · Tugrul Konuk · Jan Kautz · +1 more
Princeton University · NVIDIA · University of Maryland
Research Digest··3 min read
In one line
PivotOPD jointly trains agents to prevent and recover from pivotal mistakes, achieving top average performance across three benchmarks.
Before you read this
Three ideas this paper builds onon-policy distillationA training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.131 papers →multi-turn tool useThe capability of an agent to invoke external tools sequentially over multiple conversational turns to complete complex tasks.52 papers →failure attributionIdentifying which component or step in a system caused an observed failure, often using structured trace representations.146 papers →
Read it as
He and colleagues study why language agents fail during multi-turn tasks, where one bad action can alter the environment and compound later errors.
The authors define a pivotal mistake as an action that moves an agent farther from task completion.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.