The paper introduces Retrospection-Only Fine-Tuning (ROFT), an online procedure where an agent attempts a task, receives feedback, writes a retrospective explanation, and is fine-tuned only on those explanation tokens. No reward-based policy update or teacher is used. In software-engineering tasks with a 4B-parameter model, ROFT achieves 49.2% and 26.8% solve rates on held-out SWE-bench Verified and Pro after 20 updates, outperforming GRPO's 48.0% and 25.3% after 40 updates, and it makes faster early progress.
Training on self-explanations improves coding agents without reinforcement learning
Retrospection-only fine-tuning transferred lessons from an agent’s explanations into better actions on held-out software-engineering tasks.
Big Tech
Jonathan Light · Christopher Zhang Cui · Jeonghye Kim · Roger Creus Castanyer · Emiliano Penaloza · Zhengyan Shi · +4 more
RPI · UC San Diego · KAIST · Mila · Microsoft Research
Research Digest··2 min read
Thread:Agent Self-Improvement
In one line
Fine-tuning an agent on its own self-generated retrospections improves task performance without external rewards or RL.
Before you read this
Two ideas this paper builds onRead it as
The authors introduce Retrospection-Only Fine-Tuning, or ROFT, in which an agent attempts a task, explains its experience, then updates its weights using only the explanation tokens.
The authors tested ROFT on software-engineering tasks using Qwen3.5-4B.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.