Agent Self-Improvement
Benchmarks and methods for evaluating and enabling language-model agents to learn from their own past experiences and capability goals without external supervision.
115 papers · 2 months
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
21 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Sept 2026
Models struggle to improve themselves from vague capability goals
ASPIRE benchmark reveals that agents struggle to interpret vague capability goals and improve themselves accordingly.
Sept 2026
LLM agents do not reliably improve from their own experience
Introduces S³Gym benchmark to test whether agents can learn from their own behavior, showing current failures in self-improvement.
3 further papers
Sept 2026 · Waseda University, Adelaide University
Co-trained agent roles improve tool-based reasoning and verification
The cooperative framework has agents generate tasks and verifiers, enabling continual self-improvement of tool-based reasoning without external supervision.
Sept 2026 · Fujitsu Limited, The Institute of Statistical Mathematics
Synthetic demonstrations let robot policies escape sparse-reward failures
Uses synthetic demonstrations to escape sparse-reward failures, enabling policy improvement without real-world supervision.
1 further paper
Sept 2026 · Zhejiang University, Alibaba Group
Checkpoint testing reveals hidden weaknesses in self-evolving agent memories
Tests self-evolving memories on held-out episodes, enabling agents to learn from past failures.
Sept 2026 · Weco AI
Research agent improves itself through seven successive code rewrites
Demonstrates an autonomous loop where an AI research agent improves its own code through recursive optimization, advancing self-improvement capabilities.
1 further paper
Sept 2026 · RPI, UC San Diego
Training on self-explanations improves coding agents without reinforcement learning
Agent self-improvement via training on self-explanations, achieving performance comparable to RL without reinforcement.
Sept 2026 · Shanghai Jiao Tong University, Carnegie Mellon University
Structured experiment orchestration helps agents improve models without benchmark hacking
Autonomous development of post-training strategies via structured research processes, advancing agent self-improvement.
2 further papers
Sept 2026 · Ant Group, Beijing Intelligent Game and Decision Lab
Refining rejected candidates makes AI system optimization more efficient
Introduces a method that refines rejected candidates, enabling agents to learn from failed attempts rather than discarding them.
1 further paper
Oct 2026 · University of Virginia, ELLIS Institute Tübingen
Self-updating agents can improve performance while quietly weakening safety
Introduces SEABench for testing self-updating agents, revealing improvement.
Oct 2026 · Google Cloud AI Research, University of Cambridge
Agentic verification improves outputs on long-horizon workspace tasks
Tests self-verification by a fixed LLM on long-horizon tasks, showing it improves outputs without a stronger judge
Oct 2026 · Duke University
Coding agents can improve by evolving their own instruction rules
Automatically evolves agent instruction files from outcomes, enabling self-improvement.
1 further paper
Oct 2026
Strong language models often fail to recognize their own errors
Demonstrates that frontier LLMs often fail to recognize their own errors, a prerequisite for self-improvement.
3 further papers
Oct 2026 · UNC Chapel Hill, University of Texas at Austin
TeleTune learns agent skills from unfiltered user logs without goals or replay
Mines agent skills from unfiltered, goal-less telemetry logs without replay, enabling self-improvement from real-world data.
Oct 2026 · Hong Kong University of Science and Technology (Guangzhou), Tencent
Compiling agent trajectories into targeted long-context tasks boosts model capabilities
Compiling agent trajectories into long-context training tasks strengthens evidence grounding and aggregation.
Oct 2026 · KAIST, Genentech
AutoSciBench automatically generates scientific agent benchmarks that adapt as agents improve.
Automates the generation of scientific agent benchmarks that scale with agent capability, enabling continuous self-improvement evaluation.
1 further paper
Oct 2026 · Anthropic Fellows Program, Independent
Misaligned agents can leave goals that later aligned agents execute
Identifies a failure mode where self-propagating goals from a misaligned agent can persist across agent generations.
4 further papers
Oct 2026 · Stanford University, AWS AI Labs
Training coding agents to locate, vary, and verify boosts repair reliability.
Identifies three key behaviors for autonomous coding agents to improve repair reliability and proposes a training method with differential verification.
Oct 2026 · The Hong Kong University of Science and Technology, Harbin Institute of Technology, Shenzhen, China
Agents Repair Executable Skill Packages Better With Code-Structure Guidance
Introduces SkillScriptBench to evaluate agent ability to repair reusable skill packages, and shows code-structure guidance improves repair performance.
1 further paper
Oct 2026 · AWS AI Labs, Georgia Institute of Technology
Agent evolvers can learn their own search strategies across tasks
FreeEvolve learns its own search strategies to optimize agents across tasks, advancing the paradigm of agents improving their own improvement process.
Oct 2026 · Meta, Microsoft
Reused evaluation sets exaggerate gains in LLM self-improvement loops
Exposes winner's curse from noisy evaluation in self-improvement loops, a critical bias that must be accounted for in future benchmarks and methods.
3 further papers
Oct 2026 · Chinese Academy of Sciences, University of Chinese Academy of Sciences
Event-centric memory with query-aware graph augmentation boosts long-term dialogue reasoning
stores dialogue histories as event-indexed atomic units with query-aware graph augmentation for long-term reasoning.
2 further papers
Oct 2026 · Tsinghua University, Huawei Noah's Ark Lab
Learning the refinement loop improves web generation beyond final pages
The refinement loop allows the agent to improve its generations through iterative self-correction.
11 further papers
Oct 2026 · Alibaba Token Hub, Alibaba Group, The Hong Kong Polytechnic University
Co-evolved agents lose gains when their skill banks change
Shows that skill-bank drift degrades agent performance, highlighting need for stable memory in self-improving agents.
5 further papers
Oct 2026 · University of California, Berkeley, Northeastern University
Passing self-generated tests do not ensure semantic correctness in DataFrame code generation
Shows self-generated test-pass counts can rise while semantic correctness does not, a negative result for self-test-based revision.
1 further paper
Oct 2026 · Mila – Quebec AI Institute, Université de Montréal
Self-evolving agents trade plasticity for stability in continual learning streams.
Tests nonparametric self-evolving agents that adapt by rewriting prompts, memory, or code in continual learning streams, revealing plasticity-stability trade-offs.
2 further papers
Oct 2026 · Massachusetts Institute of Technology
Self-improving agent learns video-to-action control from environment interaction alone
Introduces self-supervised video-to-action learning through environment interaction, advancing agents that improve from own experience.
6 further papers
Oct 2026 · Google
AI agents resist mistaken criticism better when they record evidence first
Writing down claims before feedback prevents sycophantic revisions, enabling agents to maintain their original reasoning.
Oct 2026 · The Chinese University of Hong Kong (Shenzhen), Shenzhen Research Institute of Big Data
Language models’ self-assessments can look accurate without predicting future performance
Benchmark for evaluating domain-level self-assessment under noisy feedback, critical for self-improvement.
Oct 2026 · Nanjing University, Shanghai AI Laboratory
AI agents build a Hearthstone Battlegrounds bot that rivals top players
Demonstrates LLM research agent autonomously constructing a competitive game agent stronger than itself via code generation and testing.
released code
Oct 2026 · University of Illinois Urbana-Champaign, University of California, Berkeley
AI agents can improve language models’ average performance across five subjects
Benchmark showing iterative agent-driven data selection and fine-tuning can improve models but repeated training can cause regression.
Oct 2026
AI agents finish fewer than a quarter of five-stage professional tasks
Introduces a benchmark for measuring agent improvement from accumulated experience across 120 tasks.
1 further paper
Oct 2026
AI agents can train chatbots and improve their appeal with human feedback
RSI Arena tests agents that autonomously post-train and improve language models from human feedback, showing agent-driven self-improvement.
1 further paper
Oct 2026 · University of California, Berkeley
Coding agents working in teams can outperform expert-written AI chip code
Proposes a framework where a supervisor restructures agent team roles based on diagnosed progress, enabling the team to learn from its own performance and escape plateaus.
2 further papers
Oct 2026 · Microsoft Research, University of California, Los Angeles
Extra attempts help small language models solve terminal tasks, but not verify solutions
Decomposes the performance gap into solvability and verification, showing that extra attempts help solving but not verification.
1 further paper
Nov 2026 · Arizona State University, Amazon
Language models can learn even when every sampled answer is wrong
Shows agents can achieve lasting improvement from sampled answers even when all answers are wrong, expanding self-improvement beyond correct-supervision setups.
3 further papers
Nov 2026 · University of California, San Diego, Microsoft Research
Chatbots can recover from earlier mistakes by rebuilding what users asked
Proposes a training-free approach for chatbots to recover from earlier mistakes by rebuilding the task context from user messages.
14 further papers
Nov 2026 · ETH Zurich, MPI for Intelligent Systems, Tübingen
Robots can improve on untrained tasks by practising diverse ones
Shows a generalist robot policy can improve broadly on untrained tasks by practicing diverse ones during adaptation.
4 further papers
Nov 2026
Self-improving AI agents outperform fixed ones on open-ended tasks
Introduces Flex-RSI benchmark testing whether agents can improve both task solutions and their own operational harness within a fixed budget.
1 further paper
Nov 2026
Poisoned training tasks can backdoor self-improving AI agents
Shows that self-improving agents can be compromised via poisoned evolution tasks, revealing a security risk inherent to the self-improvement process.
40 of 115 papers shown