The authors built an agentic system that attempts to improve a robot's performance on a household task (placing condiments on the top shelf of a fridge) by autonomously writing and testing new skills. After 123 rounds of self-improvement, the agent successfully discovered one capability on its own (an active-viewing model) but the target task never succeeded. The paper identifies three structural reasons: chained perception modules like SAM 3 detect objects but not relational concepts like "the top shelf", skill chains cause learning to concentrate on early steps, and the evaluation harness misdirects optimization through weak tests and misleading memory. The findings show that the bottleneck is not the agent itself but the surrounding system architecture.
Robot self-improvement stalls when perception, skill chains and tests mislead
Across 123 autonomous improvement rounds, new capabilities passed simulation tests but never completed the target household task.
Top University
Jiaming Wang (National University of Singapore)
Research Digest··2 min read
In one line
An agentic robot self-improvement system failed because perception modules can't resolve relational concepts, skill chains bias learning, and evaluation harnesses misdirect optimization.
Before you read this
Three ideas this paper builds onrecursive self-improvementAn AI system repeatedly uses its own outputs to enhance its capabilities, creating a feedback loop for autonomous improvement.66 papers →agent skillsLearned capabilities that enable AI agents to perform specialized tasks with improved effectiveness.120 papers →agent harnessA system that structures, verifies, monitors, or controls how an AI agent interacts with external tools, environments, or execution runs.186 papers →
Read it as
Wang built an agentic robotics system that diagnoses failures, writes skills or installs external models, and tests changes without human-written robot code.
The system used coding agents to inspect robot failures, modify a skill library, find and install external perception or planning models, and validate every change in simulation.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.