A single robot policy that can perform many different tasks usually needs extra training to work well in a new setting. Doing this for each new task is slow and expensive. This paper tests whether a robot can improve on many tasks at once, without human help, by learning from its own successes and failures.
The researchers compared three ways for a robot to improve itself. One method simply saved successful attempts and retrained on them. Another used a "critic", a separate model that scores how good an action is, to pick the best action at test time. The third used that critic to update the robot's own policy to produce better actions. They tested these on simulated robot tasks like picking up objects.
The critic-based approach that updated the policy was the most effective. It improved the robot on tasks it trained on and, crucially, let the robot perform better on tasks it had never seen before. This shows that a robot can get broadly better at many tasks at once, which could make deploying generalist robots much more practical.