When you give a language model a task, its performance depends not only on the model itself but also on the prompts, tools, and workflow that guide it. This whole setup is called its harness. A natural question is whether the model can improve not just its answer but also the way it works. This matters because engineers deploying agents want them to get better on their own.
Researchers built a benchmark called Flex-RSI to test this. They gave frontier language models tasks in vision and robotics and let each agent improve its solution and its own harness over multiple rounds, always starting from the same general setup. They tested 18 different models on 24 tasks.
The main finding: helps most when the agent has to build its own solution, not just pick one answer. It also helps more when feedback scores each attempt on a scale rather than just pass or fail. The biggest gaps between top and open-weight models appeared on tasks that check physical stability or exact correctness. These findings matter because they show that how you set up the harness and feedback loop is as important as the model itself.