When teaching a robot or software agent to complete long sequences of tasks, you often need to give it feedback on each step it takes. Getting that feedback normally requires a lot of extra work, like having a person label every action. This paper looks at whether a single model can do three jobs at once: choose actions, evaluate them, and train a better policy.
The researchers compared several decision models and standard large language models. They asked the models to act in two simulated environments, and separately to judge actions they had already taken. They found that when acting, the models often skipped an important first step to do an easier one first. But when judging after seeing the whole sequence, the same models correctly spotted which actions helped complete the task.
Using those judgments as feedback during training, a smaller policy improved from 72.8% to 82.9% success. The key is to ask whether each action helped rather than whether the task will succeed. This means you can use a cheap model as a trainer even if it would not be good at choosing actions itself.