The authors start from an observation: a critic trained to predict expected return can have good aggregate accuracy while failing to distinguish which of two intermediate actions makes eventual success more likely, especially for steps far from the end of a trajectory.
Critics that predict task success, not just returns, improve long-horizon agent credit assignment
SCALE adds an explicit success probability head to actor-critic RL and uses reliability-weighted changes in that probability to credit intermediate actions, raising success rates on five agent benchmarks.
Chinese Tech
Jiaxi Liu · Shixiong Kai · Zhentao Tang · Yang Li · Yuqi Cui · Mingxuan Yuan · +1 more
Shanghai Jiao Tong University · Huawei Noah's Ark Lab
Research Digest··3 min read
Liu et al.
§