6 model to estimate values for intermediate states after observing the completed trajectory.
Frozen LLM critics sharpen credit assignment for long-horizon agents
LVRL uses a pretrained model to redistribute sparse terminal rewards across individual actions without training a separate value network.
Big Tech
Kaijie Zhu · Weixiang Yan · Omar Attia · Abbas Kazerouni · Alexander Toshev · Andrew Szot
Apple
Research Digest··2 min read
Thread:Multi-Turn Credit Assignment
Zhu et al.
Why this paper
From Apple · Part of Multi-Turn Credit Assignment, now 4 papers
In one line
A frozen LLM value critic, policy-aware via in-context conditioning, improves step-level credit assignment in agentic RL, beating GRPO and PPO.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (3 noted)
- ✓Reports numbers on named benchmarks (4 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§