The authors built ProbePO on Group Relative Policy Optimization (GRPO), which assigns a group-normalized advantage to each completed response and applies that same value to every token in the response.
Sparse probes improve credit assignment for language model reasoning
ProbePO selectively evaluates intermediate reasoning states, then redistributes token-level reinforcement signals without training a separate value critic.
Academic
Diyi Hu · Ruizhong Qiu · Lin Qiu · Hanghang Tong · Hanqing Zeng
Independent Researcher · University of Illinois Urbana-Champaign · Meta
Research Digest··3 min read
Thread:Multi-Turn Credit Assignment
Hu et al.
§