Dynamic rubrics improve credit assignment for long-horizon agent training

DRACO converts trajectory-level rubric judgments into step-specific training signals without ground-truth verifiers or a learned attribution model.

Big Tech

Shubham Gandhi · Saurabh Goyal · Kiran Kate · Yara Rizk

Carnegie Mellon University · IBM Research

Research Digest·5 September 2026·2 min read

Thread:Credit Assignment in Agentic RL

In one line

DRACO uses dynamic rubrics to assign per-step credit in GRPO, improving long-horizon agent training without ground-truth rewards.

Before you read this

Three ideas this paper builds on

credit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.171 papers →agentic reinforcement learningTraining approach that lets LLM-based agents learn multi-step tool use and planning via reinforcement learning, often with dense or per-tool rewards.102 papers →advantage estimationIn policy optimization, a method that calculates the relative value of actions using group statistics or baselines to guide learning.45 papers →

Read it as

DRACO addresses the problem of training agents on long-horizon tasks when no ground-truth success signal is available. It generates rubrics dynamically during training, scores them once per completed trajectory, and redistributes that judgment over the steps responsible for annotated rubrics to produce differentiated per-step advantages in GRPO. The redistribution is closed-form and does not introduce any trained attribution module.

On AppWorld, DRACO gains 15.9 points over the base model and 5.3 points over GRPO trained with a sparse ground-truth reward, despite not using any verifiers itself. On out-of-domain Tau-Bench, it gains 5.3 points over the base model even without a frontier judge, beating both ground-truth-reward training and other rubric-based training settings. These results are significant because many real-world agent domains lack programmable verifiers, making DRACO's outcome-blind approach widely applicable.

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

What it released

Code

Concepts

credit assignment multi-criteria rubrics agentic reinforcement learning advantage estimation reinforcement learning group relative policy optimization