RRT models each rubric criterion with difficulty and discrimination parameters from , using a Response Parameter Network (RPN) to predict these from prompt and criterion text. The reward is the maximum a posteriori estimate of latent quality from the verdict pattern, replacing the standard additive sum of points. During training, an online expectation maximization algorithm updates the RPN as the policy distribution shifts.
On medical and science datasets with a 4B policy, RRT improved macro criterion score by 1.7 points over (GRPO). Gains reached 2.8 to 5.6 points on hard and very hard criteria. Adaptive Fisher selection with half the criteria budget kept the macro score within 0.1 points of GRPO with full judging.
This work is interesting because it reduces the number of judge requests—a major cost in RL from rubric feedback—while improving reward quality. The approach leverages the latent structure of criteria, offering a principled alternative to fixed point weights.