6B to 14B.
Better static judges do not guarantee better training signals for generative reward models.
In creative writing, Answer-Only rewarders often beat Thinking judges in online DPO even though Thinking judges have higher static accuracy.
Top University
Angel Raychev · Nathan Lile · Rafael Rafailov
SynthLabs.ai · Stanford University · Thinking Machines
Research Digest··2 min read
The authors studied generative reward models (GenRMs) as online preference providers for policy optimization in creative writing.
§