The authors trained multiple reward models with a diversity-promoting regularizer, encouraging the models to capture complementary preference signals rather than reproduce the same errors.
Diverse reward-model ensembles reduce reward hacking during language-model alignment
Reward Model Boosting combines deliberately varied reward models through a learned aggregator, producing a more reliable training signal than individual or simply averaged models.
Chinese Tech
Jiabin Fan · Dezhi Ye · Yongchang Hao · Lili Mou
Alberta Machine Intelligence Institute (Amii) · University of Alberta · Tencent · Canada CIFAR AI Chair
Research Digest··2 min read
Fan et al.
Why this paper
From Tencent and 3 others · Released code
In one line
Reward Model Boosting combines deliberately diverse reward models with a boosted tree aggregator, improving reward accuracy and reducing reward hacking in RLHF.
What it released
CodeProject
What we could check
- ✓Code link in the paper (github.com)
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§