RMB jointly trains multiple reward models with an HSIC regularizer that discourages correlated predictions, then learns a lightweight decision-tree aggregator to combine their complementary strengths. This targets shared blind spots that simple reward-model averaging may preserve.
Experiments reportedly improve reward prediction accuracy on both in-distribution and out-of-distribution data. Best-of-N sampling and PPO experiments also show less and better RLHF performance, though the supplied text provides no numerical effect sizes.