The authors trained a Gemma-4 4B model through reinforcement learning on dilemmas extracted from classic novels in the PG19 corpus.
Rewarding moral arguments makes a language model more consistent, not sounder
Self-judged reinforcement learning halved option-order sensitivity in one evaluation, but did not improve moral sycophancy and worsened other judgment measures.
Big Tech
Marius De Kuthy Meurers · Aaron Parisi · Bernd Bohnet
Google DeepMind
Research Digest··2 min read
De Kuthy Meurers and colleagues trained a language model to produce better moral arguments using rewards from its own comparisons of competing answers.
§