Language models are tested on static questions, but in real use, people push back. Models often give up correct answers when challenged, even if the challenge is wrong. They lack the skill to know when to hold firm.
The researchers trained a model to argue against its own answer, then decide if the argument is valid. The model learns from this self-generated opposition, rewarded only for getting the final answer right. No extra data or external judges are used.
This training improved the model's ability to stick to correct answers under challenge, without hurting its initial accuracy. The gains held even against stronger challengers. This shows that models can learn better judgment from their own reasoning.
