Training a language model to reason often requires labeled correct answers, which are expensive or impossible to get for many tasks. Multi-agent debate, where several copies of a model discuss a problem and then vote, offers a way to learn without labels, but simple voting can be fooled by groupthink or ties.
The authors introduce Alignment by Confident Debate (AbCD). Instead of taking the majority answer as correct, AbCD groups equivalent answers into clusters and picks the cluster whose members show the strongest confidence in their answer, measured by the probability of the specific answer tokens. The chosen cluster is treated as the preferred answer for training, and the others as dispreferred, without ever needing a true answer key.
On math and code-generation tasks, AbCD boosted accuracy by up to 21.5 percentage points over using simple debate or majority voting alone. This shows that confidence-weighted clustering can turn a model's own debates into reliable training data, reducing the need for expensive human or automated correct-answer labels.