The authors ran five human experiments involving 507 participants and more than 40 experiments using proprietary and open-source language models as judges.
Debate Corrects Human Judgments, but Usually Fails Weaker AI Judges
Across human and language-model experiments, adversarial arguments improved human accuracy more reliably than consultancy or automated judging protocols.
Research Lab
Maria Victoria Carro · Denise Alejandra Mester · Facundo Nieto · Oscar Agustin Stanchi · Guido Ernesto Bergman · Giovanni Franco Gabriel Marrafini · +11 more
Università degli Studi di Genova · FAIR · IALAB · Universidad de Buenos Aires (UBA) · Instituto de Investigación en Informática LIDI
Research Digest··2 min read
Carro and colleagues tested whether debate can overcome a judge’s prior beliefs when evaluating disputed scientific and technological claims.
§