What they did
The authors evaluated multi-agent decision-making on DDXPlus, a medical differential-diagnosis benchmark, using five LLM backbones. For each instance, they paired agents’ forward posteriors—predictions derived from evidence—with a reverse posterior constructed through Bayesian backward reasoning from an explicit likelihood.
They measured agreement between the two paths using Jensen–Shannon divergence, a symmetric measure of distributional difference. This signal supported three aggregation strategies: selecting the most consistent agent (MinJS), reweighting forward predictions (FwdJS), and fusing forward and reverse distributions in log space (LogLin).
Key findings
- MinJS outperformed random agent selection across all five LLM backbones.
- FwdJS generally improved on the strongest evaluated baseline by assigning more weight to agents whose forward predictions agreed with the reverse anchor.
- LogLin achieved the best overall performance among the evaluated methods, with its largest gains on cases where agents produced conflicting answers.
- Although the reverse posterior was less accurate on its own, it was a more useful aggregation anchor than forward-only alternatives. When labels were available, a lightweight two-stage calibration improved it further.
Why it matters
The results suggest that multi-agent systems can benefit from combining differently factorized estimates, rather than merely voting over agents that may share correlated reasoning errors. A reverse inference path can provide an independent consistency check without requiring labeled examples, offering a practical way to arbitrate disagreement.
Caveats
The evidence is limited to DDXPlus and five LLM backbones, so it is unclear whether the gains extend to open-ended tasks, other domains, or settings without a tractable explicit likelihood. The abstract does not report effect sizes, and the reverse posterior’s weaker standalone accuracy indicates that its value depends on aggregation rather than superior independent prediction.