The authors designed a measure that turns a reported belief (probability) into the number of independent, equally informative readings that would produce the same posterior under Bayes' rule.
LLM aggregators partly count restated evidence as new readings, measure shows
A copy weight metric quantifies how much repeated statements inflate beliefs, and simple protocol changes nearly eliminate the effect.
Academic
Jianxin Gao · Runze Li · Tianyi Yu · Liangwei Ren · Bohan Chen · Zining Wang
China Agricultural University · Jilin University · Tianjin University of Finance and Economics · Tianjin University of Science and Technology
Research Digest··3 min read
The authors develop a metric that converts an LLM aggregator's reported probability into the number of independent readings it implies, so a restated copy receives a weight between 0 (counts only sources) and 1 (counts every statement).
Why this paper
From China Agricultural University and 3 others
In one line
LLM aggregators treat repeated evidence as partly independent, inflating confidence and triggering decisions unsupported by the underlying sources.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (4 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§