← All concepts

confidence calibration

When a model says it is sure about an answer, we want that to mean it is likely correct. This is the idea of confidence calibration: making the model's expressed certainty match its true accuracy. Poor calibration means a model can be very confident in wrong answers, which is risky in real-world use. Researchers study this to build more trustworthy AI.

The papers show that models are often overconfident, especially on hard questions or new domains. While techniques like using internal model states or test-time adjustments can improve calibration, these gains frequently do not transfer across tasks. A key difficulty is that calibration methods may teach models to predict accuracy on familiar data without fixing overconfidence on novel, challenging cases.

Papers
92
Released code
2
First seen
Sept 2026
Latest
Nov 2026

92 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1AI moderators need different signals for human disagreement and costly mistakes

    A cheap content moderation model matches frontier LLM accuracy but reveals two separate uncertainties.

    Oct 2026
  2. 2Extra attempts help small language models solve terminal tasks, but not verify solutions

    Small language models fall behind frontier models on terminal tasks not only in solving them but also in verifying solutions and anticipating their own success.

    Oct 2026
  3. 3Language models can learn facts while misjudging what they know

    Large language models need explicit calibration of answerability judgments after knowledge updates to avoid confident wrong answers.

    Nov 2026

Who is working on it

University of Chinese Academy of Sciences 4Google 3Amazon 3Alibaba Group 3Carnegie Mellon University 3Institute of Information Engineering, Chinese Academy of Sciences 2

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.