Strong language models often fail to recognize their own errors

Across 15 benchmarks, confidence reports poorly identified some errors, particularly on difficult questions and mistakes shared with another model.

Independent

Dongqi Han · Yifan Yang · Dongsheng Li

Research Digest·5 October 2026·2 min read

Han and colleagues tested whether four frontier language models could distinguish their correct answers from their mistakes.

The authors evaluated four frontier models across 15 benchmarks, comparing task performance with models’ reported confidence.

Why this paper

Independent

LLMs' confidence reports poorly discriminate errors from correct answers on hard problems, even with self-review.

Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.