The authors argue that RL with binary rewards trains correctness but not calibrated confidence, leading to overconfident verbalized scores that measure commitment rather than evidence.
Internal confidence probe eclipses verbalized scores in calibration for reasoning models
Probe-SD distills internal calibration into model outputs, cutting calibration error by an order of magnitude.
Big Tech
Yadong Xi · Rongsheng Zhang · Tangjie Lv · Ziyang Luo · Ruochen Zhao
NetEase · Amazon · Singapore University of Technology and Design
Research Digest··3 min read
Xi et al.
Why this paper
From Amazon and 2 others
In one line
Internal hidden state probes provide well-calibrated confidence for reasoning models and can distill that calibration into the model's outputs.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§