Reasoning models trained with reinforcement learning on binary correctness rewards are overconfident: their verbalized confidence reflects how willing the model is to commit to an answer, not how likely that answer is to be right. Post-hoc rescaling rarely transfers across distributions.
The authors show that a linear probe on the hidden state between the chain of thought and the answer is substantially better calibrated. Its expected calibration error is 5 to 38 times lower than the verbalized score across four benchmarks and two model families. However, when used to pick among multiple sampled answers, the probe nearly ties majority voting but falls far short of the oracle, indicating it captures confidence but not correctness well.
They introduce probe-guided self-distillation (Probe-SD): the model samples its own traces, the probe scores them, the stated confidence is overwritten, and the base checkpoint is finetuned on this relabeled data. On Qwen3-14B, this cuts ECE from 0.178 to 0.024 in-domain and from 0.542 to 0.113 out-of-domain, beating post-hoc recalibration and self-consistency distillation. The method is notable because it achieves well-calibrated confidence and enables weighted voting using only supervised finetuning, not online reinforcement learning.