The authors compared unaligned base models with instruction-tuned versions across five model families on factual benchmarks. On long-tail factual queries, instruction tuning multiplied high-confidence errors (p >= 0.95) by 10x to 35x. Layer-wise Logit Lens probing showed overconfidence emerges in late layers, where wrong-answer margins expand past 4.0 points after staying near zero earlier.
To counter this, they introduced an entropy-dependent margin bound for direct , called BDM-DPO. In multi-epoch experiments with Mistral-7B, it reduced high-confidence errors by up to 35.3% relative to standard DPO while keeping performance on evaluated general reasoning benchmarks. The result reframes confident hallucinations as a side effect of alignment, not just missing knowledge or decoding randomness.