The authors found that existing black-box uncertainty quantification methods, like paraphrase-based self-consistency and confidence verbalization, perform no better than simple repeated sampling for large reasoning models (LRMs) because RL-based alignment suppresses output variability. To address this, they introduced relaxation operators that broaden the model's output distribution and proposed Jailbreak for Uncertainty (J4U), a technique that repurposes jailbreak-style prompt transformations to increase variability. Across 4 LRMs and 3 datasets, J4U consistently outperformed repeated sampling and state-of-the-art black-box methods, achieving up to 15.8% reduction in Expected Calibration Error (ECE) versus 3.1% for the strongest baseline. This is interesting because it provides a practical, black-box-compatible calibration tool for high-stakes deployment of LRMs, challenging the assumption that alignment necessarily harms uncertainty quantification.
Jailbreak prompts improve uncertainty calibration in black-box reasoning models
The authors show that prompt-level relaxation via jailbreaks broadens output distributions and yields significantly better-calibrated confidence than existing black-box UQ baselines.
Research Lab
Lucas Biechy · Cédric Eichler · Adrien Boiret · Nicolas Anciaux
Petscraft · Inria · Université Paris-Saclay · INSA CVL · Université d'Orléans
Research Digest··1 min read
In one line
Jailbreak prompts, repurposed for uncertainty estimation, significantly improve confidence calibration in black-box large reasoning models.
Before you read this
Three ideas this paper builds onconfidence calibrationConfidence calibration aligns a model's predicted confidence with its actual accuracy across predictions.92 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.296 papers →safety alignmentProcess of training models to recognize and refuse harmful requests, ensuring outputs remain safe.116 papers →
Read it as
Biechy et al. investigate uncertainty quantification (UQ) for large reasoning models (LRMs) in black-box settings, where logits are unavailable.
The authors focus on question-answering for LRMs, where alignment via tends to produce systematic overconfidence and suppress output variability.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.