When an LLM solves a math problem, it is hard to tell if it is confident or guessing. Simple signals like token probabilities only measure local word-by-word uncertainty, not the overall answer. Reliable methods require many extra completions, which is costly.
The researchers noticed that Sequential Monte Carlo (SMC) decoding, a method that runs multiple candidate answers in parallel, already contains information to estimate a global uncertainty measure called Rényi entropy, at zero extra cost.
The best version of this signal improved uncertainty detection on the MATH500 benchmark by up to 0.044 area under the ROC curve over the standard log-probability baseline. A cheap eight-particle probe using this signal identified which problems need more computation, saving 34.2% of runs while matching full accuracy.