Ask a language model a hard question once and it often fails; generate many candidate answers and pick one, and accuracy should rise. The picking is done by the model judging its own answers, and that judging stops improving once there are many candidates.
Measuring how well models rank their own answers splits the error into a steady part (the same misjudgements repeated, called bias) and a wobbly part (noise that averages out, called variance). More computing shrinks the noise, not the steady part, which levels off.
The fix: score candidates in small random groups rather than one by one. It beats the strongest baseline by up to seven percentage points using about a third less computing.