The evaluation toggled reasoning on and off while holding model weights, prompts, sampling settings, tokenizers, and grading fixed. Across five benchmarks and 74,944 samples, reasoning made independently generated wrong answers more likely to coincide in every dataset-scale comparison.
Reasoning reduced answer diversity, but its higher accuracy also reduced the number of problems where diversity could affect voting. After accounting for available improvement, reasoning and non-reasoning modes captured about the same share of the remaining headroom.
Confidence weighting did not solve the concentrated-error problem. Across 280 method-dataset-model combinations, no weighted method beat plain majority voting after correction, and confidence signals could reverse direction when reasoning was enabled.