LLMs (large language models) now often produce a long reasoning trace, a visible chain of thought, before writing code. That extra text costs time and computing power, but many evaluations only check whether the final code is correct.
The authors ran coding and bug-fixing tasks through several models, with and without reasoning, and shortened stored traces to see if a model could finish from less.
Reasoning helped on some benchmarks: success reached 76.5% on a code-generation set and 7.9% on real bug-fixing, still low. On another set, extra thinking made no difference despite thousands of tokens per task, and shortened traces worked as well. More reasoning is not always better; evaluations should report its cost.