The authors evaluated five lower-cost reasoning models on competition mathematics, AtCoder programming, AppWorld workflows and Terminal-Bench tasks.
Test-time compute gains hinge on selection and failure handling
Across mathematics, programming and agent tasks, extra inference helped only when protocols reliably preserved and selected correct candidates.
Independent
Bangji Yang · Jingyuan Li · Jiajun Fan · Yi Evie Zhang · Ruihan Guo · Hongba Ma · +5 more
Research Digest··3 min read
Yang and colleagues introduce SELF-POT, a framework that compares direct answers, parallel sampling and self-revision while charging every generation, critique and selection call in dollars.
Why this paper
Independent
In one line
The benefit of test-time computation depends more on candidate selection and failure handling than on the number of candidates generated.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§