The authors trained models on valid, minimal iGSM solution traces, then separately measured final-answer accuracy and trace validity.
Correct answers can conceal invalid grade-school math reasoning traces
Programmatic verification showed that answer accuracy and trace validity separate on difficult problems and under corrupted supervision.
Academic
Ratish Puduppully · Pranabendu Misra · Paarth Iyer · Durgesh Kalwar · Vardhan Palod · Subbarao Kambhampati
IT University of Copenhagen · Chennai Mathematical Institute · Indian Institute of Technology Jammu · Arizona State University
Research Digest··3 min read
Puduppully et al.
Why this paper
From IT University of Copenhagen and 3 others
In one line
Correct answers do not certify valid chain-of-thought traces, especially under distribution shift, and plausible minimal traces do not establish selective planning.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§