The authors designed ConfLens to track changes in a model's answer belief at each reasoning step.
Early answer confidence reveals when language models take reasoning shortcuts
The authors introduce ConfLens and DACS, a method that detects shortcut reasoning by tracking when a model becomes prematurely committed to its final answer.
Chinese Tech
Zhaohan Zhang · Junjie Liu · Chengzhengxu Li · Chen Shen · Xiaoming Liu · Chao Shen · +3 more
Queen Mary University of London · Tongyi Lab, Alibaba Group · Xi'an Jiaotong University
Research Digest··2 min read
Zhaohan Zhang and colleagues propose ConfLens, a framework that monitors how an LLM's confidence in its final answer evolves during chain-of-thought reasoning.
Why this paper
From Tongyi Lab, Alibaba Group and 2 others
In one line
Shortcut reasoning in LLMs is detectable by tracking premature rises in answer confidence during generation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§