The authors analyzed reasoning traces at token level, using normalized log probability, a model-relative measure of how confidently each token was generated.
Language models can cut reasoning tokens by targeting low-value text
TokenProbe uses token-level confidence signals during reinforcement learning to compress reasoning traces while preserving reported task accuracy.
Big Tech
Runjia Zeng · Hang Hua · Yiyang Liu · Zhiqiang Tao · Ruixiang Tang · Qifan Wang · +2 more
Purdue University · Rochester Institute of Technology · MIT-IBM Watson AI Lab · University of Missouri-Kansas City · Rutgers University
Research Digest··2 min read
Zeng and colleagues examine whether every token in a chain-of-thought trace contributes equally to solving a problem.
Why this paper
From Meta AI and 5 others
In one line
Normalized log probability identifies core and redundant tokens; compressing redundant tokens reduces token usage by 76% while preserving reasoning quality.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§