The authors formalized the under-thinking problem as a preference alignment issue at critical thought switching points.
New preference optimization curbs LLMs' wasteful thought switching, cuts tokens by 63%
SteadyThought framework identifies unnecessary switching points and uses preference optimization to encourage persistent reasoning, reducing token consumption by up to 63.8% without losing accuracy.
Chinese Tech
Zhenyue Peng · Lin Yang · Jie Wang · Xiaoqi Ni · Hanzhu Chen · Xize Liang · +4 more
Chinese Academy of Sciences · University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · Huawei Technologies Co., Ltd. · Tianjin University
Research Digest··2 min read
The authors propose SteadyThought (ST), a thought-level preference optimization framework to address 'under-thinking' in large reasoning models, where models frequently switch reasoning paths even when a promising path is found.
§