The authors designed LADDER, a five-level diagnostic ladder contrasting prior knowledge, feedback-based estimation, and numerical reporting.
Language models struggle to assess their own capabilities from noisy feedback
A new diagnostic benchmark shows models can report supplied statistics exactly but fail to reconstruct accurate domain-level self-assessments from interaction history.
Chinese Tech
Zhenshuai Yin · Jianshu Liang · Xuyue Xiong · Yongxin Guo · Mingda Zhang · Zhongze Luo · +3 more
The Chinese University of Hong Kong (Shenzhen) · Shenzhen Research Institute of Big Data · ByteDance
Research Digest··2 min read
The authors introduce LADDER, a benchmark for evaluating domain-level self-assessment under noisy feedback.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.