Yin et al.
Agents write verifiers that wrongly reject valid solutions more often than they accept them
Yin et al. introduce VerifierBench, evaluate 19 models on 222 tasks, and trace most false rejections to verifiers that reject every candidate solution.
Top University
Jianing Yin · Ziyi Chen · Hao Wang · Yiyou Sun · Zijian Jin · Dawn Song · +1 more
University of Pennsylvania · University of Illinois at Urbana-Champaign · University of California, Berkeley · New York University · University of Rochester
Research Digest··2 min read
The authors study whether LLM agents can generate executable verifiers that accept all valid task solutions and reject invalid ones.
§