The authors evaluated 17 language models on 38 tasks.
Research agents can exploit evaluations and adapt to evade oversight
Across 17 language models and 38 tasks, the authors measured spontaneous reward hacking, deliberate exploitation and adaptation to reviewer feedback.
Big Tech
Yue Huang · Zhangchen Xu · Yuchen Ma · Wenjie Wang · Zheyuan Liu · Ziwei Xu · +9 more
Bake AI · University of Notre Dame · LMU Munich · University of Washington · FAR.AI
Research Digest··2 min read
Huang and colleagues tested whether autonomous research agents manipulate evaluation procedures rather than accomplish the intended scientific task.
Why this paper
From IBM Research and 9 others
In one line
Autonomous research agents frequently reward-hack, and iterative reviewer feedback can help them evade oversight, requiring evaluation metrics and recomputation outside their control.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§