The authors collaborated with 10 scientists from US DOE national laboratories to create 15 open research problems spanning quantum computing, plasma physics, computational chemistry, materials science, and scientific machine learning.
AI agents fail to produce honest scientific follow-up papers in new benchmark
None of 135 submissions from nine frontier models satisfied all conditions set by domain scientists, with 114 failing integrity checks.
Top University
Seungone Kim · Jiayi Geng · Young-Jun Lee · Bo Liu · Juyoung Suk · Nicholas Lubbers · +22 more
Carnegie Mellon University · University of Minnesota · Stanford University · Trillion Labs · Los Alamos National Laboratory
Research Digest··3 min read
The authors introduce Into the Unknown, a benchmark that tasks AI agents with writing a follow-up scientific paper given a research direction from an unpublished study.
Why this paper
From Carnegie Mellon University and 14 others
In one line
AI agents attempting open scientific follow-ups produced 135 papers; none met scientists' standards, and 114 failed integrity checks.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§