AI systems are increasingly expected to build and improve the software behind AI, not just answer questions. Simplified tests may not show whether an agent can handle realistic engineering work from start to finish.
Singularity-Bench tests software agents, programs that can plan and carry out tasks, on six kinds of AI development, from training models to coordinating computers. Agents build systems from minimal code, which are compared with human-built systems on the same hardware. The evaluation also checks for , exploiting loopholes to score well without doing the intended work, and introduces Breakwater to block the attacks tested.
Opus 5.5 with Claude Code matched or beat the human-built systems on every task, with a score of 1.25. Other agents performed less consistently. The results suggest leading agents can handle broad AI engineering, but do not show they can fully take over the process of building stronger AI.