Developers often ask to do a series of jobs on the same program, such as building a component then upgrading it. But most tests only check one job at a time, hiding whether the agent can keep working on its own earlier code.
The researchers created a of 69 long projects, each with 5 to 15 tasks that build on the agent's own previous work. The tasks cover real engineering work like adding features or switching to a new version of a library.
The best agent succeeded on only 18.5% of full projects. More than half of the projects were not solved by any agent. This shows current coding agents often fail when work accumulates, so developers should not rely on them for long sequences without careful checking.