Software tasks rarely spell out everything. An engineer can make a change that passes review, looks correct, and quietly breaks something nobody mentioned, which surfaces weeks later. A , which is a large language model (a program trained on huge amounts of text and code to produce replies) wired up to edit files and run tools, is usually judged only on whether it does what it was told.
This work builds a test set of real cases from open-source projects where one accepted code change had to be fixed later because it broke something. The agent is given the original task and scored on whether it finishes it and also avoids the later breakage.
Across eleven models, agents finished the stated task about three times in five, but avoided the hidden defect only about one time in seven. Told what the defect was, they fixed it readily, so the failure is noticing, not repairing.