Computer programs that automatically fix software bugs, called , often keep working even after they have passed their own tests. This extra work uses time and computing resources and often does not improve the final patch.
Researchers replayed recorded runs of these agents from thousands of bug-fixing tasks. They scored intermediate patches against hidden tests to see when a correct solution first appears and how much extra work follows.
They found that stopping right after a patch passes its own tests could save up to 57% of the cost, but quality sometimes suffered. A rule that waits until the same patch passes twice saved 23% in a controlled test, but they could not prove it maintains the same quality. This shows that developers can reduce computing costs by stopping earlier but must verify that the final fix remains correct.
