The study examined 1,262 performance issues fixed by across 582 repositories, drawn from 71,677 agent pull requests. The authors manually coded each issue and test, and re-ran 53 fixes to verify performance claims.
They found that 57% of closed fixes were merged, but the merge rate depended heavily on the agent's prior track record in the repository, not on the fix's content or tests. Only 18 of 30 merged fixes actually delivered the claimed speedup; 9 showed no gain or regressed.
These results show that a merge does not imply the fix is effective. Maintainers should be wary of trusting agent performance fixes without independent , as social factors may drive acceptance more than actual improvement.