What they did
The authors first quantitatively compared modern general-purpose agents with purpose-built root cause analysis (RCA) agents. They found that general agents often performed better, but remained insufficiently accurate for production use, motivating them to focus on the surrounding adaptation layer—the agent harness—rather than building another specialist agent from scratch.
They developed OpsHarness, whose data plane stores layered operational knowledge and reusable diagnostic procedures called “idea cards.” Its control plane manages setup, diagnosis, learning, and verification. To evolve, the system contrasts successful and failed diagnostic trajectories, extracts atomic proposed updates, and accepts them only after a two-stage verification process intended to limit overfitting and regressions.
Key findings
- OpsHarness achieved 59.0% top-1 root-cause accuracy across two public benchmarks and an industrial deployment.
- The authors report a 63.4% improvement over a bare general-purpose agent.
- Reported accuracy was 4.02 times that of the evaluated baseline RCA agents.
- The comparison suggests that adapting a rapidly improving general agent through an external harness can outperform rebuilding equivalent diagnostic capabilities inside a specialized agent.
Why it matters
The study shifts attention from bespoke RCA-agent architectures toward reusable scaffolding around general agents. If diagnostic experience can be accumulated without degrading prior capabilities, operations teams may be able to adapt mainstream agents to specific systems while benefiting from continued improvements to the underlying models.
Caveats
Top-1 accuracy remains 59.0%, leaving a substantial error rate for production incident response. The evaluation covers two public benchmarks and one industrial setting, so it does not establish generalization across other infrastructure stacks, incident distributions, or long-term operational use; the durability of self-evolution and the dual-gate process under changing systems also remains an open question.