What they did

The authors built a budget-aware propose-and-verify system for evolving agent harnesses. HarnessLens jointly explores tasks and user-configurable harness components, then uses agent execution trajectories to identify candidate modifications.

Rather than testing every modification on a fixed task suite, the framework selects tasks related to the behavior being changed. An attributable-evidence gate accepts changes only when evaluation results provide evidence linking the modification to the observed improvement.

Key findings

• Across three agent harnesses and four benchmarks, HarnessLens increased average held-out performance by 7.6–13.6%.
• It consumed substantially less evaluation budget than competing baselines, according to the authors.
• Selective, behavior-relevant verification reduced testing of unrelated capabilities.
• Explicit attribution was designed to prevent aggregate benchmark gains from concealing regressions in specific behaviors.

Why it matters

Harness optimization can require many costly model rollouts, especially when every candidate change is evaluated against an entire task suite. HarnessLens suggests that targeted verification can make automated harness development more sample-efficient while retaining checks against localized regressions.

Caveats

The reported evidence covers three harnesses and four benchmarks, so it is unclear how well the method transfers to other agent architectures, models, or real-world workloads. The abstract does not provide absolute evaluation costs, per-benchmark results, statistical uncertainty, or details about failures of the task-selection and attribution mechanisms.