What they did
The authors placed Harness-of-Harness (HoH) above existing coding-agent harnesses, organizing their work into iterative planning–coding–testing loops. HoH divides development into small, verifiable increments, balances repairs against new capabilities, separates implementation-time tests from independent evaluation, and retains versioned project histories.
They evaluated HoH on GameCraft-Bench, FrontierSWE, and ProgramBench using Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3. They also ran a multi-day deployment in which the system developed a first-person-shooter game over more than 70 iterations.
Key findings
- HoH outperformed the corresponding standalone coding harnesses across all three reported harness–model pairings and benchmarks.
- The authors report an average relative performance gain of 52.25%, with a maximum gain of 82.86% after three iterations.
- In the longer deployment, HoH produced a human-playable first-person-shooter with a storyline, implemented core mechanics, visuals, and integrated audio over more than 70 iterations.
- The framework sustained work by progressively exposing deliverables, role-specific tools, and reusable skills rather than prescribing a fixed internal workflow.
Why it matters
The results suggest that improving the orchestration around a coding agent can materially increase its effectiveness without changing the underlying model. In particular, versioned histories, independent evaluation, and narrowly scoped increments may help autonomous development systems continue making useful changes over longer periods instead of repeatedly repairing or recreating prior work.
Caveats
The reported gains cover three benchmark suites and three specific harness–model combinations, so it is unclear how broadly they transfer to other models, repositories, or production constraints. The multi-day game is a single showcase rather than evidence of reliable deployment across many large projects, and the abstract does not report human quality ratings, cost, compute use, failure rates, or comparisons with human-led development.