What they did

The authors designed a batch-level harness-evolution method that jointly analyzes failures from multiple task instances. This cross-instance aggregation is intended to separate systematic defects in the runtime harness from failures attributable to a particular model or isolated task.

Ecdysis also uses Failure-Driven Collaborative Refinement, in which multiple diagnostic roles assess likely causes and iteratively refine specifications for modifying the harness. The resulting procedure was compared with existing iterative harness-evolution methods on training efficiency and the reasoning accuracy of the evolved harnesses.

Key findings

• Ecdysis achieved up to a 1.84× speedup in harness training relative to the evaluated existing methods.
• The resulting harnesses improved reasoning accuracy by 18.56% over those comparison methods.
• Aggregating evidence across task instances reduced reliance on repeated, failure-by-failure execution and code revision.
• The authors report that prioritizing recurring failures helped avoid unnecessary accommodation to model-specific errors.

Why it matters

Runtime harnesses can materially affect how well an LLM agent plans, invokes tools, and recovers from errors, but optimizing them through repeated trial and error is costly and can overfit to observed tasks. Ecdysis suggests that treating harness improvement as structured failure diagnosis—especially across multiple tasks—can produce more general modifications with fewer agent executions.

Caveats

The supplied abstract does not identify the evaluated models, tasks, baselines, sample sizes, computational costs, or statistical uncertainty, making it difficult to judge how broadly the reported gains generalize. The 1.84× figure is a best-case speedup, and the stated 18.56% accuracy improvement is not clarified as relative or absolute; independent evaluation on unseen task distributions would be important.