What they did

The authors designed EvoTrainer to automate the typically static training harness in LLM RL. The framework operates in cycles: it diagnoses rollout-level evidence of failures, revises diagnostics, backtests candidate interventions, and accumulates reusable skills. The policy and harness co-evolve via a meta-level optimization loop, where the harness learns to identify and correct specific failure modes.

They evaluated EvoTrainer on three benchmarks: mathematical reasoning (MATH), competitive-programming code generation (Codeforces-style problems), and repository-level software engineering (SWE-bench). Baselines included human-engineered RL setups using GRPO and other standard algorithms under identical data, codebase, and evaluation protocols.