What they did

The authors derived three ways to increase environment difficulty from the objective used to train agents across multiple interaction turns. They implemented these transformations with a loop-engineered multi-agent harness that evolves tasks off-policy, rather than relying on fresh rollouts from the model being trained.

The resulting environments were scheduled generation by generation during long-horizon reinforcement learning. The authors assessed task difficulty using rollouts from Hy4 preview, Claude Opus 5, and GPT-5.6 Sol, then trained Qwen3.6-27B and Qwen3.6-35B-A3B and evaluated them on Terminal-Bench 2.1.

Key findings

• Environment evolution consistently produced harder terminal tasks for all three frontier models used in the rollout experiments.
• Qwen3.6-27B improved by 14.4 percentage points on Terminal-Bench 2.1 after training with the evolved environments.
• Qwen3.6-35B-A3B improved by 18.0 percentage points on the same benchmark.
• Because evolution is off-policy, task generation does not depend exclusively on weaknesses revealed by the current training model’s own rollouts.

Why it matters

Training terminal agents requires tasks that remain solvable enough to teach useful behavior but difficult enough to provide a learning signal. The results suggest that explicitly evolving and scheduling environments can extend the usefulness of synthetic training tasks as agents become stronger, reducing dependence on repeated on-policy task discovery.

Caveats

The reported training evidence covers two Qwen models and one terminal-agent benchmark, so it is unclear how well the gains transfer to other model families or interactive domains. Greater measured difficulty also does not by itself guarantee task diversity, realism, or useful skill coverage, and the abstract does not isolate the contribution of each evolution direction or the multi-agent harness.