What they did
The authors designed a framework that decouples the training and inference harnesses of modern AI agents. OpenForgeRL uses a lightweight proxy that intercepts model calls made by an agent harness (e.g., OpenClaw, Codex) during rollouts, recording them as training data for a standard RL codebase (veRL). A Kubernetes orchestrator launches each rollout in an isolated remote container, allowing scalable training on arbitrary environments without modifying the harness or the RL stack. They validated the framework across three settings: a tool-using claw agent, a multimodal GUI browser agent, and a general computer-use agent.
Key findings
- OpenForgeClaw achieved 31.7 pass@1 and 55.9 pass@3 on ClawEval, and 33.7 on QwenClawBench, outperforming open baselines of comparable size.
- OpenForgeGUI reached 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager, matching or surpassing models several times larger.
- The framework required only hundreds to a few thousand training tasks to yield these improvements.
- RL training improved agent reliability, including self-verification, tool coverage, and completion of multi-step plans, though error recovery remained weak.
- Different harnesses (ZeroClaw, OpenClaw, Codex) varied substantially in how learnable they were under the same RL pipeline.
Why it matters
This work bridges a gap between production-grade agent harnesses and standard open RL stacks, enabling researchers to directly train agents in the same stateful, multi-tool environments they will be deployed in. By making end-to-end RL training practical for complex harnesses, it opens the door to systematically studying how harness design and RL interact, and may accelerate progress toward more reliable agents.
Caveats
The framework was validated on a limited set of harnesses and environments. The experiments used mostly relatively small backbone models (7B-13B parameters), so scaling behavior to larger models is not yet clear. Error recovery and other higher-level agentic competencies remained poor, and the authors note that the required Kubernetes infrastructure may present a deployment barrier for smaller labs.