OpenForgeRL: Bridging the Gap Between Agent Harnesses and RL Training

What Changed

The landscape of AI agent development has shifted toward sophisticated inference harnesses such as Claude Code, Codex, and OpenClaw. These systems are designed to manage multi-turn reasoning, complex tool execution, and external system integration. However, a significant architectural divide has emerged: while these harnesses are essential for deployment, they are notoriously difficult to integrate into standard supervised fine-tuning (SFT) or reinforcement learning (RL) training pipelines. Current open-source RL stacks often struggle to handle the stateful, multi-process nature of these inference harnesses, forcing researchers to rely on simplified, non-representative environments during the training phase.

OpenForgeRL addresses this disconnect by providing a framework that enables end-to-end training of agents directly within their native harnesses. By decoupling the training process from the inference environment, OpenForgeRL allows for the optimization of agents in the exact settings where they will eventually operate. This represents a move away from synthetic, isolated training environments toward realistic, harness-native training, enabling agents to learn from the actual complexities of tool use and GUI navigation.

Technical Details

The core of OpenForgeRL is a dual-component architecture designed to bridge the gap between high-level inference harnesses and low-level RL training stacks. The first component is a lightweight proxy that intercepts and serves the harness's model calls. This proxy acts as a bridge, recording the interactions between the harness and the agent as structured training data. This data is then formatted for use in standard RL codebases, such as veRL, allowing for seamless integration with existing training infrastructure.

The second component is a Kubernetes-based orchestrator. Because modern agent harnesses often require multiple processes and stateful interactions, running them in a standard training loop is inefficient or impossible. The OpenForgeRL orchestrator manages this by running each rollout in its own isolated, remote container. This containerization ensures that the environment state is preserved and that the harness operates exactly as it would in a production deployment, without interfering with the primary training process.

By separating the training logic from the environment execution, OpenForgeRL allows for massive parallelization. Researchers can scale the number of rollouts across a Kubernetes cluster, enabling the training of agents on complex tasks that would otherwise be computationally prohibitive. This architecture supports a wide range of environments, from tool-based agents that interact with code repositories to multimodal agents that perform computer-use tasks through GUI interaction.

Benchmark Analysis

OpenForgeRL has been validated across several complex benchmarks, demonstrating that harness-native training leads to superior performance compared to open baselines of similar size. In the domain of tool and claw-based agents, OpenForgeClaw showed significant proficiency. On the ClawEval benchmark, it achieved a pass^3 score of 31.7 and a pass@3 score of 55.9. Furthermore, on the QwenClawBench, it reached a score of 33.7.

In the realm of GUI and browser-based agents, OpenForgeGUI demonstrated robust capabilities. It achieved a score of 37.7 on the OSWorld-Verified benchmark, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. These results indicate that training within the native environment allows the agent to better navigate the complexities of real-world interfaces, often matching or surpassing models that are several times larger.

Developer Implications

For developers and researchers, OpenForgeRL lowers the barrier to entry for training high-performance agents. Previously, the difficulty of integrating custom harnesses into RL pipelines meant that many agents were trained on simplified proxies, leading to a 'reality gap' where models performed well in training but failed in production. OpenForgeRL eliminates this gap by ensuring the training environment is identical to the deployment environment.

Furthermore, the framework provides new insights into agent behavior. By analyzing how different harnesses—such as ZeroClaw or Codex—influence the learning process, developers can better understand which environmental factors contribute to agent reliability. The research indicates that while RL significantly improves agentic reliability in areas like self-verification, tool coverage, and multi-step planning, there remains a persistent challenge in error recovery. Developers can now use OpenForgeRL to specifically target these weaknesses by designing training tasks that focus on recovery scenarios within the actual deployment harness.

Bottom Line

OpenForgeRL provides a critical piece of infrastructure for the next generation of AI agents. By enabling end-to-end training within native inference harnesses, it solves a major bottleneck in the development of complex, tool-using, and GUI-navigating agents. As the field moves toward more autonomous, multi-turn systems, the ability to train directly in the environment of deployment will likely become a standard requirement for building reliable, production-ready AI agents.