Qwen-AgentWorld-35B-A3B: A Native Language World Model for Agentic Environment Simulation

What Changed

Qwen-AgentWorld-35B-A3B represents a significant development in language world models (LWMs) by being explicitly designed as a "native world model" from its initial training stages. Unlike models that adapt general-purpose LLMs for environment simulation post-hoc, Qwen-AgentWorld integrates environment modeling as its core training objective from the Continual Pre-Training (CPT) stage. This fundamental shift enables the model to simulate agentic environments by predicting the next environment state based on an agent's actions and interaction history, utilizing long chain-of-thought reasoning.

This model unifies seven distinct agent interaction domains: MCP (tool calling), Search, Terminal, SWE (software engineering), Android, Web, and OS. This comprehensive coverage spans both text and graphical user interface (GUI) interaction environments, a notable consolidation within a single model. The training pipeline involves a three-stage process: CPT for environment knowledge injection, Supervised Fine-Tuning (SFT) to activate next-state-prediction reasoning, and Reinforcement Learning (RL) using GSPO to sharpen simulation fidelity. This structured approach aims to produce a generalizable, scalable, and controllable simulator capable of zero-shot generalization to out-of-distribution (OOD) environments.

Technical Details

Qwen-AgentWorld-35B-A3B is a Causal Language Model built upon the Qwen3.5-35B-A3B-Base architecture. It features a total of 35 billion parameters, with 3 billion activated during inference. The model's hidden dimension is 2048, and it utilizes a padded token embedding of 248320. The architecture comprises 40 layers, with a hidden layout structured as 10 blocks, each containing three Gated DeltaNet layers followed by a Mixture of Experts (MoE) layer, and one Gated Attention layer followed by an MoE layer.

Key architectural components include:

• Gated DeltaNet: Employs 32 linear attention heads for V and 16 for QK, each with a head dimension of 128.
• Gated Attention: Features 16 attention heads for Q and 2 for KV, with a head dimension of 256. It also incorporates a Rotary Position Embedding Dimension of 64.
• Mixture of Experts: Consists of 256 experts, with 8 routed experts and 1 shared expert activated. Each expert has an intermediate dimension of 512.

The model supports a substantial context length of 262,144 tokens, crucial for multi-turn environment simulation. Deployment is compatible with popular inference frameworks such as Hugging Face Transformers, vLLM, and SGLang. For vLLM, the --language-model-only flag is required due to the model architecture including visual component definitions while the checkpoint only contains language model weights. The model's inference process defaults to a thinking mode (<think>...</think>) to reason about environment state transitions before generating predicted observations.

Benchmark Analysis

Qwen-AgentWorld-35B-A3B was evaluated on AgentWorldBench, an open-ended evaluation rubric that scores predicted environment observations across five dimensions: Format, Factuality, Consistency, Realism, and Quality, normalized to a 0-100 scale. The model's performance is detailed below, alongside other prominent models:

Qwen-AgentWorld-35B-A3B achieved an overall score of 56.39. While not leading in all categories, it demonstrates competitive performance across the seven domains, particularly in Search (36.69) and SWE (65.63), where it outperforms several other models including its base Qwen3.5-35B-A3B. The larger Qwen-AgentWorld-397B-A17B variant shows stronger overall performance (58.71) and leads in Search and SWE categories.

Developer Implications

Developers working on agentic AI systems, particularly those requiring robust environment simulation, can leverage Qwen-AgentWorld-35B-A3B as a foundational component. The model's native world model design and comprehensive domain coverage streamline the development of agents that interact with complex, multi-modal environments. The ability to simulate diverse environments—from Linux terminals and web browsing to software engineering tasks—within a single model reduces the need for specialized models or complex integration efforts.

The model's compatibility with standard inference frameworks like Hugging Face Transformers, vLLM, and SGLang ensures ease of deployment and integration into existing ML pipelines. The provided quickstart guides for API serving and direct inference, along with domain-specific system prompt templates, offer clear pathways for immediate implementation. Developers can use the recommended sampling parameters (temperature=0.6, top_p=0.95, top_k=20) and an adequate output length (up to 32,768 tokens) to optimize simulation fidelity. The AgentWorldBench evaluation framework is also available for developers to assess and fine-tune their agent's interactions within simulated environments.

Bottom Line

Qwen-AgentWorld-35B-A3B establishes a new paradigm for agentic environment simulation through its native language world model approach. By embedding environment modeling as a core training objective and unifying seven distinct interaction domains, it offers a versatile and robust simulator. Its architecture, featuring a Gated DeltaNet, Gated Attention, and a Mixture of Experts, combined with a substantial context length, supports detailed and long chain-of-thought reasoning for predicting environment states. The model's performance on AgentWorldBench indicates its capability across various domains, positioning it as a significant tool for developers building and evaluating general-purpose AI agents.

Pneumetron