Intern-S2-Preview: Scaling Scientific Agentic Foundation Models

What Changed

The landscape of scientific AI is shifting from static, text-based large language models toward agentic systems capable of interacting with complex, heterogeneous scientific environments. Intern-S2-Preview represents a significant evolution in this trajectory, moving beyond simple question-answering to support long-horizon task execution. Unlike previous iterations that relied on general-purpose training, this model series is built on a specialized pipeline that integrates multimodal pre-training with multi-task reinforcement learning (RL). The most notable architectural departure is the introduction of a Memory Decoder, which allows the system to specialize in specific scientific domains—such as biology or numerical forecasting—without the computational overhead of retraining the massive 397B parameter backbone. This modular approach addresses the critical challenge of maintaining general reasoning capabilities while achieving high-fidelity performance in niche scientific fields.

Technical Details

The Intern-S2-Preview architecture is designed to handle the multi-modal nature of scientific data, ranging from rendered documents to interleaved image-text corpora. The training pipeline is a multi-stage process that begins with scientific multimodal pre-training. Following this, the researchers apply a unified post-training phase that includes supervised fine-tuning and scalable multi-task reinforcement learning. A key component of this pipeline is the use of both black-box and white-box agentic RL, which allows the model to optimize its decision-making processes in scientific environments.

To ensure stability and efficiency during training, the team implemented several practical techniques:

• Partial Rollout with Off-Policy Correction: This allows the model to learn from incomplete task trajectories, improving sample efficiency.
• Adaptive Length Regularization: This manages the computational cost of long-horizon tasks by dynamically adjusting the sequence length.
• Online Speculative Decoding: This technique accelerates inference by using a smaller draft model to predict tokens, which are then verified by the 397B backbone.
• Trace-Aware Experience Assembly: This method allows the model to better organize and utilize past experiences for agentic tasks.

At the core of the architecture is the 397B parameter backbone, which has been extended to support time series modeling. This enables the model to perform numerical forecasting, a requirement for many scientific discovery tasks. Furthermore, the Memory Decoder acts as a separate, memory-augmented path. By keeping the 397B backbone frozen, developers can attach this 4B parameter Memory Decoder to adapt the model to new scientific domains rapidly, significantly reducing the cost and time required for specialization.

Benchmark Analysis

The integration of the Memory Decoder provides a measurable performance boost in specialized tasks. On the Biology-Instructions benchmark, the base Intern-S2-Preview model achieved a score of 56.92. With the addition of the Memory Decoder, this score increased to 60.32. This improvement demonstrates the efficacy of the memory-augmented path in enhancing domain-specific reasoning without modifying the frozen backbone.

Developer Implications

For developers and researchers, Intern-S2-Preview offers a pragmatic path to deploying high-performance scientific models. The primary implication is the decoupling of the massive 397B backbone from domain-specific adapters. This modularity means that organizations can maintain a single, powerful foundation model while deploying lightweight, specialized Memory Decoders for different scientific disciplines. This significantly lowers the barrier to entry for fine-tuning, as the frozen backbone remains untouched, preserving the model's general reasoning capabilities.

Additionally, the inclusion of online speculative decoding is a practical optimization for production environments. By offloading the initial token generation to a smaller draft model, developers can achieve lower latency, which is essential for agentic workflows where the model must interact with external tools in real-time. The focus on trace-aware experience assembly also suggests that developers should prioritize high-quality, structured data when fine-tuning for agentic tasks, as the model relies on these traces to navigate long-horizon problems.

Bottom Line

Intern-S2-Preview marks a shift toward modular, agentic AI in the scientific domain. By combining a massive 397B backbone with a 4B Memory Decoder, the researchers have created a system that balances general-purpose reasoning with specialized accuracy. The integration of advanced RL techniques and speculative decoding addresses the core bottlenecks of efficiency and long-horizon task execution, providing a robust framework for future scientific discovery tools.

Pneumetron