Xiaomi-Robotics-U0: A 38-Billion-Parameter Model for Unified Embodied Synthesis

What Changed

Traditional foundation models for image and video generation, while powerful in generalization and controllability, face limitations when directly applied to embodied scenarios. These limitations stem from the need for multi-view consistency, geometric coherence, and specific robot embodiment constraints. Existing approaches often adapt these foundation models using limited robot-specific data, which can inadvertently compromise the extensive visual knowledge gained during large-scale pre-training.

Xiaomi-Robotics-U0 introduces a significant shift by treating embodied generation as a direct extension of foundation image and video generation. This 38-billion-parameter multimodal autoregressive model is designed for unified embodied synthesis. Instead of merely adapting existing models, Xiaomi-Robotics-U0 jointly optimizes a suite of generative tasks: text-to-image generation, image editing, embodied scene generation, embodied transfer, and embodied video generation. This unified framework is engineered to maintain the generalization strengths of the underlying pre-trained world foundation model while effectively adapting it to the complexities of embodied settings. It is notably the first model to support high-quality multi-view scene generation across diverse robot embodiments and to offer structured, controllable embodied transfer for fine-grained editing, all while preserving multi-view consistency and interaction dynamics.

Technical Details

Xiaomi-Robotics-U0 operates as a 38-billion-parameter multimodal autoregressive model. Its core technical innovation lies in its unified framework, which integrates multiple generative tasks within a single optimization process. This contrasts with prior methods that often involve fine-tuning or adapting pre-trained models for specific embodied tasks, potentially leading to a loss of general visual knowledge.

The model's architecture is designed to handle the unique requirements of embodied AI. This includes ensuring multi-view consistency, where generated scenes remain coherent from different perspectives, and geometric coherence, which is crucial for realistic physical interactions. Furthermore, it incorporates robot embodiment constraints, meaning the generated content is compatible with the physical and operational limitations of various robot forms.

Key capabilities of Xiaomi-Robotics-U0 include:

• Text-to-Image Generation: Generating static images from textual descriptions.
• Image Editing: Modifying existing images based on prompts or conditions.
• Embodied Scene Generation: Creating complex, multi-view scenes that are consistent and geometrically sound, suitable for robot interaction.
• Embodied Transfer: Enabling structured and controllable transfer of elements or styles within embodied scenes, allowing for fine-grained editing while maintaining consistency and interaction dynamics.
• Embodied Video Generation: Producing dynamic video sequences that depict robot actions and environmental changes in an embodied context.

The joint optimization across these diverse tasks allows Xiaomi-Robotics-U0 to leverage the broad visual understanding of a world foundation model while simultaneously learning the specific nuances required for embodied intelligence. This approach aims to prevent the degradation of visual knowledge often observed when adapting general-purpose models to highly specialized domains.

Benchmark Analysis

Xiaomi-Robotics-U0 has demonstrated state-of-the-art performance across various single-step and sequential generation tasks. In human evaluations for embodied scene generation and transfer, it outperformed GPT-Image-2.0. The model also achieved the top rank on World Arena for embodied video generation.

For challenging real-world manipulation tasks, Xiaomi-Robotics-U0 improved the out-of-distribution success rate of pi_0.5 from 36.9% to 63.2%. These results indicate that foundation world models can function effectively as both embodied world models and scalable data engines for embodied intelligence.

Developer Implications

For developers working in robotics, embodied AI, and generative models, Xiaomi-Robotics-U0 offers a powerful new tool. The model's ability to perform high-quality multi-view scene generation across multiple robot embodiments simplifies the creation of diverse and realistic simulation environments. This can accelerate the development and testing of robot control policies and perception systems without the need for extensive real-world data collection, which is often costly and time-consuming.

The introduction of structured, controllable embodied transfer provides developers with fine-grained control over scene editing. This capability is crucial for iterating on robot task designs, adjusting environmental parameters, or simulating specific interaction dynamics with precision. The preservation of multi-view consistency and interaction dynamics during editing ensures that generated content remains physically plausible and useful for training or evaluation.

Furthermore, the model's strong performance in embodied video generation can aid in creating synthetic datasets for training vision-based robot systems, enhancing data diversity and robustness. The improved out-of-distribution success rate on manipulation tasks suggests that models like Xiaomi-Robotics-U0 can contribute to more generalized and robust robot behaviors, potentially reducing the sim-to-real gap.

Developers can access the code and checkpoints at the provided link (robotics.xiaomi.com/xiaomi-robotics-u0.html), enabling them to integrate and experiment with this model in their own projects. This accessibility fosters further research and application development in embodied AI.

Bottom Line

Xiaomi-Robotics-U0 represents a significant advancement in the field of embodied AI and generative modeling. By unifying various generative tasks within a 38-billion-parameter multimodal autoregressive framework, it effectively bridges the gap between general-purpose foundation models and the specific requirements of embodied scenarios. The model's ability to maintain the generalization power of pre-trained world models while adapting to multi-view consistency, geometric coherence, and robot embodiment constraints is a key differentiator.

Its strong benchmark performance, particularly in outperforming GPT-Image-2.0 in human evaluations for embodied scene generation and transfer, and its top ranking in embodied video generation on World Arena, underscore its capabilities. The substantial improvement in out-of-distribution success rates for real-world manipulation tasks highlights its potential to drive more robust and adaptable robotic systems.

Ultimately, Xiaomi-Robotics-U0 demonstrates that large-scale foundation world models can serve a dual role: as sophisticated embodied world models for simulating complex interactions and as scalable data engines for generating high-quality, diverse data essential for advancing embodied intelligence. This work sets a new precedent for how foundation models can be extended and applied to create more intelligent and capable robotic agents.

Pneumetron