ATSplat: Optimizing Feed-Forward 3D Gaussian Splatting with Adaptive Token Expansion

What Changed In the evolution of 3D reconstruction, 3D Gaussian Splatting (3DGS) has emerged as a dominant paradigm, offering high-fidelity novel-view synthesis by optimizing primitives in 3D space. However, the transition from optimization-based 3DGS to feed-forward models has introduced a significant bottleneck: the reliance on pixel-aligned regression. Existing feed-forward methods typically regress Gaussians directly from input pixels and lift them along camera rays. This approach forces the number and placement of primitives to be tethered to the input image resolution and viewpoint, rather than the intrinsic complexity of the scene. Consequently, these models often produce dense, redundant Gaussian sets that consume unnecessary memory and compute. ATSplat addresses this limitation by introducing a sparse-to-adaptive framework that restores the scene-adaptive capacity allocation found in original 3DGS optimization. By utilizing Adaptive 3D Tokens, ATSplat decouples primitive placement from the input image grid, allowing the model to concentrate its representational capacity where it is most needed—in complex, under-reconstructed regions of the scene. ## Technical Details The architecture of ATSplat is built upon a hierarchical approach to scene representation. The process begins by lifting coarse patch-level depth and camera cues into a set of sparse 3D anchor tokens. These tokens serve as the structural scaffold for the scene, providing a compact representation that is independent of the input resolution. Each anchor token is subsequently regressed into local Gaussians, augmented with learnable 3D offsets. This design choice is critical; it allows the model to position primitives in 3D space based on scene geometry rather than being constrained by the pixel-to-ray mapping of the input camera. A core innovation in the framework is the Adaptive Token Expansion module. This module is responsible for the dynamic allocation of primitives. It predicts a token-level uncertainty score, which is supervised by rendering error maps generated during the training phase. When the model identifies a region with high uncertainty, the Adaptive Token Expansion module selectively expands the corresponding tokens using learnable expansion layers. This mechanism ensures that the Gaussian budget is spent efficiently, focusing on challenging areas like fine-grained textures or complex geometric occlusions, while keeping simpler regions sparse. By moving from a dense, grid-based approach to a sparse, token-based approach, ATSplat maintains a high-quality reconstruction with a fraction of the primitives required by previous feed-forward methods. ## Benchmark Analysis The performance gains achieved by ATSplat are significant, particularly in terms of model efficiency and rendering speed. Compared to existing dense feed-forward 3DGS methods, ATSplat demonstrates a 5.7x reduction in the total number of Gaussians required to represent a scene. This efficiency does not come at the cost of visual quality; the model achieves state-of-the-art rendering performance on standard datasets such as RealEstate10K and DL3DV. In terms of computational throughput, ATSplat is capable of rendering novel views at 1136 FPS at a resolution of 512x960. Furthermore, the reconstruction process is highly optimized, with the model completing the reconstruction of a scene from 12 input images in less than one second on a single commercial GPU. The final representation consists of approximately 311K Gaussians, a compact footprint that facilitates faster streaming and lower memory overhead. ## Developer Implications For developers working on real-time 3D applications, ATSplat represents a shift toward more sustainable and scalable neural rendering. The primary implication is the reduction in memory footprint and the decoupling of resolution from primitive count. This makes the framework significantly more viable for deployment on hardware with limited resources, such as mobile devices or edge computing platforms, where memory bandwidth and storage are often the primary constraints. Moreover, the sparse-to-adaptive nature of the tokens allows for easier integration into dynamic pipelines. Because the model is not strictly bound to the input grid, it is theoretically more robust to variations in input image resolution and camera parameters, simplifying the data preprocessing pipeline. The use of learnable 3D offsets also implies that the model can be fine-tuned more effectively on specific scene types, as the anchor tokens provide a flexible base that can adapt to different geometric distributions without requiring a complete retraining of the pixel-aligned regression layers. ## Bottom Line ATSplat effectively solves the redundancy problem inherent in feed-forward 3DGS by reintroducing adaptive capacity allocation. By shifting the focus from pixel-aligned regression to sparse 3D anchor tokens, the framework achieves a 5.7x reduction in Gaussian density while maintaining high-speed, high-fidelity rendering. For the AI/ML community, this work provides a robust template for balancing the efficiency of feed-forward architectures with the geometric precision of optimization-based 3DGS, setting a new benchmark for real-time 3D scene reconstruction.