SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

What Changed The field of video diffusion transformers (DiTs) has long been constrained by the quadratic complexity of standard softmax attention, which limits the ability to generate long, high-resolution sequences without massive computational overhead. SANA-Video 2.0 addresses this by introducing a hybrid architecture that combines the efficiency of linear attention with the expressive power of softmax attention. By training from scratch rather than linearizing existing models, the researchers have created a system that scales to 5B and 14B parameter counts while maintaining high-quality outputs at 720p resolution on a single GPU. ## Technical Details The core innovation in SANA-Video 2.0 is the Hybrid Linear-Softmax Attention mechanism. To mitigate the O(N^2) complexity of traditional attention, the model employs gated linear attention for the majority of token mixing. To ensure that the model does not lose the full-rank token interactions necessary for high-fidelity video, it integrates periodic gated-softmax anchors at a 3:1 ratio. This configuration was determined through reduced-resolution proxy studies to be the optimal trade-off between quality and computational efficiency. Furthermore, the architecture introduces Block Attention Residuals (AttnRes). This mechanism routes completed block summaries into subsequent linear layers, which facilitates anchor-feature reuse and increases the effective rank of deep layers by approximately 12%. The model is further optimized through the Sol-Engine, a full-stack optimization suite that includes kernel fusion, caching, and sparse attention. This stack provides a 3.58x speedup on top of the architectural improvements. ## Benchmark Analysis The performance gains of SANA-Video 2.0 are significant when compared to traditional full-softmax video DiTs. With 40-step sampling, the model achieves a VBench score of 84.30. In terms of latency, it generates 480p video in 13.2 seconds on a single H100 GPU. At 720p resolution, the compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline. When utilizing the full Sol-Engine optimization, the 5B pipeline reaches 720p/5s in 13.06 seconds, representing a 120x speedup compared to the Wan 2.2-A 14B model. ## Developer Implications For developers, SANA-Video 2.0 represents a shift toward more hardware-accessible video generation. The ability to run high-quality 720p video generation on a single H100 GPU reduces the barrier to entry for production-scale deployment. The from-scratch training approach ensures that the hybrid attention mechanism is deeply integrated into the model weights, rather than being a post-hoc approximation. The reliance on the Sol-Engine suggests that future implementations will benefit heavily from custom kernel development and sparse attention strategies, which are becoming standard requirements for high-performance generative AI pipelines. ## Bottom Line SANA-Video 2.0 demonstrates that quadratic attention is not a strict requirement for high-quality video generation. By strategically mixing linear and softmax attention and utilizing block-level residuals, the model achieves competitive VBench scores while drastically lowering latency. This architecture provides a scalable path forward for long-form, high-resolution video generation in resource-constrained environments.

Pneumetron