DeepSeek Open-Sources DeepInfer, Its Production Inference Framework for V4.1-Flash

DeepSeek has open-sourced DeepInfer, the inference framework it says powers its own large-scale production services. The code is on GitHub under the Apache 2.0 license and had roughly 5,500 stars at the time of writing.

What DeepInfer Is

DeepInfer targets DeepSeek-V4.1-Flash and ships with support for both Nvidia SM100 GPUs and Huawei Ascend 950 NPUs.

The repository includes custom kernels such as FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect, along with TileLang. For parallelism, it combines attention data parallelism with balanced context parallelism, plus MoE expert parallelism with EPLB.

The README is open about the project's lineage. DeepInfer was "originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM."

Features

The framework lists DSpark speculative decoding, batch-invariant inference, deterministic sampling, and structured output through regex, JSON and EBNF. DeepSeek describes it as production software rather than a research prototype.

Analysis

Releasing a production serving stack under a permissive license gives outside teams access to the same code DeepSeek says it runs itself. Support for both Nvidia and Huawei hardware also gives teams a way to serve V4.1-Flash without depending only on Nvidia GPUs. How DeepInfer compares with established open frameworks in real workloads will need independent benchmarks, and those results should be checked before anyone treats the performance claims as settled.

Pneumetron