DeepSeek open-sourced DeepInfer on October 20, 2026 — the inference framework it says powers its own large-scale production services — under the Apache 2.0 license. Why it matters: inference is where AI companies actually pay per answer, and DeepSeek is now showing the serving stack behind its low prices instead of keeping it private.
October 20, 2026
What happened?
DeepSeek published DeepInfer on GitHub on October 20, describing it as a high-performance inference framework for running DeepSeek models. The first release is built around DeepSeek-V4.1-Flash and targets offline, high-throughput inference. A companion tool, deepseek-recipe, for token encoding and decoding was released alongside it.
The release is a reversal of an earlier position. Reporting on the launch notes that in April 2025 DeepSeek said it had decided not to release its full inference engine, partly because the code was a heavily customised fork tied to its internal systems. DeepInfer’s documentation still credits the open-source vLLM project as its main base, with ideas drawn from SGLang and NVIDIA’s TensorRT-LLM.
What is inside DeepInfer?
The framework bundles custom kernels, distributed parallelism — including mixture-of-experts expert parallelism across chips — speculative decoding, and structured-output support for formats such as JSON. It has been optimised for two hardware families: NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. Putting a Huawei chip on equal footing with NVIDIA hardware in an official DeepSeek release is notable for teams weighing non-NVIDIA infrastructure.
What performance does DeepSeek claim?
The figures attached to V4.1-Flash are company-reported and have not been independently tested: 155 trillion tokens per day per cluster, a 96% input cache hit rate, average time-to-first-token of around 400 milliseconds, and generation at 230 tokens per second, rising to up to 300 tokens per second for coding tasks. Read those as DeepSeek’s own production numbers, not verified benchmarks.
What is missing from this release?
This is not a complete, turnkey serving service. Coverage of the README notes that a full production setup still needs components this release leaves out, including a KV cache system, prefill/decode disaggregation, and load balancing across replicas. Teams will need to supply that plumbing themselves, and independent, reproducible benchmarks are still needed to judge how large the practical advantage really is.
Why it matters
DeepSeek built its reputation on doing AI cheaply, and serving efficiency is a big part of how. By open-sourcing the engine under Apache 2.0 — a license that allows use, modification and commercial deployment — it gives developers and enterprises a starting point for running DeepSeek models on their own hardware, on either NVIDIA or Huawei chips. For everyone else, it raises the pressure on closed serving stacks: if the techniques behind low-cost inference are public, rivals have to justify their margins some other way. The sensible stance, as always, is to wait for third-party tests before treating the headline throughput numbers as settled.
Frequently asked questions
What is DeepInfer?
DeepSeek’s inference framework — the software that serves a trained model to users — now open-sourced under Apache 2.0. It currently supports DeepSeek-V4.1-Flash for offline inference.
Which chips does DeepInfer support?
NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
Can I run a full production service with DeepInfer alone?
Not yet. The release omits several service components, including a KV cache system and load balancing, which operators must provide themselves.
Sources: Crypto Briefing, RuntimeWire, Tesla News Wire (all reporting DeepSeek’s October 20, 2026 release)