# DeepSeek DeepInfer explained: what the open-source release means

> DeepSeek open-sourced DeepInfer, its production inference framework, under Apache 2.0 on 20 October.

*DeepSeek has published the inference software behind its own services under the Apache 2.0 licence, and here is what that does and doesn't tell you.*

By Bez · SuggestedTech
Canonical: https://suggestedtech.com/news/deepseek-deepinfer-explained-what-the-open-source-release-means

DeepSeek has released the software it uses to run its AI models for the public. The tool is called DeepInfer, and anyone can now download, use and change it for free.

## What happened

On 20 October, DeepSeek published DeepInfer on [GitHub](https://github.com/deepseek-ai/DeepInfer) under the Apache 2.0 licence, a permissive open-source licence. DeepInfer is an inference framework. Inference is the stage where a trained model answers your questions, as opposed to training, when it learns.

The project's README describes it as "the high-performance inference framework that powers DeepSeek's large-scale production services." It targets DeepSeek-V4.1-Flash and supports two kinds of chips: Nvidia SM100 GPUs and Huawei Ascend 950 NPUs (neural processing units, chips built for AI work).

The README also says DeepInfer was "originally built primarily on vLLM", another inference framework, with ideas from SGLang and TensorRT-LLM.

The release includes several low-level programs, called kernels, that do the heavy calculations. They are FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect, along with TileLang. Features listed include:

- Attention data parallelism with balanced context parallelism, which spreads work across many chips.
- Mixture-of-experts (MoE) expert parallelism with EPLB, for models that route each request to specialised sub-networks.
- Draft-token acceleration with confidence-scheduled verification, which DeepSeek calls DSpark.
- An asynchronous, zero-overhead batch scheduler.
- A batch-invariant mode.
- Constrained generation, which forces output to follow a regex, a JSON schema or an EBNF grammar.

DeepSeek claims the system handles about 155 trillion tokens per day per cluster, with a 96% input cache hit rate. Tokens are the small chunks of text a model reads and writes. That figure has not been independently verified.

## What it means for you

If you only chat with AI tools, you won't see a change directly. The notes don't say DeepSeek's apps or prices are changing.

The release matters most to developers and companies that run models themselves. They can now inspect and reuse the same serving software DeepSeek says it uses in production, including on Huawei chips.

The release comes after DeepSeek said last week that it plans to pre-train on Huawei Ascend 950 chips by the end of 2026.

Some details are unsettled. Reports differ on how many stars the repository has: one says 5.5k, another more than 8,000 within hours. One report links the launch to a US Department of Homeland Security ban on DeepSeek in federal agencies the day before. That link is unverified.

## What happens next

DeepSeek has not said what it plans to add to DeepInfer. The repository is the place to check for updates and for current figures on its popularity.

## Key takeaways

- DeepSeek has made the software that runs its own services free to use and modify.
- It targets DeepSeek-V4.1-Flash and runs on Nvidia and Huawei chips.
- The headline speed figures are DeepSeek's own claims and are not independently verified.

## Sources

- [DeepInfer repository](https://github.com/deepseek-ai/DeepInfer) — GitHub (deepseek-ai), 2026-10-20