DeepSeek Open-Sources DeepInfer, the Engine Behind Its Models

DeepSeek just open-sourced the engine that runs its models

On October 17, 2026, DeepSeek shipped three things in one weekend: an updated checkpoint called DeepSeek-V4.1-Flash-1017, and two new web APIs (Search and Fetch). Three days later, on October 20, the company published DeepInfer, the inference framework that powers its own large-scale production services, on GitHub under Apache License 2.0.

The model release matters to anyone who calls an API. The engine release matters to anyone who pays the compute bill. Here is what is actually in it, what is missing, and what it costs to use the rest.

Why this is a reversal

In April 2025, DeepSeek wrote a short note on GitHub explaining that it had considered open-sourcing its inference engine and decided not to. The reasons were practical: the code was an old fork of vLLM, it was tightly coupled to internal infrastructure, and a small research team had no bandwidth to maintain a large open-source project.

The new README acknowledges that history directly. It states that DeepInfer "was originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM." Roughly eighteen months of cleanup turned an internal fork into something publishable. That is the real story here — not a gift, but a refactor that finally crossed the line.

What ships, and what does not

From the repository README, the feature list is concrete:

• Optimized kernels: FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, plus custom TileLang kernels
• Parallelism: attention DP and balanced CP, MoE expert parallelism with EPLB load balancing
• DSpark speculative decoding with confidence-scheduled verification (default max draft length 5, via DEEPINFER_USE_NEXTN_DECODE)
• Decoder SWA bounded replay, which replays only a short suffix during prefill (DEEPINFER_YOCO_PREFILL_KEEP_TOKENS=140 by default)
• Asynchronous scheduling that overlaps CPU batch construction with device execution, on by default
• Batch-invariant inference and deterministic sampling — NVIDIA hardware only
• Structured outputs constrained by regex, JSON Schema, or EBNF grammar

The README is equally explicit about the gap. Three things required for a complete inference service are not included: a KV cache system, VE/prefill/decode disaggregation, and load balancing across data-parallel replicas for both prefill and decode.

Read that list carefully. What you get is offline inference on one machine or within a single scale-up domain. What you do not get is a multi-tenant API server you can point customers at on day one.

The hardware requirement is the real gate

Prerequisites: Linux, GCC 13.0 or newer, Python 3.12, CUDA 13 on NVIDIA, CANN 9.2 on Ascend. Supported accelerators: NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, selected at build time with DEEPINFER_ACCELERATOR.

Exactly one model is supported: DeepSeek-V4.1-Flash, whose default checkpoint is fetched into the Hugging Face cache on first run.

Then there is the size of that model. Its model card lists a 552B backbone plus 196B of "Engram" conditional memory, a 763B-parameter checkpoint spread across 48 safetensors files totaling 510.3 GB. This is not a laptop demo. Open weights plus an open engine still means a cluster.

Benchmarks you should read with both eyes open

DeepInfer's performance charts were measured on 4 SM100 GPUs for prefill and 64 SM100 GPUs for decode, each within one scale-up domain. Bloomberg's Saritha Rai noted that no independent benchmarking of the gains exists yet. Treat the throughput curves as vendor claims.

For the model itself, the published comparison table (harness noted in the model card: Claude Code for Terminal-Bench 4.0, mini-SWE-V2 for DeepSWE v1.1) contains both wins and losses.

• DeepSWE v1.1: 77.0 (was 74.2; Claude Opus 5 is 74.0)
• Terminal-Bench 4.0: 42.3 (was 31.2; Opus 5 is 51.8)
• Automation-Bench v1.0.6: 60.5 (was 54.8; Opus 5 is 50.3)
• Agents' Last Exam: 35.4 (was 31.8; Opus 5 is 28.6)
• HLE without tools: 42.6 (was 36.8; Opus 5 is 56.3)
• ProgramBench: 29.1 (was 20.3; Opus 5 is 37.0)

The pattern: DeepSeek wins on agentic execution benchmarks where long-horizon tool use dominates, and loses badly on hard knowledge-recall questions. That is a useful signal for choosing workloads, not a headline.

Pricing: unchanged, but read the clock

The API model name is still deepseek-flash and prices did not move. Per million tokens, from DeepSeek's pricing page:

• Input, cache miss: $0.15 off-peak / $0.30 peak
• Input, cache hit: $0.003 off-peak / $0.006 peak
• Output: $0.60 off-peak / $1.20 peak
• Context: 1M tokens, max output 384K tokens

Peak hours are 01:00–04:00 and 06:00–10:00 UTC Monday through Friday, excluding Chinese public holidays. Everything else, weekends included, is off-peak at half price. If your pipeline can schedule around those windows, your effective rate is simply the lower number.

The new Search and Fetch APIs are billed per successful request: $0.75 per 1,000 calls each, not per token. Failed requests, empty result sets, and timeouts are not billed. Account-level limits are 100 QPS for Search and 25 QPS for Fetch, raised from 5 QPS each at launch.

One deadline to flag: server-side tool calls through DeepSeek Harness, Claude Code, and Codex are free until October 31, 2026 at 16:00 UTC. After that, they bill per request.

For context, Exa lists $7 per 1,000 search requests and Perplexity's Search API lists $5 per 1,000 (or $1 in fast mode). At $0.75, DeepSeek is undercutting the field by roughly an order of magnitude on list price. Whether result quality holds up is a separate question that only your own evaluation can answer — DeepSeek's claim that it filters SEO spam, GEO manipulation, and low-quality AI content is a vendor statement.

What to do this week

Three concrete actions, depending on who you are.

If you call the DeepSeek API: nothing breaks. The model name and price are unchanged; you get faster completion and shorter outputs for free. If you rely on long-form generation, tighten your instructions, because shorter outputs are now the stated behavior. If you use Search or Fetch in an agent: model the bill before the free window closes on October 31. Ten thousand searches a day runs about $225 a month at list price. If you were hoping to self-host: check the hardware first. Without SM100 or Ascend 950 silicon, and without a KV cache layer and a load balancer you build yourself, DeepInfer is not an option yet.

The honest framing is this: open-sourcing the engine does not make frontier inference free. It makes the vendor's economics auditable. Given that every number in this release is company-reported, auditability is worth more than the license.