English · 简体中文

Learn DeepInfer

Understand DeepInfer through architecture, animated mechanisms, and guided source reading.

Start learning ↗　·　Architecture map ↗　·　Source map

---

How does a request enter the engine? Who chooses the next round of work? How is state reused, and how do tokens reach experts? Learn DeepInfer connects these questions across 12 bilingual lessons, with visual explanations and pinned source references.

A visual field guide to the DeepInfer framework: architecture, mechanisms, conceptual equations, and source reading. Interactive examples use preset states to explain each mechanism.

Start with the official design · Eight questions about stages, state reuse, and scheduling → Read alongside DeepSeek's official article, with clear boundaries between design motivations, public source, and internal components.

Open the website's architecture map from the image to connect these questions to the course.

Start with the system map

Identify four responsibilities, then connect them by following a request. This is the course map; the actual request chain appears below.

One mechanism at a time

Responsibility	Lessons	The question to answer
01 · Engine orchestration	s01 Request lifecycle · s02 Execution boundaries · s04 Sampling & state	Who accepts requests, runs a round, and updates state?
02 · State management	s03 KV cache · s07 Block mapping · s08 Prefix reuse	How does logical context map to reusable storage?
03 · Scheduling & coordination	s05 Batching · s06 Continuous scheduling · s09 DSpark · s10 Execution overlap	What runs this round, and which stages can overlap?
04 · Experts & balance	s11 Expert routing · s12 EPLB	Where do tokens go, and where do expert replicas belong?

Trace a request through the source

LLM.generate creates and enqueues requests. LLMEngine.step coordinates rounds; the Scheduler selects work; Worker and ModelRunner organize execution; postprocessing advances request state. A step may cover multiple requests, prompt chunks, or draft verification.

Source entry	What to follow
entrypoints/llm.py	Request admission, output collection, and return
engine/llm_engine.py	Round orchestration and engine state
csrc/engine/scheduler.hpp	Request selection, budgets, and block planning
worker/worker.py · model_runner.py	Execution coordination, sampling, and deferred processing
worker/cache_engine.py	Cache-state organization and storage
model_executor/eplb/policy.py	Expert replica allocation and placement

Source references are pinned to DeepInfer@8b16149 · Open the complete source map →

See the state change

Observe	Interact	Connect
Batch admission, cache reuse, draft verification, expert dispatch	Step, play, or scrub; transitions connect consecutive states	Chapter comparison, bilingual explanations, pinned source entries

Explore continuous batching ↗　·　Explore expert routing ↗　·　Compare two chapters ↗

The website includes light and dark themes, course search, mobile layouts, and reading progress stored in your browser.

Expand the website preview

Put the relationships into equations

Expand equations and definitions

The notation describes responsibilities at a conceptual level. Read the source for concrete admission conditions, cache formats, and routing rules.

Round budget — $\mathcal{S}_t$ is the selected request set; $q_i$ is token work assigned to request $i$, and $B_t$ is the round budget.

$$ \sum_{i\in\mathcal{S}_t}q_i\leq B_t $$

Block addressing — $t$ is a zero-based position, $P$ is block size, and $\mathrm{table}[b]$ resolves the physical block.

$$ b=\left\lfloor t/P\right\rfloor,\quad o=t\bmod P,\quad p=\mathrm{table}[b] $$

Prefix identity — A parent prefix, token block, and relevant context identify reusable state. $H$ denotes a conceptual identity function.

$$ K=H!\left(K_{\mathrm{parent}},;\mathrm{tokens},;\mathrm{context}\right) $$

Expert combination — $E_e$ is an expert function, $w_e$ its combine weight, and $\mathcal{S}(x)$ the routed expert set. Shared experts and routing-specific details are omitted.

$$ y=\sum_{e\in\mathcal{S}(x)}w_eE_e(x) $$

Read the illustrated architecture guide → · Continue with paged state, continuous batching, and MoE routing.

Develop the website

With Node.js 22, run from the repository root:

git clone https://github.com/learn-deepinfer/learn-deepinfer.github.io.git
cd learn-deepinfer.github.io/web
npm ci
npm run dev

Open localhost:5178/en/ · npm run build builds 32 Chinese and English route documents.

web/src/data/          Bilingual lessons and DeepInfer source references
web/src/components/    Interactive mechanisms and state transitions
docs/                  Architecture guides, source map, deployment notes
docs/diagrams/         Editable frontend diagram sources
docs/assets/diagrams/  Rendered figures for the README and guides

Deployment notes · Render the diagrams

Help make it clearer

Contributions are welcome: clearer diagrams, source-map corrections, translations, and more readable state transitions. Link content changes to upstream source; run npm run build for website changes.

Open an issue · Send a pull request

Credits & license

An independent community learning project about deepseek-ai/DeepInfer. The progressive course structure and documentation design draw inspiration from shareAI-lab/learn-claude-code.

The independent design notes draw on DeepSeek's official article 《DeepSeek V4.1 Flash 大规模推理部署实践》.

Apache-2.0 · Third-party notices

---

Understand each mechanism in the context of the whole system.

Start reading ↗