Learning notes and experiments for understanding modern Machine Learning System.
Currently focusing on LLM serving system and inference optimization.
- Introduction to LLM Inference Part 1
- ORCA paper review
- PagedAttention paper review
- Softmax: From Naive to Blocked Softmax
- Rejection Sampling in Speculative Decoding
- Triton Notes
- tinyorca deep dive
- Inside nano-vLLM
- How Multiprocess Serving Works in vLLM
- Comparing Round-Robin Routing vs KV-Aware Routing on Nemotron-3-Nano
- NVIDIA GPU Architecture: From GPC to SM
- GPU Memory Hierarchy in CUDA
- CUDA Intro
- [KR] NPU Memory Architecture and Data Reuse
- Triton notes and kernels: GPU kernel experiments, notes, and interactive diagrams
- microengine: a minimal serving engine
- tinyorca: a minimal implementation of ORCA
- nano-vllm-bench: a performance comparison between nano-vLLM and nano-vLLM-v1
- tiny-speculators: a from-scratch implementation of speculative decoding model training.
- Dynamo Deployment Guide: deployed and benchmarked models including Nemotron Nano and Qwen3 on up to 16× H100 GPUs with experiments spanning P/D disaggregation, KV-aware routing, KV offloading, and replayed multi-turn coding-agent workloads.
- MLA Lab: roofline analysis and H100 experiments comparing MLA weight absorption methods