Repository navigation

v0.2.11: Driver API, Dual CUDA Build, RTX 5090 Support - #94

Merged

m96-chan merged 45 commits into

mainfrom

feature/v0.2.11

Dec 22, 2025

Merged

v0.2.11: Driver API, Dual CUDA Build, RTX 5090 Support#94
m96-chan merged 45 commits into
mainfrom
feature/v0.2.11

Commits on Dec 18, 2025
- chore: bump version to 0.2.11
  
  m96-chan
  and
  claude
  committed
  b0b42aa
- feat(v0.2.11): add CUDA Events API and zero-copy view optimization
  
  m96-chan
  and
  claude
  committed
  c81dfea
- feat(v0.2.11): add batch decode support (seq_len > 1)
  
  m96-chan
  and
  claude
  committed
  59dfa8d
- perf(v0.2.11): dispatch to optimized path for M=1 in batch decode
  
  m96-chan
  and
  claude
  committed
  ac35cc5
- bench(v0.2.11): add E2E batch decode benchmarks
  
  m96-chan
  and
  claude
  committed
  3740647
- fix(model): support explicit head_dim for Qwen3-0.6B and similar models
  
  m96-chan
  and
  claude
  committed
  aed5040
- bench: add speculative decoding potential analysis
  
  m96-chan
  and
  claude
  committed
  b342e49
- feat(v0.2.11): add self-speculative decoding verification framework
  
  m96-chan
  and
  claude
  committed
  7e49193
- feat(v0.2.11): add Jacobi decoding for parallel iterative generation
  
  m96-chan
  and
  claude
  committed
  08ac023
- feat(v0.2.11): add GPU-side Lookahead KV Cache for Jacobi decoding
  
  m96-chan
  and
  claude
  committed
  7c7a6e8
Commits on Dec 19, 2025
- feat(v0.2.11): add GPU-side Lookahead for Self-Speculative decoding
  
  m96-chan
  and
  claude
  committed
  619e6b7
- fix(cuda-graph): add stream parameter to RoPE kernels for graph capture
  
  m96-chan
  and
  claude
  committed
  4ec4e7f
- docs: update CLAUDE.md for v0.2.11 architecture
  
  m96-chan
  and
  claude
  committed
  0bed2e0
- docs: add batch decode benchmark results to CLAUDE.md
  
  m96-chan
  and
  claude
  committed
  81c20d5
- feat(llm): add batch decode zero-allocation path
  
  m96-chan
  and
  claude
  committed
  fb4f250
- fix(prefill): fix RoPE not applied due to DataType comparison bug
  
  m96-chan
  and
  claude
  committed
  d1af682
- fix(chat_cli): add UTF-8 encoding for Windows console
  
  m96-chan
  and
  claude
  committed
  ba1f7f9
Commits on Dec 20, 2025
- feat(llm): add bf16 direct loading and O(1) streaming decoder
  
  m96-chan
  and
  claude
  committed
  0cd987f
- refactor(llm): split model.py and nn_kernels.cuh into modular files
  
  m96-chan
  and
  claude
  committed
  fe326e2
- refactor(llm): add decode strategy module
  
  m96-chan
  and
  claude
  committed
  b571058
- chore(llm): add deprecation warnings for CUDA Graph methods
  
  m96-chan
  and
  claude
  committed
  ff03055
- fix(llm): use forward_fixed_cache in DecodeM1.step for bfloat16 compat
  
  m96-chan
  and
  claude
  committed
  c6dfabb
- bench(llm): add all decode strategies benchmark
  
  m96-chan
  and
  claude
  committed
  f44cf85
- fix(llm): add bfloat16 RoPE support for batch decode
  
  m96-chan
  and
  claude
  committed
  704eabb
- bench(llm): use batch_size=8 for TensorCore efficiency
  
  m96-chan
  and
  claude
  committed
  de40947
- docs: add decode strategy benchmark baseline to CLAUDE.md
  
  m96-chan
  and
  claude
  committed
  ee4e05d
- feat(chat_cli): fix UTF-8 streaming + add CUDA Graph support
  
  m96-chan
  and
  claude
  committed
  f561f6c
- fix(cuda-graph): correct warmup path and bfloat16 RoPE handling
  
  m96-chan
  and
  claude
  committed
  dad8c91
- feat(ops): add GPU dtype cast kernels for float32↔bfloat16/float16
  
  m96-chan
  and
  claude
  committed
  f313291
- feat(ops): add rope_inplace_f32table kernel for bf16/f16 with f32 tables
  
  m96-chan
  and
  claude
  committed
  9eb94e2
Commits on Dec 21, 2025
- fix(cuda-graph): add capture stream to bias_add_inplace for Qwen2.5
  
  m96-chan
  and
  claude
  committed
  d05f58b
- chore: comment out CUDA Graph debug code
  
  m96-chan
  and
  claude
  committed
  34bb43b
Commits on Dec 22, 2025
- feat: add RTX 5090 (SM 120) support
  
  m96-chan
  and
  claude
  committed
  c999de9
- refactor: convert cuda_runtime to Driver API + dual CUDA packaging
  
  m96-chan
  and
  claude
  committed
  c6334a1
- ci(release): add dual CUDA build support (12.9 + 13.1)
  
  m96-chan
  and
  claude
  committed
  c97238f
- fix(lint): add per-file-ignores for bench/test/demo files
  
  m96-chan
  and
  claude
  committed
  3aca76c
- fix(mypy): add type hints for dynamically added attributes
  
  m96-chan
  and
  claude
  committed
  84e25a5
- fix(lint): remove unused os import in basic.py
  
  m96-chan
  and
  claude
  committed
  083a64c
- fix(mypy): add type annotation for _batch_token_ids_np
  
  m96-chan
  and
  claude
  committed
  35e81fe
- fix(memory): add async memcpy and pinned memory functions
  
  m96-chan
  and
  claude
  committed
  385adef
- fix(llm): export QWEN2_SPEC and update tests
  
  m96-chan
  and
  claude
  committed
  7c9ca30
- feat(cuda-graph): add volatile reads for graph replay and refactor DecodeM1
  
  m96-chan
  and
  claude
  committed
  ace0872
- docs: update README for v0.2.11
  
  m96-chan
  and
  claude
  committed
  d984af4
- refactor(model): remove deprecated decode methods (1086 lines)
  
  m96-chan
  and
  claude
  committed
  47f6849
- refactor(ops): split basic.py into submodules aligned with C++ structure
  
  m96-chan
  and
  claude
  committed
  bd641d9

Commits

Commits on Dec 18, 2025

Commits on Dec 19, 2025

Commits on Dec 20, 2025

Commits on Dec 21, 2025

Commits on Dec 22, 2025