Pinned Loading

1. JustVugg/colibri JustVugg/colibri Public

   A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.

   C 48.3k 5.6k

2. kvcache-ai/Mooncake kvcache-ai/Mooncake Public

   Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

   C++ 7k 1.4k

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 96.5k 24.5k