Pinned Loading

1. TensorRT-LLM TensorRT-LLM Public

   Forked from NVIDIA/TensorRT-LLM

   TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

   Python

2. cutlass cutlass Public

   Forked from NVIDIA/cutlass

   CUDA Templates and Python DSLs for High-Performance Linear Algebra

   C++

3. mini-sglang mini-sglang Public

   Forked from sgl-project/mini-sglang

   A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.

   Python

4. FreeToken FreeToken Public

   Forked from FlashML-org/FreeToken

   FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.

   Python

5. cuda-kernels cuda-kernels Public

   A collection of high-performance CUDA kernels and experiments for learning and optimizing GPU compute primitives.

   Cuda 1

6. matmul-cpu matmul-cpu Public

   High-performance CPU GEMM kernels (C = A·Bᵀ + C) optimized for LLM inference, featuring AVX2/AVX-512 SIMD and multi-threading. Benchmarked against OpenBLAS.

   C++ 1