Compression-aware Qwen3-VL inference engine with torch.compile, CUDA Graph, scaled-FP8 KV, visual compaction, and multimodal prefix caching.
-
Updated
Sep 8, 2026 - Python
#
multimodal-inference
Here are 2 public repositories matching this topic...
A lightweight, config-driven framework for unified vision-language model inference across local and cloud backends, built on LiteLLM and Transformers.
transformers gemini openai vlm huggingface inference-framework llm anthropic vision-language-model vllm ollama litellm multimodal-ai mlx-vlm multimodal-inference
-
Updated
Oct 5, 2026 - Python
Add this topic to your repo
To associate your repository with the multimodal-inference topic, visit your repo's landing page and select "manage topics."