[bot] Instrument llama-cpp-python (114,222 weekly downloads)

[braintrust-bot]

<!-- library-gap-audit: llama-cpp-python -->

Summary

llama-cpp-python is the widely-used Python binding for llama.cpp, providing a local/offline LLM inference execution surface with an OpenAI-compatible API shape. This repository has zero instrumentation for it — no integration directory, wrapper, nox session, or matrix entry.

• Current version 0.3.35 (released 2026-08-17)
• Repo-wide case-insensitive grep for llama_cpp/llama-cpp under py/ returns zero real matches (only incidental hits in unrelated cassette fixture text)
• Not listed in py/pyproject.toml [tool.braintrust.matrix], not a noxfile.py session, not in py/src/braintrust/integrations/ or py/src/braintrust/wrappers/

This is a distinct execution surface from already-instrumented libraries: it is not transformers (Hugging Face's own local pipeline, already instrumented via integrations/transformers/), not ollama (a separate server/CLI-based runtime, tracked separately as an open gap), and not vllm (a separate offline-inference engine, also tracked separately as an open gap). llama-cpp-python embeds the llama.cpp C++ inference engine directly in-process via ctypes bindings and exposes its own distinct Llama class API.

What is missing

The Llama class exposes the core execution entrypoints:

• Llama.__call__ / Llama.create_completion — text completion, OpenAI-compatible response schema, with streaming support (stream=True)
• Llama.create_chat_completion (and create_chat_completion_openai_v1, which returns typed OpenAI SDK response objects) — chat completion, including function/tool calling and JSON/JSON-schema-constrained output, with streaming support
• Llama.create_embedding / Llama.embed — embedding execution, requires embedding=True at model load

All of these are the primary way applications call locally-loaded GGUF models through this library; none currently produce a Braintrust span.

Weekly downloads

Weekly downloads: 114,222 (as of 2026-09-21; https://pypistats.org/api/packages/llama-cpp-python/recent)

Braintrust docs status

not_found — checked https://www.braintrust.dev/docs/integrations/ai-providers (lists OpenAI, Anthropic, Gemini, Mistral, Baseten, Cerebras, Cohere, ElevenLabs, Fireworks, Groq, Hugging Face, Lepton, Ollama, Perplexity, Replicate, Together, TypeSafe, xAI, plus cloud providers — no llama-cpp-python) and https://www.braintrust.dev/docs/guides/traces/integrations (no mention of llama.cpp or llama-cpp-python).

Upstream sources

• GitHub: https://github.com/abetlen/llama-cpp-python
• PyPI: https://pypi.org/project/llama-cpp-python/ (v0.3.35, released 2026-08-17)
• API docs (Llama class): https://llama-cpp-python.readthedocs.io/en/latest/api-reference/

Local repo files inspected

• py/src/braintrust/integrations/ — no llama_cpp/ directory
• py/src/braintrust/wrappers/ — no llama-cpp-python wrapper
• py/pyproject.toml [tool.braintrust.matrix] — no llama-cpp-python entry
• py/noxfile.py — no test_llama_cpp session
• Repo-wide case-insensitive grep for llama_cpp, llama-cpp under py/ — only incidental matches in unrelated cassette YAML fixture text, no actual integration code