Skip to content

[bot] Google GenAI: Chat session API (chats.create, send_message) not instrumented in Python (documented as traced in JS) #799

Description

@braintrust-bot

Summary

The google-genai Python SDK's Chat session API (client.chats.create(model=...) returning a Chat/AsyncChat with send_message()/send_message_stream()) has no dedicated instrumentation in this repo's Python integration. Braintrust's own docs confirm this capability is already designed and shipped for the legacy JS/TS SDK ("Chat spans (send_message, send_message_stream): message input and model as metadata") — it is simply missing from the Python integration.

What is missing

py/src/braintrust/integrations/google_genai/patchers.py and integration.py register patchers only for Models/AsyncModels (generate_content, generate_content_stream, embed_content, generate_images) and Interactions/GAOSInteractions (create/get/cancel/delete, sync and async). There is no patcher targeting google.genai.chats.Chats/AsyncChats/Chat/AsyncChat anywhere in the file, and no reference to chat, Chat, or send_message in tracing.py or patchers.py.

Verified against the pinned upstream version (google-genai==2.24.0, per py/uv.lock): Chat.send_message() does internally delegate to self._modules.generate_content(...) — i.e. it reuses the already-patched Models.generate_content path, so a generate_content span is still produced per turn. However, Chat/AsyncChat (subclass _BaseChat) separately maintains cross-turn conversation state — self._curated_history and self._comprehensive_history, updated via record_history() after every turn — that is invisible to any single generate_content span. There is no session-level span capturing the chat object's lifecycle, turn count, or curated-vs-comprehensive history divergence (e.g. when a turn is marked invalid and excluded from curated history sent to the model on subsequent turns).

Braintrust docs status: supported (JS/TS) / not_found (Python)

The Gemini integration page, Python section, documents only: content generation spans (generate_content/generate_content_stream), embedding spans (embed_content), image generation spans (generate_images), and interaction spans (interactions.create/get/cancel/delete). It contains zero mentions of chat, chats, send_message, or ChatSession in the Python section.

The same page's legacy-SDK section (for @google/generative-ai, JS/TS) explicitly documents: "Chat spans (send_message, send_message_stream): message input and model as metadata." This confirms Braintrust considers chat-session tracing a real, intended capability — it has just not been ported to the Python google-genai integration.

Upstream sources

Local repo files inspected

  • py/src/braintrust/integrations/google_genai/integration.py (full file — patcher list)
  • py/src/braintrust/integrations/google_genai/patchers.py (full file — no Chat/chats patcher)
  • py/src/braintrust/integrations/google_genai/tracing.py (grepped for all top-level class/def names and chat — no matches)
  • py/uv.lock (confirms pinned google-genai==2.24.0)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions