Skip to content

LOU-D9: OpenTelemetry GenAI semantic conventions and traced flows - #57

Merged
LinuxDevil merged 3 commits into
mainfrom
lou-d9-otel-genai
Oct 1, 2026
Merged

LinuxDevil merged 3 commits into
mainfrom
lou-d9-otel-genai

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

What

Spans now follow the OpenTelemetry GenAI semantic conventions, and flows are traced.

  • invoke_agent {name} (INTERNAL) > chat {model} (CLIENT) / execute_tool {tool} (INTERNAL), with gen_ai.operation.name, gen_ai.provider.name, gen_ai.agent.*, gen_ai.conversation.id, gen_ai.request.model|temperature|max_tokens, gen_ai.response.model|finish_reasons, gen_ai.usage.input_tokens|output_tokens, gen_ai.tool.name|call.id|description|type, plus error.type and span ERROR status.
  • Content (gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, gen_ai.tool.call.arguments|result) is only recorded with captureContent: true (or OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true).
  • Old attribute names are still emitted (dual-emit) and documented as deprecated in docs/observability.md with an old -> new table.
  • FlowExecutor: invoke_workflow {flow} run span, flow.node {type} per node, chat/execute_tool spans parented under their node; parentSpanId nests a run under an agent span.
  • All names are centralized in src/execution/semconv.ts (spec URL, retrieval 2026-10-01, Development stability).
  • Span gains optional kind/status; createOtelTraceExporter maps them to OTel SpanKind and ERROR status and passes string[] attributes through.

Why

Ad-hoc attribute names did not light up GenAI-aware backends; flows were invisible.

How verified

  • npx tsc --noEmit: clean
  • npm run lint: 0 errors (433 warnings, all pre-existing categories; none added in new files)
  • npm run build and create-loushy-agent build: success
  • npm run test:coverage: Test Files 105 passed, Tests 1420 passed | 3 skipped
  • npm run fallow: exit 0, no issues, no duplication
  • npm run docs:verify-snippets: all 44 snippets type-check, 6 also run
  • npm run example:tracing:console: runs, emits the new span tree
  • agent-forge: typecheck, typecheck:server, vitest (108 passed), test:server (105 passed)
  • Merged origin/main (d7927e7) immediately before the final run.

Notes for reviewer

  • The spec moved from opentelemetry.io to github.com/open-telemetry/semantic-conventions-genai; I read it from there (gen-ai-spans.md, gen-ai-agent-spans.md, JSON schemas). The content env var is only given as an example name in the spec, not a normative setting.
  • Span NAMES changed (required by the spec); attribute names are dual-emitted but span names cannot be. agent-forge's TracePanel and one runRegistry test now match on gen_ai.operation.name.
  • Deprecated content attributes (input, prompt, args, result) keep their previous behavior (recorded unless redactContent), since existing tests encode it. Only the new gen_ai.* content is opt-in. Flipping the legacy default would be a breaking change worth its own ticket.
  • Call-site edits in AgentExecutor.ts are limited to the execute() span wrapper and runToolCallInSpan; generateStep.ts holds the model-call span.
  • gen_ai.response.model is only set when the provider's rawResponse exposes a model id.

🤖 Generated with Claude Code

LinuxDevil and others added 3 commits October 1, 2026 09:57
Spans used ad-hoc names (agent.run, llm.generate, tool.call, model,
promptTokens, ...), so GenAI-aware backends could not recognise them, and
flows were not traced at all.

- Agent runs are now `invoke_agent {name}` (INTERNAL), model calls
  `chat {model}` (CLIENT), tool calls `execute_tool {tool}` (INTERNAL), with
  the gen_ai.* attributes from the current spec (gen_ai.provider.name,
  gen_ai.request.*, gen_ai.response.*, gen_ai.usage.*, gen_ai.tool.*,
  gen_ai.agent.*, gen_ai.conversation.id) plus error.type and span status.
- All names live in src/execution/semconv.ts (spec URL, retrieval date and
  Development stability noted there); span construction in genAiSpans.ts.
- The previous attribute names are still emitted (dual-emit) and documented
  as deprecated in docs/observability.md. Span names changed, as the spec
  requires; agent-forge's trace panel now keys off gen_ai.operation.name.
- Content (gen_ai.input/output.messages, system_instructions,
  tool.call.arguments/result) is never recorded unless `captureContent: true`
  (or OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true).
- Span gains optional `kind` and `status`; createOtelTraceExporter maps them
  to OTel SpanKind/ERROR status and passes string[] attributes through.
- FlowExecutor wraps a run (`invoke_workflow {name}`) and every node
  (`flow.node {type}`) in spans, parents the chat/execute_tool spans under
  their node, and nests under `parentSpanId` when given.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@LinuxDevil
LinuxDevil merged commit 142fe1d into main Oct 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant