Skip to content

[EPIC-D][LOU-D48] OpenTelemetry GenAI metrics and a loushy.cost_usd span attribute - #95

Merged
LinuxDevil merged 2 commits into
mainfrom
lou-d48-otel-metrics-cost
Oct 1, 2026
Merged

LinuxDevil merged 2 commits into
mainfrom
lou-d48-otel-metrics-cost

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

What

  • createOtelTraceExporter({ meter?, metrics? }) (@loushy/build-ai-agent/otel) now also records the OTel GenAI client metrics through @opentelemetry/api's metrics API (metrics.getMeter(...) by default, or the meter option; metrics: false turns them off):
    • gen_ai.client.token.usage (histogram, {token}): one record per gen_ai.token.type (input/output) per model call, attributes gen_ai.operation.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.response.model (when known).
    • gen_ai.client.operation.duration (histogram, s): every model call (chat) and tool call (execute_tool; the spec lists it as a valid operation), plus error.type on failures.
    • Spec bucket boundaries are passed as histogram advice.
  • Spans gain loushy.cost_usd: per step on chat spans (from the run's measured cost, or the price table for flows), cumulative on parent spans (invoke_agent, and a delegating execute_tool) through a small rollup in withSpan(), so any TraceExporter sees it. Absent when the price table does not know a contributing model. loushy.usage.estimated is now also set on run spans whose tokens were estimated (it was already on chat spans).
  • @opentelemetry/sdk-metrics added as a devDependency (tests and the example). Docs "Metrics" section and attribute table, example prints metrics, CHANGELOG entry, regenerated llms-full.txt.

Why

Differentiator 6 of the audit: OTel GenAI conventions plus cost, out of the box, on any backend. Spans already followed the conventions (LOU-D9); this adds the metrics half and the cost attribute.

Spec note: the opentelemetry.io page was not reachable from the sandbox; names, units and attributes were read from the semantic-conventions v1.40 gen-ai-metrics.md (which defines gen_ai.client.token.usage with gen_ai.token.type). The newest semantic-conventions-genai repo replaces that histogram with per-type counters; this PR follows the ticket and v1.40 and says so in docs/observability.md.

Acceptance criteria

  • Both histograms recorded per model call with the required attributes; nothing recorded with metrics: false; tool calls get gen_ai.client.operation.duration with execute_tool (spec allows it)
  • loushy.cost_usd per step on chat, cumulative on invoke_agent, absent when unpriced; loushy.usage.estimated on run spans
  • Tests: real SDK MeterProvider (global) and recording-meter tests for one priced mockModel run; span attribute tests in genAiSemconv/tracing tests; import-graph test that only execution/otel.ts imports @opentelemetry/* (so the main package works without the optional peer)
  • Docs "Metrics" section, attribute table, example prints metrics, llms regenerated, CHANGELOG (Added)
  • Scope: only otel.ts, genAiSpans.ts, semconv.ts, tracing.ts, tests, examples/tracing, docs, package.json/lock (devDependency). Executor, resume, createAgent, subagents, cli, evals, testing and session untouched.

How verified

  • npx tsc --noEmit: pass
  • npm run lint: 0 errors, 405 warnings (none in changed files)
  • npm run test:types: no errors
  • npx vitest run: 144 files passed, 2103 tests passed, 3 skipped
  • npm run build && npm run build --workspace=packages/create-loushy-agent: pass
  • npm run docs:verify-snippets -- --skip-build: all 108 snippets type-check
  • npm run docs:llms:check: pass (exit 0)
  • npm run test:coverage: pass (thresholds met); npm run fallow: exit 0 (the pre-existing providers duplicate and agentDir/fsUtil target are unrelated to this change)
  • npm run example:tracing:otel: prints spans and both metrics

🤖 Generated with Claude Code

https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR


Generated by Claude Code

claude added 2 commits October 1, 2026 15:04
…pan attribute

createOtelTraceExporter() now records gen_ai.client.token.usage and
gen_ai.client.operation.duration through @opentelemetry/api's metrics API
(global MeterProvider, or the new `meter` option; `metrics: false` disables).
Spans gain loushy.cost_usd (per step on chat spans, cumulative on parents via
a withSpan() rollup, absent when the model is unpriced) and
loushy.usage.estimated on run spans. Adds @opentelemetry/sdk-metrics as a
devDependency for tests and the example.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
@LinuxDevil
LinuxDevil merged commit 3354f96 into main Oct 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants