[EPIC-D][LOU-D48] OpenTelemetry GenAI metrics and a loushy.cost_usd span attribute

CHANGELOG.md

@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased] - 2026-09-28
 
 ### Added
+- OpenTelemetry GenAI metrics and cost (LOU-D48): `createOtelTraceExporter()` (`@loushy/build-ai-agent/otel`) now also records the GenAI client metrics `gen_ai.client.token.usage` (histogram, `{token}`, one record per `gen_ai.token.type` `input`/`output`) and `gen_ai.client.operation.duration` (histogram, `s`, for every model call and tool call) through `@opentelemetry/api`'s metrics API: the global `MeterProvider`, or the new `meter` option; `metrics: false` turns them off. Without a registered `MeterProvider` they are discarded by OpenTelemetry's no-op meter. Spans gain `loushy.cost_usd` (estimated USD: per step on `chat` spans, cumulative on `invoke_agent` spans, absent when the price table does not know a model) and `loushy.usage.estimated` on `invoke_agent` spans (it was already on `chat` spans). Any `TraceExporter` sees the cumulative attributes: `withSpan()` sums the cost of finished child spans onto their parent. See docs/observability.md.
 - One `store` option (LOU-D30): `createAgent({ store })` takes an `AgentStore` (`{ sessions?, checkpoints?, approvals? }`, new export) and wires all three from it. `SqliteStore` is one; the new `memoryStore()` returns in-memory sessions, checkpoints and approvals, and docs/sessions.md shows the file-based combination (`FileSessionStore`, `LocalStorageCheckpointStore`, `StorageServiceApprovalStore`). `agent.session({ id })` then keeps its transcript in `store.sessions` and checkpoints every turn in `store.checkpoints`; `store.approvals` is the default `approvalStore`. `SendOptions.sessionId` makes `send()` / `stream()` a durable run checkpointed in `store.checkpoints` (an error without one), so a one-shot durable job needs no `AgentExecutor`, and `agent.approvals.resolve()` keeps checkpointing it. The new `agent.resume(sessionId)` finishes an interrupted `send(..., { sessionId })` run, or else the session's pending turn (as `session.resume()`), and returns `null` when nothing is pending. A session's own `store` / `checkpointStore` and an explicit `approvalStore` still win, part by part. Without `store`, nothing changes.
 - One tool-error shape (LOU-U14): every failed tool call reaches the model as `{ error, toolName, message, kind }` (message capped at 2,000 characters, transcript message flagged `isError`), whichever path failed: `kind` is `'execution'`, `'validation'` (plus `issues`), `'not-found'`, `'rejected'` (plus `note`), `'not-run'`, `'mcp'` or `'sandbox'`. New exports `toolErrorResult()`, `ToolErrorResult`, `ToolErrorKind` and `ToolErrorInput`; `McpToolError` and the sandbox guard's error (named `SandboxRequiredError`, not exported) carry a `toolErrorKind`. `kind` is additive on thrown-error and validation results. The paths that built their own shapes changed, see Changed. See docs/tools.md.
 - Background sub-agents end with the lead run (LOU-Y4.2): when a run ends (any `finishReason`, or an error), its background sub-agents still queued or running are cancelled, and the run resolves once their runs have stopped, so none of their events reaches `onEvent` afterwards. `withSubagentOptions(subagents, { awaitBackgroundOnFinish: true })` waits for them instead (an aborted or failed lead still cancels them). The new `result.backgroundTasks` lists the run's background tasks as `{ taskId, agent, status, elapsedMs, ... }` (absent when it started none). `createAgent({ subagentOptions: { maxConcurrent, awaitBackgroundOnFinish } })` sets the same options (over any attached with `withSubagentOptions()`, without changing the `subagents` value). The executor hook behind it is public: `ExecuteOptions.onRunEnd({ result } | { error })` is called exactly once per `execute()` / `stream()` run, however it ends; an error it throws rejects the run. See docs/sub-agents.md.

docs/observability.md

@@ -31,7 +31,7 @@ For real OpenTelemetry spans, import `createOtelTraceExporter()` from
 `@loushy/build-ai-agent/otel` (needs the optional peer dependency
 `@opentelemetry/api` and a registered `TracerProvider`). It carries span kind
 and error status over to OpenTelemetry. See `examples/tracing`
-(`npm run example:tracing:console`, `npm run example:tracing:otel`).
+(`npm run example:tracing:console`, `npm run example:tracing:otel`, which also prints metrics).
 
 ## Span tree
 
@@ -57,7 +57,7 @@ child.
 ## Attributes
 
 Constants for every name live in `src/execution/semconv.ts` (exported as
-`GenAiAttr`, `GenAiOperation`, `ErrorAttr`, `FlowAttr`, `LegacyAttr`).
+`GenAiAttr`, `GenAiOperation`, `GenAiMetric`, `ErrorAttr`, `FlowAttr`, `SdkAttr`, `LegacyAttr`).
 
 ### Agent run: `invoke_agent {agent name}`
 
@@ -68,6 +68,8 @@ Constants for every name live in `src/execution/semconv.ts` (exported as
 | `gen_ai.agent.id` | The agent's `id`, when set |
 | `gen_ai.provider.name` | The provider's `name` |
 | `gen_ai.conversation.id` | `sessionId`, when set |
+| `loushy.cost_usd` | Cumulative estimated USD of the run (every model call and delegated child run); absent when any model used has no known price |
+| `loushy.usage.estimated` | `true` when any of the run's tokens were estimated |
 
 ### Model call: `chat {model}` (CLIENT)
 
@@ -80,6 +82,8 @@ Constants for every name live in `src/execution/semconv.ts` (exported as
 | `gen_ai.response.model` | When the provider reports it |
 | `gen_ai.response.finish_reasons` | e.g. `["stop"]`, `["tool_call"]` |
 | `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | Token usage |
+| `loushy.cost_usd` | Estimated USD of this step, when the built-in price table (or `registerModel`) knows the model; absent otherwise |
+| `loushy.usage.estimated` | `true` when the provider reported no usage and the tokens were estimated |
 
 ### Tool call: `execute_tool {tool}`
 
@@ -114,6 +118,43 @@ A failed span gets `error.type` and an error span status (OpenTelemetry
 `name` (`_OTHER` when a non-`Error` is thrown). A tool call that returns an
 error to the model (`isError`) is marked `error.type = tool_error`.
 
+## Metrics
+
+`createOtelTraceExporter()` also records the OpenTelemetry GenAI client
+metrics through `@opentelemetry/api`'s metrics API, using the global
+`MeterProvider` (`metrics.getMeter(...)`), or the `meter` option:
+
+| Metric | Type, unit | Recorded | Attributes |
+| --- | --- | --- | --- |
+| `gen_ai.client.token.usage` | Histogram, `{token}` | Once per token type for every model call that reports (or estimates) usage | `gen_ai.operation.name` (`chat`), `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.response.model` (when known), `gen_ai.token.type` (`input` or `output`) |
+| `gen_ai.client.operation.duration` | Histogram, `s` | Every model call (`chat`) and tool call (`execute_tool`) | `gen_ai.operation.name`, `gen_ai.provider.name` and `gen_ai.request.model` (model calls), `gen_ai.response.model` (when known), `error.type` (when it failed) |
+
+Both use the spec's recommended bucket boundaries. They follow the
+`gen-ai-metrics` page of semantic-conventions v1.40 (`Development` stability):
+the newest conventions split the token histogram into per-type counters,
+which this SDK does not emit yet. Agent and flow spans are not counted in the
+duration metric, only the model and tool calls inside them.
+
+```ts
+import { metrics } from '@opentelemetry/api';
+import { ConsoleMetricExporter, MeterProvider, PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
+import { createOtelTraceExporter } from '@loushy/build-ai-agent/otel';
+
+// Swap ConsoleMetricExporter for an OTLP exporter to reach any backend.
+metrics.setGlobalMeterProvider(
+  new MeterProvider({ readers: [new PeriodicExportingMetricReader({ exporter: new ConsoleMetricExporter() })] })
+);
+
+const exporter = createOtelTraceExporter(); // metrics on by default
+// createOtelTraceExporter({ metrics: false })  // spans only
+// createOtelTraceExporter({ meter })           // an already-obtained Meter
+```
+
+Without a registered `MeterProvider`, OpenTelemetry's no-op meter discards
+the records. The metrics API lives in `@opentelemetry/api`, so nothing is
+recorded (and nothing is loaded) when that optional peer is not installed.
+The cost is an estimate from the price table (see `registerModel`), not a bill.
+
 ## Message and argument content is opt-in
 
 Prompts, model output and tool arguments/results are sensitive and large, so

examples/tracing/run-otel.ts

@@ -9,10 +9,21 @@
  * that decides *where* the spans go; the SDK-to-OTel translation is no
  * longer hand-rolled here.
  *
+ * LOU-D48: the same exporter also records the GenAI metrics
+ * (`gen_ai.client.token.usage`, `gen_ai.client.operation.duration`) through
+ * whatever global MeterProvider is registered; here an OTel MeterProvider
+ * prints them to the console on shutdown. The spans carry `loushy.cost_usd`.
+ *
  * Run with:
  *   npm run example:tracing:otel
  */
 
+import { metrics } from '@opentelemetry/api';
+import {
+  ConsoleMetricExporter,
+  MeterProvider,
+  PeriodicExportingMetricReader,
+} from '@opentelemetry/sdk-metrics';
 import {
   NodeTracerProvider,
   ConsoleSpanExporter,
@@ -28,6 +39,13 @@ async function main() {
   provider.register();
   const tracer = provider.getTracer('loushy-tracing-example');
 
+  // Metrics go wherever the global MeterProvider sends them. `shutdown()`
+  // below flushes them to the console once.
+  const meterProvider = new MeterProvider({
+    readers: [new PeriodicExportingMetricReader({ exporter: new ConsoleMetricExporter() })],
+  });
+  metrics.setGlobalMeterProvider(meterProvider);
+
   // In real usage the tracer name is enough - `createOtelTraceExporter()`
   // will resolve one via `trace.getTracer(...)` for you. Passing the
   // tracer explicitly here just reuses the one obtained above.
@@ -36,6 +54,7 @@ async function main() {
   await runWeatherScenario(exporter);
 
   await provider.shutdown();
+  await meterProvider.shutdown();
 }
 
 main().catch((error) => {

examples/tracing/weather-scenario.ts

@@ -77,6 +77,8 @@ function createWeatherAgent() {
     .setType(AgentType.SmartAssistant)
     .setName('Weather Agent')
     .setPrompt('You are a helpful weather assistant.')
+    // A priced model id, so spans carry `loushy.cost_usd` (the provider is still a mock).
+    .setSettings({ model: 'gpt-4o-mini' })
     .addTool('getWeather', { tool: 'getWeather', options: {} })
     .build();
 }

llms-full.txt

@@ -3378,7 +3378,7 @@ For real OpenTelemetry spans, import `createOtelTraceExporter()` from
 `@loushy/build-ai-agent/otel` (needs the optional peer dependency
 `@opentelemetry/api` and a registered `TracerProvider`). It carries span kind
 and error status over to OpenTelemetry. See `examples/tracing`
-(`npm run example:tracing:console`, `npm run example:tracing:otel`).
+(`npm run example:tracing:console`, `npm run example:tracing:otel`, which also prints metrics).
 
 ## Span tree
 
@@ -3404,7 +3404,7 @@ child.
 ## Attributes
 
 Constants for every name live in `src/execution/semconv.ts` (exported as
-`GenAiAttr`, `GenAiOperation`, `ErrorAttr`, `FlowAttr`, `LegacyAttr`).
+`GenAiAttr`, `GenAiOperation`, `GenAiMetric`, `ErrorAttr`, `FlowAttr`, `SdkAttr`, `LegacyAttr`).
 
 ### Agent run: `invoke_agent {agent name}`
 
@@ -3415,6 +3415,8 @@ Constants for every name live in `src/execution/semconv.ts` (exported as
 | `gen_ai.agent.id` | The agent's `id`, when set |
 | `gen_ai.provider.name` | The provider's `name` |
 | `gen_ai.conversation.id` | `sessionId`, when set |
+| `loushy.cost_usd` | Cumulative estimated USD of the run (every model call and delegated child run); absent when any model used has no known price |
+| `loushy.usage.estimated` | `true` when any of the run's tokens were estimated |
 
 ### Model call: `chat {model}` (CLIENT)
 
@@ -3427,6 +3429,8 @@ Constants for every name live in `src/execution/semconv.ts` (exported as
 | `gen_ai.response.model` | When the provider reports it |
 | `gen_ai.response.finish_reasons` | e.g. `["stop"]`, `["tool_call"]` |
 | `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | Token usage |
+| `loushy.cost_usd` | Estimated USD of this step, when the built-in price table (or `registerModel`) knows the model; absent otherwise |
+| `loushy.usage.estimated` | `true` when the provider reported no usage and the tokens were estimated |
 
 ### Tool call: `execute_tool {tool}`
 
@@ -3461,6 +3465,43 @@ A failed span gets `error.type` and an error span status (OpenTelemetry
 `name` (`_OTHER` when a non-`Error` is thrown). A tool call that returns an
 error to the model (`isError`) is marked `error.type = tool_error`.
 
+## Metrics
+
+`createOtelTraceExporter()` also records the OpenTelemetry GenAI client
+metrics through `@opentelemetry/api`'s metrics API, using the global
+`MeterProvider` (`metrics.getMeter(...)`), or the `meter` option:
+
+| Metric | Type, unit | Recorded | Attributes |
+| --- | --- | --- | --- |
+| `gen_ai.client.token.usage` | Histogram, `{token}` | Once per token type for every model call that reports (or estimates) usage | `gen_ai.operation.name` (`chat`), `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.response.model` (when known), `gen_ai.token.type` (`input` or `output`) |
+| `gen_ai.client.operation.duration` | Histogram, `s` | Every model call (`chat`) and tool call (`execute_tool`) | `gen_ai.operation.name`, `gen_ai.provider.name` and `gen_ai.request.model` (model calls), `gen_ai.response.model` (when known), `error.type` (when it failed) |
+
+Both use the spec's recommended bucket boundaries. They follow the
+`gen-ai-metrics` page of semantic-conventions v1.40 (`Development` stability):
+the newest conventions split the token histogram into per-type counters,
+which this SDK does not emit yet. Agent and flow spans are not counted in the
+duration metric, only the model and tool calls inside them.
+
+```ts
+import { metrics } from '@opentelemetry/api';
+import { ConsoleMetricExporter, MeterProvider, PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
+import { createOtelTraceExporter } from '@loushy/build-ai-agent/otel';
+
+// Swap ConsoleMetricExporter for an OTLP exporter to reach any backend.
+metrics.setGlobalMeterProvider(
+  new MeterProvider({ readers: [new PeriodicExportingMetricReader({ exporter: new ConsoleMetricExporter() })] })
+);
+
+const exporter = createOtelTraceExporter(); // metrics on by default
+// createOtelTraceExporter({ metrics: false })  // spans only
+// createOtelTraceExporter({ meter })           // an already-obtained Meter
+```
+
+Without a registered `MeterProvider`, OpenTelemetry's no-op meter discards
+the records. The metrics API lives in `@opentelemetry/api`, so nothing is
+recorded (and nothing is loaded) when that optional peer is not installed.
+The cost is an estimate from the price table (see `registerModel`), not a bill.
+
 ## Message and argument content is opt-in
 
 Prompts, model output and tool arguments/results are sensitive and large, so

package.json

@@ -205,6 +205,7 @@
     "@ai-sdk/openai": "^0.0.42",
     "@modelcontextprotocol/sdk": "^1.30.1",
     "@opentelemetry/api": "^1.9.1",
+    "@opentelemetry/sdk-metrics": "^2.11.0",
     "@opentelemetry/sdk-trace-node": "^2.11.0",
     "@types/dockerode": "^4.0.1",
     "@types/node": "^20",

src/execution/genAiSemconv.test.ts

@@ -388,3 +388,51 @@ describe('traced flows (LOU-D9)', () => {
     expect(on.spans[3].attributes['gen_ai.tool.call.arguments']).toBe(JSON.stringify({ city: 'Paris' }));
   });
 });
+
+describe('loushy.cost_usd and loushy.usage.estimated spans (LOU-D48)', () => {
+  async function runPriced(model: string, scripted: boolean) {
+    const { agent, toolRegistry } = setup();
+    const { exporter, spans } = memoryExporter();
+    const provider = scripted
+      ? modelWithOneToolCall()
+      : mockModel([{ toolCalls: [{ id: 'c1', name: 'get_weather', args: { city: 'Paris' } }] }, 'It is 21C.']);
+    const result = await AgentExecutor.execute({
+      agent: { ...agent, settings: { model } },
+      input: 'Weather in Paris?',
+      provider,
+      toolRegistry,
+      exporter,
+    });
+    return { spans, result };
+  }
+
+  const gpt4oMini = (input: number, output: number) => (input * 0.15 + output * 0.6) / 1e6;
+
+  it('records the step cost on each chat span and the cumulative cost on the invoke_agent span', async () => {
+    const { spans, result } = await runPriced('gpt-4o-mini', true);
+    const [run, chat1, tool, chat2] = spans;
+
+    expect(chat1.attributes['loushy.cost_usd']).toBeCloseTo(gpt4oMini(10, 5), 12);
+    expect(chat2.attributes['loushy.cost_usd']).toBeCloseTo(gpt4oMini(20, 7), 12);
+    expect(run.attributes['loushy.cost_usd']).toBeCloseTo(gpt4oMini(30, 12), 12);
+    expect(run.attributes['loushy.cost_usd']).toBeCloseTo(result.usage.costUsd as number, 12);
+    expect(tool.attributes).not.toHaveProperty('loushy.cost_usd');
+    for (const span of [run, chat1, chat2]) expect(span.attributes).not.toHaveProperty('loushy.usage.estimated');
+  });
+
+  it('omits loushy.cost_usd when the model has no known price', async () => {
+    const { spans } = await runPriced('test-model', true);
+    expect(spans.some((s) => 'loushy.cost_usd' in s.attributes)).toBe(false);
+  });
+
+  it('flags loushy.usage.estimated on the chat spans and the run span when tokens were estimated', async () => {
+    const { spans, result } = await runPriced('gpt-4o-mini', false);
+    const [run, chat1, , chat2] = spans;
+
+    expect(result.usage.estimated).toBe(true);
+    expect(run.attributes['loushy.usage.estimated']).toBe(true);
+    expect(chat1.attributes['loushy.usage.estimated']).toBe(true);
+    expect(chat2.attributes['loushy.usage.estimated']).toBe(true);
+    expect(run.attributes['loushy.cost_usd']).toBeCloseTo(result.usage.costUsd as number, 12);
+  });
+});