Skip to content

feat(telemetry): add task and role attribution and /api/task-summary endpoint - #636

Draft
lis186 wants to merge 8 commits into
mainfrom
feat/agentflow-telemetry
Draft

lis186 wants to merge 8 commits into
mainfrom
feat/agentflow-telemetry

Conversation

@lis186

@lis186 lis186 commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Summary

Adds task and role attribution telemetry support and a /api/task-summary aggregation endpoint to support fine-grained multi-agent task and workflow tracking.

Changes

  • server/entry.js: Append 'task' and 'role' to INDEX_FIELDS (strictly following append-only schema evolution rules).
  • server/forward.js: Extract x-ccxray-task and x-ccxray-role request headers into entry deployment metadata for HTTP/SSE requests.
  • server/ws-proxy.js: Extract x-ccxray-task and x-ccxray-role from WebSocket handshake headers for OpenAI/Codex wire sessions.
  • server/routes/api.js: Add GET /api/task-summary?task=<task_id>&project=<cwd> to aggregate calls, tokens (input/output/cache read/cache create/reasoning/total), USD cost, and cache hit rate.
  • test/task-summary.test.js: Added unit tests for parameter validation and aggregation logic.

Verification

  • npm test: All 2,579 unit tests pass, including the G1: INDEX_FIELDS preserves every legacy field name and order regression test.
  • Zero breaking changes for existing ccxray users without these headers.

Status

  • Draft: Preparing to run an iOS app test project to verify live task attribution and cost tracking before merging.

Justin Lee and others added 8 commits September 19, 2026 17:14
Header attribution only ever reached Claude Code. Codex on a ChatGPT login
and the Grok CLI cannot send custom headers, so their traffic was never
attributed. Every CLI does accept a base URL, so attribution now rides a
path prefix that ccxray strips before forwarding:

  /_ccxray/attr/<encodeURIComponent("task=..&role=..&project=..")>

- server/attribution.js: one parser and one sanitizer for the prefix and
  the x-ccxray-task|role|project headers. The prefix wins per key.
- hub.applyClientRoute strips every leading /_ccxray/client/<pid> and
  /_ccxray/attr/<segment> prefix, in any order, so a worker launched from
  inside a ccxray session can append its own attribution. It strips all
  of them, never a bounded number: a cap left the remainder in req.url,
  which is forwarded upstream verbatim.
- /_api/health advertises capabilities: ['task-attribution'] so an
  integrator never injects the prefix into a ccxray that would forward
  it upstream and 404 every worker call.
- CCXRAY_TASK / CCXRAY_ROLE / CCXRAY_PROJECT on a `ccxray <agent>` launch
  register with the hub as launch-time attribution.
- INDEX_FIELDS appends taskProject. It is not the cwd-derived dashboard
  project: a worker usually runs in a disposable clone.
- fix(ws-proxy): merging attribution into `identity` switched off the
  env-identity fallback, silently dropping userEmail/team from every
  attributed WebSocket turn. Gate on the hub identity, as forward.js does.

Tests run the real proxy against a mock upstream for Claude, Codex over
HTTP and WebSocket, and Grok, and assert neither the prefix nor any
x-ccxray-* header reaches the upstream. Mutation-checked: removing the
prefix strip, or gating env identity on the merged identity in either
request path, turns a test red.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t filters

The first cut added the cumulative input/output sums to `total` on every
entry that lacked a provider total, so N entries grew quadratically, and
it read usage keys the store never writes.

- Extract a pure aggregator (server/task-summary.js) over the canonical
  usage fields, which are disjoint for every provider. The per-entry
  total is their sum; a provider total_tokens is never trusted, because
  OpenAI's counts cached input inside input.
- Add an optional `role` filter and a `by_role` breakdown, plus models,
  agents, session count, and first/last timestamps.
- `project` matches the declared taskProject exactly. An entry that
  declared none falls back to its cwd, matched as a whole path segment:
  a substring test let `ipadpos` count a worker in `.../ipadpos-web`.
- Report `coverage` (entries in memory vs CCXRAY_MAX_ENTRIES) so a caller
  can tell "no calls" from "calls aged out of the window".
- Serve /_api/task-summary, keeping /api/task-summary as an alias.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
docs/task-attribution.md is the integrator contract: the capability check,
the three carriers and their precedence, per-CLI base-URL settings, and the
/_api/task-summary response. README gains a short section and the
CCXRAY_TASK / CCXRAY_ROLE / CCXRAY_PROJECT rows; CLAUDE.md lists the two new
server modules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…el handling

An orchestrator's coordinator is one long-lived process whose base URL is
fixed at launch while the task changes with every Ask, so the /_ccxray/attr/
prefix cannot label it. Its SESSION ID can: ccxray already records it on
every entry, and both Claude Code (CLAUDE_CODE_SESSION_ID) and Codex
(CODEX_THREAD_ID) expose the same value to the host's tool calls.

ccxray keeps no binding state. The caller passes intervals:

  GET /_api/task-summary?task=<id>&session=<sessionId>@<fromMs>-<toMs>…

Entries of those sessions inside those intervals join the task under role
`coordinator`; an entry labelled for a DIFFERENT task is skipped; results are
de-duplicated by entry id. Coordinator entries are read from the index on
disk, not the in-memory window, because a coordinator makes many calls. With
no `session` param the route is byte-identical to before (proved against 600
aggregator cases and the live handler). /_api/health advertises
`session-intervals`.

Hardening from independent review:
- Role labels and tool names are client-supplied. `byRole[roleKey]` with
  roleKey `constructor` read Object.prototype.constructor and `+=` threw,
  taking the proxy down on a GET. Roles, tools, and skills now accumulate in
  Maps and are emitted with Object.fromEntries (own properties only).
- An unknown cost was summed as an exact $0. `cost_confidence`
  { priced, unknown, fallback, no_usage } is reported for the total and per
  role so a caller can mark the figure (ADR 0017's aggregate rule).
- docs: cwd fallback is a whole-path-segment match; coordinator selection is
  not limited by CCXRAY_MAX_ENTRIES.

Full suite: 2620 pass, 0 fail.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…confidence for cost receipts

Adds to GET /_api/task-summary, on the total, every by_role entry, and
coordinator:
- charges: one bucket per (model, billing_provider, component, rate, basis)
  with decimal-string quantity, usd_per_unit and usd computed exactly with
  BigInt (quantity/1e6 × rate); basis recorded|fallback|unpriced; an entry
  with cost but no rate for a component yields an unpriced bucket
- last_ingested_at, pending_requests, uncomputable_requests (entries whose
  charges contain an unpriced bucket)
- cost_confidence on coordinator (was missing; integrators validating every
  aggregate with one rule rejected every real coordinator response)
- from/to epoch-ms window (inclusive/exclusive) restricting labelled entries;
  session SPECs keep their own intervals; reported as `window`

cost_usd is now the exact decimal sum of per-entry cost.cost strings (shortest
round-trip repr, no toPrecision) rounded half-up to four places, so it agrees
with the charge buckets at rounding ties. Numeric-string legacy costs are
priced; NaN/negative stay unpriced with an unpriced bucket so confidence and
basis can never disagree.

Coordinator (session interval) selection excludes entries carrying a worker
role, so a labelled request that shares the host session id is never counted
in both a worker scope and the coordinator scope.

Health capabilities gain 'cost-charges'. docs/task-attribution.md documents
the charges shape, the window params, and the coordinator rule.

Verified by fifteen independent GPT 6 astra rounds against the joint
receipt spec (final PASS) and live against Claude Code and Codex hosts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant