Skip to content

feat(providers): Codex/OpenCode local telemetry; stop showing unknown cost as $0 - #26

Merged
bernardopg merged 2 commits into
mainfrom
feat/local-harness-telemetry
Sep 8, 2026
Merged

feat(providers): Codex/OpenCode local telemetry; stop showing unknown cost as $0#26
bernardopg merged 2 commits into
mainfrom
feat/local-harness-telemetry

Conversation

@bernardopg

Copy link
Copy Markdown
Owner

Summary

Codex and OpenCode now feed the same expanded telemetry card that pi and Hermes use, and the cost pipeline stops turning missing data into $0.

Local harness telemetry

  • New providers/get-local-analytics reads usage metadata (never prompts or responses) from Codex rollout sessions and the OpenCode SQLite store, emitting the shared window shape in scripts/local-analytics.jq: 7-day chart, top models, top projects, per-window input/output/cache/reasoning/session counters.
  • Codex token_count events carry cumulative counters and are re-emitted unchanged on turns that add no tokens, so only positive deltas are summed; a counter reset restarts from zero instead of producing a negative row. last_token_usage is deliberately unused, as it repeats too.
  • opencode reports from the local database by default, so the card works for free and third-party models used through OpenCode without an OpenCode Zen subscription. The Zen quota path is unchanged and still used when no local database exists or OPENCODE_USAGE_SOURCE=api is set; get-provider-health follows the same rule instead of always demanding an API key.
  • Both scans are cached 120s like the pi/Hermes adapters — a Codex scan spawns one jq per session file and took ~5s, far too much per poll (now 16ms warm). The cache records the source path it scanned, so a snapshot is never replayed for a different CODEX_HOME/OPENCODE_DATA_DIR, and a vanished source errors instead of serving stale telemetry.

Cost honesty

  • Hermes rows with cost_status = 'unknown' were summed as if measured, mixing placeholder estimates (one row alone claimed ~$180k) into displayed spend. providers/local-cost-common centralizes the rule so the summary and analytics adapters cannot drift: unknown makes the window unknown, included counts as a true 0, and a window totals only when every row has a known cost. Ledgers predating cost_status fall back to positive values only, since a default 0 cannot establish that a request was free.
  • pi sessions reporting 0 cost alongside non-zero tokens (unpriced/custom models) are treated as unknown rather than free.
  • formatCost renders unknown as and sub-cent amounts as <$0.01, so "no data" and "no charge" are visually distinct. Cards carry a note that local costs may be estimates, and the 7-day chart scales by tokens so it stays meaningful when costs are unknown.

Before/after on a real environment — note the week is now honestly unknown rather than a partial sum presented as the total:

pi      today "$19.57 · 18.6M tok"   week "— · 512.4M tok"
hermes  today "$0 · 0 tok"           week "— · 11.6M tok"

Also fixed

  • Weekday labels came from jq's locale-dependent strftime("%a"), so two cards could disagree on the same day (qua next to Wed); the widget now derives them from the row's ISO date.
  • package-release asserts the new runtime files ship — a release missing scripts/local-analytics.jq would break only at runtime.

Testing

New tests/test-local-analytics.sh covers cumulative deduplication, model/project attribution, unknown vs. recorded zero, cache TTL and cache-key-by-source, and the Hermes cost-status rules.

Local gates all pass: fixture suites, shellcheck -S warning, bash -n, qmllint, i18n key parity.

…nown cost as $0

Codex and OpenCode now feed the same expanded telemetry card that pi and
Hermes use, and the cost pipeline stops turning missing data into $0.

Local harness telemetry:

- New providers/get-local-analytics reads usage metadata (never prompts or
  responses) from Codex rollout sessions and the OpenCode SQLite store,
  emitting the shared window shape in scripts/local-analytics.jq: 7-day
  chart, top models, top projects, and per-window input/output/cache/
  reasoning/session counters.

- Codex token_count events carry cumulative counters that are re-emitted
  unchanged on turns adding no tokens, so only positive deltas are summed;
  a counter reset restarts from zero instead of producing a negative row.
  last_token_usage is deliberately unused, as it repeats too.

- opencode reports from the local database by default, so the card works
  for the free and third-party models used through OpenCode without an
  OpenCode Zen subscription; the Zen quota path is unchanged and still
  used when no local database exists or OPENCODE_USAGE_SOURCE=api is set.
  get-provider-health follows the same rule instead of always demanding
  an API key.

- Both scans are cached for 120s like the pi and Hermes adapters: a Codex
  scan spawns one jq per session file and took ~5s, far too much to repeat
  on every widget poll. The cache records the source path it scanned, so a
  snapshot is never replayed for a different CODEX_HOME/OPENCODE_DATA_DIR
  and a vanished source errors instead of serving stale telemetry.

Cost honesty:

- Hermes rows with cost_status = 'unknown' were summed as if measured,
  mixing placeholder estimates (one row alone claimed ~$180k) into the
  displayed spend. providers/local-cost-common now centralizes the rule so
  the summary and analytics adapters cannot drift: unknown makes the window
  unknown, 'included' counts as a true 0, and a window totals only when
  every row has a known cost. Ledgers predating cost_status fall back to
  positive values only, since a default 0 cannot establish a free request.

- pi sessions reporting 0 cost alongside non-zero tokens (unpriced or
  custom models) are treated as unknown rather than free.

- formatCost renders unknown as "—" and sub-cent amounts as "<$0.01", so
  "no data" and "no charge" are visually distinct. The cards carry a note
  that local costs may be estimates, and the 7-day chart scales by tokens
  so it stays meaningful when costs are unknown.

Also: weekday labels came from jq's locale-dependent strftime("%a"), so
two cards could disagree on the same day ("qua" next to "Wed"); the widget
now derives them from the row's ISO date. package-release asserts the new
runtime files ship, since a release missing scripts/local-analytics.jq
would break only at runtime.
@github-actions github-actions Bot added documentation Improvements or additions to documentation area:providers Provider adapters (providers/) area:i18n Translations (i18n/) area:qml QML UI code area:ci CI, workflows, tooling labels Sep 8, 2026
CI's "Confirm all scripts are executable" gate requires the mode bit on
every tests/*.sh; the new suite was committed as 100644.
@bernardopg
bernardopg merged commit 35f8219 into main Sep 8, 2026
10 checks passed
@bernardopg
bernardopg deleted the feat/local-harness-telemetry branch September 8, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ci CI, workflows, tooling area:i18n Translations (i18n/) area:providers Provider adapters (providers/) area:qml QML UI code documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant