From 5792c316cc30b418eb6fe80790f8aca1dae3fde9 Mon Sep 17 00:00:00 2001 From: omartelo Date: Sun, 4 Oct 2026 18:15:55 -0300 Subject: [PATCH 1/2] feat: report a Claude Code session's own context, rate limits and cost A new mod, hooks/mod-usage.js, hooks session.measure and posts what Claude Code measured to lich's POST /mod/usage (the mod-usage contract), only from an interactive session, one report at a time, dropping a failed one and stopping on a 404. Claude Code takes a single hooks module per plugin and refuses a second modules entry, so hooks/mod.js becomes that module and registers both mods; mod-control.js is unchanged. The fixture is vendored from the lich commit that adds the contract, which is in no release yet. --- CHANGELOG.md | 6 + CLAUDE.md | 11 +- README.md | 3 +- docs/mod-usage.md | 47 +++++++ docs/providers.md | 3 +- hooks/lich.js | 2 + hooks/mod-usage.js | 119 ++++++++++++++++++ tests/contract.mjs | 17 +++ tests/fixtures/mod-usage.jsonl | 18 +++ tests/mod-usage.engine.test.ts | 65 ++++++++++ tests/mod-usage.test.mjs | 223 +++++++++++++++++++++++++++++++++ tests/refresh-fixtures.sh | 8 ++ 12 files changed, 517 insertions(+), 5 deletions(-) create mode 100644 docs/mod-usage.md create mode 100644 hooks/mod-usage.js create mode 100644 tests/fixtures/mod-usage.jsonl create mode 100644 tests/mod-usage.engine.test.ts create mode 100644 tests/mod-usage.test.mjs diff --git a/CHANGELOG.md b/CHANGELOG.md index 7257403..9894fe1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,6 +32,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 the prompt cache, and sends the answer back in its ack, cut at 16,000 characters. An ask still answering holds back no other command. lich asks with `lich ask` or an agent's `ask_session` tool (lich-plugin 0.15 contract). +- **A Claude Code session reports what it measured about itself.** The mod + posts the context window, the account's rate limits and the conversation's + cost each time Claude Code measures them, so lich's footer shows Claude Code's + own cost, the model's real window and, for a long-lived token login, the plan + gauge without spending a request to measure it. Needs a lich that serves the + mod-usage contract; an older one answers 404 and the mod stops reporting. ## [0.14.0] - 2026-10-03 diff --git a/CLAUDE.md b/CLAUDE.md index f047750..ce1495f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,9 +17,10 @@ hooks/crush-hooks.json # hook registration, Crush (hand-merged into c hooks/ # hook scripts (all four run them; only how they are addressed differs) hooks/detail.jq # what a tool call acts on, shared by two of those scripts hooks/win-run.cmd # runs one of those scripts on Windows (Codex) -hooks/lich.js # the one module hooks/hooks.json names under `modules`: registers the two mods below +hooks/lich.js # the one module hooks/hooks.json names under `modules`: registers the three mods below hooks/mod-control.js # Claude Code mod: lich drives the session hooks/agent-cards.js # Claude Code mod: a general-purpose subagent runs as a lich session +hooks/mod-usage.js # Claude Code mod: the session reports its own context, rate limits and cost opencode/lich.js # opencode client: a module, not a command omp/lich.js # omp client: a module, not a command docs/ # lich ⇄ plugin communication contracts @@ -57,6 +58,7 @@ Contracts are **canonical in the lich repository** (`docs/hooks/` there); this p - [docs/session-touched.md](docs/session-touched.md) — git-status refresh signal via `PostToolUse` (file-mutating tools only) - [docs/mod-control.md](docs/mod-control.md): lich drives a Claude Code session (prompt, abort, model, effort, command) and asks it side questions (ask) through a mod that long-polls for commands and acks each one; Claude Code only, the other harnesses have no mod system - [docs/agent-cards.md](docs/agent-cards.md): a general-purpose subagent Claude Code starts runs as a lich session, through the `lich open` and `lich wait` CLI rather than an HTTP contract; Claude Code only +- [docs/mod-usage.md](docs/mod-usage.md): a Claude Code session reports the context window, rate limits and cost it measured itself, through a mod hooking `session.measure`; Claude Code only - [docs/providers.md](docs/providers.md) — the per-harness map, including what installing on omp takes and what it cannot report Each doc carries the event mapping for **every** provider, one column each — a contract is implemented once and registered per harness. [docs/providers.md](docs/providers.md) is the map: which file each harness reads, where the event vocabularies differ, and what adding another provider takes. A harness that cannot close a state does not get that report registered: an unendable `busy` is worse on a card than a card with no indicator. @@ -88,7 +90,9 @@ pinned in a comment there, because 17.x moves fast — plus the rule that outsid lich it subscribes to nothing at all. [tests/mod-control.test.mjs](tests/mod-control.test.mjs) does the same for the Claude Code mod, handing it a fake `$` and asserting its acks against the -mod-control fixtures and its polls against the commands lich sends. +mod-control fixtures and its polls against the commands lich sends; +[tests/mod-usage.test.mjs](tests/mod-usage.test.mjs) does the same for its +usage reports against the mod-usage fixtures. [tests/agent-cards.test.mjs](tests/agent-cards.test.mjs) hands the agent-cards mod the same kind of fake `$` and asserts the `lich open` and `lich wait` argv it runs, the Agent result it answers each outcome with, and that a call it does @@ -97,7 +101,8 @@ not take, or one lich cannot take, reaches the native agent untouched. stub, shared so every client answers to the same lines. Node only, no dependencies. [tests/mod-control.engine.test.ts](tests/mod-control.engine.test.ts) and -[tests/agent-cards.engine.test.ts](tests/agent-cards.engine.test.ts) run the +[tests/agent-cards.engine.test.ts](tests/agent-cards.engine.test.ts) and +[tests/mod-usage.engine.test.ts](tests/mod-usage.engine.test.ts) run the mods under Claude Code's own engine (`claude plugin test .`); they need a `claude` binary, so CI does not run them. The kit does not check a `tool.call` result against the tool's output schema (measured on 2.1.289), so whether diff --git a/README.md b/README.md index 56d4b8e..7167f52 100644 --- a/README.md +++ b/README.md @@ -57,9 +57,10 @@ hooks/report-tool.sh # session-state hook: the tool a turn is runni hooks/report-session-start.sh # session-start hook hooks/report-title.sh # session-title hook hooks/report-touched.sh # session-touched hook -hooks/lich.js # the Claude Code mods' entry, which registers the two below +hooks/lich.js # the Claude Code mods' entry, which registers the three below hooks/mod-control.js # mod-control client, a Claude Code mod hooks/agent-cards.js # a general-purpose subagent as a lich session, a Claude Code mod +hooks/mod-usage.js # mod-usage client, a Claude Code mod opencode/lich.js # opencode client: all four reports plus the seven tools, one module omp/lich.js # omp client: the reports, one module docs/ # client-side docs, one per contract diff --git a/docs/mod-usage.md b/docs/mod-usage.md new file mode 100644 index 0000000..5d3f117 --- /dev/null +++ b/docs/mod-usage.md @@ -0,0 +1,47 @@ +# Mod: usage + +Client side of the [mod-usage contract](https://github.com/omartelo/lich/blob/main/docs/hooks/mod-usage.md) +(the endpoint and the payload are defined in the lich repository; this plugin +only implements them). + +Reports what a Claude Code session measured about itself, the figures its +status line draws: the context window and how full it is, the account's +rate-limit windows, and what the conversation has cost. lich shows those +instead of the figures it derives from the transcript and from a request it +would otherwise spend measuring a token login's plan. + +| client | Claude Code | Codex | Antigravity | opencode | omp | Crush | +|----------------------|----------------------------------------------------------|---------------------|---------------------|---------------------|---------------------|---------------------| +| `hooks/mod-usage.js` | mod, registered by `hooks/lich.js`, the `modules` entry | none: no mod system | none: no mod system | none: no mod system | none: no mod system | none: no mod system | + +Claude Code takes one hooks module per plugin and refuses a second `modules` +entry, so `hooks/lich.js` is that module and registers each mod of the plugin +from its own file. + +| event | sent | +|-------------------|----------------------------------------------------------------------| +| `session.measure` | `POST /mod/usage` with `{"session_id", "conversation_id", "context", "rate_limits", "cost_usd"?}` | + +`conversation_id` is `$.session.id()`. `context` is `{window, tokens?, percent?}` +and each of `rate_limits` is `{kind, percent_used, resets_at?}`, Claude Code's +own figures renamed: one it does not have is left out, never zeroed. + +## How it behaves + +- **Outside lich it never reports.** It reads the three variables in + `session.start` and stops there when one is missing. +- **Only an interactive session reports.** A `claude -p` run from a tool inside + a lich session inherits that session's variables, and its figures would land + on the parent's card. Its `session.start` hook matches `isInteractive: true`, + so a non-interactive run never reads the link. +- **A measurement taken before `session.start` waits for it.** Claude Code + measures the session as it starts. +- **Reports go out one at a time, in order.** lich keeps the latest, so an older + one landing last would put stale figures back. +- **The event is never held up.** The report is posted beside the chain, a + failed one is dropped and never retried (the next carries the whole figures + again), and a `404` stops reporting for the session. + +Claude Code fires `session.measure` once at start, at the end of every +main-thread turn, and when a rate-limit window moves a whole point; never +mid-turn, never on a compaction (measured on 2.1.289). diff --git a/docs/providers.md b/docs/providers.md index f54ee89..b8190df 100644 --- a/docs/providers.md +++ b/docs/providers.md @@ -21,9 +21,10 @@ This repository is one plugin, packaged for each harness it supports: | `hooks/crush-hooks.json` | Crush | hook registration, merged by hand | | `hooks/*.sh` | the four | the reports themselves | | `hooks/win-run.cmd` | Codex | runs a script on Windows | -| `hooks/lich.js` | Claude Code | the one module `hooks/hooks.json` names under `modules`; registers the two below | +| `hooks/lich.js` | Claude Code | the one module `hooks/hooks.json` names under `modules`; registers the three below | | `hooks/mod-control.js` | Claude Code | the mod-control client, a mod | | `hooks/agent-cards.js` | Claude Code | runs a general-purpose subagent as a lich session, a mod | +| `hooks/mod-usage.js` | Claude Code | the mod-usage client, a mod | | `opencode/lich.js` | opencode | the whole client, as a module | | `omp/lich.js` | omp | the whole client, as a module | | `skills/` | all | skills, same layout | diff --git a/hooks/lich.js b/hooks/lich.js index 8fe97d3..24fa8bf 100644 --- a/hooks/lich.js +++ b/hooks/lich.js @@ -4,9 +4,11 @@ import { register as registerAgentCards } from "./agent-cards.js" import { register as registerModControl } from "./mod-control.js" +import { register as registerModUsage } from "./mod-usage.js" /** @param {import('claude-code').On} on */ export function register(on) { registerModControl(on) registerAgentCards(on) + registerModUsage(on) } diff --git a/hooks/mod-usage.js b/hooks/mod-usage.js new file mode 100644 index 0000000..e53ef55 --- /dev/null +++ b/hooks/mod-usage.js @@ -0,0 +1,119 @@ +// Reports what a Claude Code session measured about itself to lich. Contract: +// ../docs/mod-usage.md, canonical in +// https://github.com/omartelo/lich/blob/main/docs/hooks/mod-usage.md +// +// A Claude Code mod, not a hook script: hooks/hooks.json names this file under +// `modules`, beside mod-control.js. It hooks `session.measure`, which fires at +// start, at the end of every main-thread turn and when a rate-limit window +// moves, and posts the context window, the rate limits and the cost to lich, +// which shows those instead of the figures it derives. +// +// Every function that takes `$` is declared here at the top level: the loader +// refuses a module that hands `$` to a nested function. + +// Sent as X-Lich-Plugin on every request; bumped at release (CLAUDE.md, Release). +const PLUGIN_VERSION = "0.14.0" + +/** + * @typedef {import('claude-code').EngineInterface} Engine + * @typedef {import('claude-code').SessionMeasureInput} Measure + * @typedef {{ base: string, token: string, session: string }} Link + * @typedef {{ + * started: Promise, + * sending: Promise, + * gone: boolean, + * }} State + */ + +/** + * @param {Engine} $ + * @returns {Promise} + */ +async function linkFromEnv($) { + const [port, token, session] = await Promise.all([ + $.env.get("LICH_PORT"), + $.env.get("LICH_TOKEN"), + $.env.get("LICH_SESSION_ID"), + ]) + if (!port || !token || !session) return undefined + return { base: `http://127.0.0.1:${port}`, token, session } +} + +/** + * The body lich takes: Claude Code's figures under the contract's names, an + * absent figure left out rather than zeroed. + * + * @param {string} session + * @param {string} conversation + * @param {Measure} e + */ +function usageBody(session, conversation, e) { + return { + session_id: session, + conversation_id: conversation, + context: { + window: e.context.window, + ...(e.context.tokens !== undefined && { tokens: e.context.tokens }), + ...(e.context.percent !== undefined && { percent: e.context.percent }), + }, + rate_limits: e.rateLimits.map((limit) => ({ + kind: limit.kind, + percent_used: limit.percentUsed, + ...(limit.resetsAt !== undefined && { resets_at: limit.resetsAt }), + })), + ...(e.cost !== undefined && { cost_usd: e.cost.usd }), + } +} + +/** + * Posts one measurement, once the session's link is known. A report that + * fails is dropped: the next measurement carries the whole figures again. + * + * @param {Engine} $ + * @param {State} state + * @param {Measure} e + */ +async function report($, state, e) { + const link = await state.started + if (!link || state.gone) return + try { + const response = await $.http.fetch(`${link.base}/mod/usage?token=${link.token}`, { + method: "POST", + headers: { "content-type": "application/json", "X-Lich-Plugin": PLUGIN_VERSION }, + body: JSON.stringify(usageBody(link.session, await $.session.id(), e)), + }) + // A lich older than the contract: nothing it would take is coming. + if (response.status === 404) state.gone = true + } catch { + // The contract drops a report that cannot be sent; it never retries one. + } +} + +/** @param {import('claude-code').On} on */ +export function register(on) { + /** @type {(link: Link | undefined) => void} */ + let start = () => {} + /** @type {State} */ + const state = { + started: new Promise((resolve) => { + start = resolve + }), + sending: Promise.resolve(), + gone: false, + } + + // Interactive sessions only: a `claude -p` started from a tool inside a lich + // session inherits that session's variables, and its figures would land on + // the parent's card. Its measurements wait on a link that never comes. + on("session.start", { isInteractive: true }, async ($, e, next) => { + start(await linkFromEnv($)) + return next(e) + }) + + on("session.measure", ($, e, next) => { + // One at a time, in order: lich keeps the latest report, so an older one + // landing last would put stale figures back on the card. + state.sending = state.sending.then(() => report($, state, e)) + return next(e) + }) +} diff --git a/tests/contract.mjs b/tests/contract.mjs index daed500..d3e92bd 100644 --- a/tests/contract.mjs +++ b/tests/contract.mjs @@ -38,6 +38,7 @@ export const FIXTURE_OF = { '/session-title': 'session-title', '/session-touched': 'session-touched', '/mod/acks': 'mod-control', + '/mod/usage': 'mod-usage', } const readFixture = (name) => @@ -116,6 +117,22 @@ export const REJECT_RULES = { 'empty session_id': (b) => b.session_id === '', 'malformed json': (b, raw) => !isJsonObject(raw), }, + '/mod/usage': { + 'missing session_id': (b) => !('session_id' in b), + 'missing conversation_id': (b) => !('conversation_id' in b), + 'missing context': (b) => !isObject(b.context), + 'zero window': (b) => isObject(b.context) && !(b.context.window > 0), + 'negative tokens': (b) => isObject(b.context) && b.context.tokens < 0, + 'percent past 100': (b) => isObject(b.context) && b.context.percent > 100, + 'fractional percent': (b) => + isObject(b.context) && 'percent' in b.context && !Number.isInteger(b.context.percent), + 'rate limit without kind': (b) => (b.rate_limits ?? []).some((l) => blank(l.kind)), + 'negative rate limit': (b) => (b.rate_limits ?? []).some((l) => l.percent_used < 0), + 'reset that is not a timestamp': (b) => + (b.rate_limits ?? []).some((l) => 'resets_at' in l && Number.isNaN(Date.parse(l.resets_at))), + 'negative cost': (b) => b.cost_usd < 0, + 'malformed json': (b, raw) => !isJsonObject(raw), + }, '/mod/acks': { 'missing session_id': (b) => !('session_id' in b), 'missing id': (b) => !('id' in b), diff --git a/tests/fixtures/mod-usage.jsonl b/tests/fixtures/mod-usage.jsonl new file mode 100644 index 0000000..a3846f2 --- /dev/null +++ b/tests/fixtures/mod-usage.jsonl @@ -0,0 +1,18 @@ +{"name":"turn end","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000,"tokens":43592,"percent":22},"rate_limits":[{"kind":"five_hour","percent_used":14,"resets_at":"2026-10-05T00:30:00.000Z"},{"kind":"seven_day","percent_used":80,"resets_at":"2026-10-05T15:00:00.000Z"}],"cost_usd":0.0977631},"accept":{"session_id":"s1","conversation_id":"c1","context":{"window":200000,"tokens":43592,"percent":22},"rate_limits":[{"kind":"five_hour","percent_used":14,"resets_at":"2026-10-05T00:30:00.000Z"},{"kind":"seven_day","percent_used":80,"resets_at":"2026-10-05T15:00:00.000Z"}],"cost_usd":0.0977631}} +{"name":"start, before the first response","body":{"session_id":"s1","conversation_id":"c1","context":{"window":1000000},"rate_limits":[{"kind":"five_hour","percent_used":0},{"kind":"seven_day","percent_used":1}],"cost_usd":0},"accept":{"context":{"window":1000000},"cost_usd":0}} +{"name":"off a subscription","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000,"tokens":100,"percent":0},"rate_limits":[],"cost_usd":0.5},"accept":{"rate_limits":[]}} +{"name":"no rate limits and no ledger","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000}},"accept":{"context":{"window":200000}}} +{"name":"exceeded spend limit","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000},"rate_limits":[{"kind":"spend_limit","percent_used":104.5}]},"accept":{"rate_limits":[{"kind":"spend_limit","percent_used":104.5}]}} +{"name":"extra fields are ignored","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000},"changed":["context"]},"accept":{"conversation_id":"c1"}} +{"name":"missing session_id","body":{"conversation_id":"c1","context":{"window":200000}},"reject":"session_id is required"} +{"name":"missing conversation_id","body":{"session_id":"s1","context":{"window":200000}},"reject":"conversation_id is required"} +{"name":"missing context","body":{"session_id":"s1","conversation_id":"c1"},"reject":"context.window is required"} +{"name":"zero window","body":{"session_id":"s1","conversation_id":"c1","context":{"window":0}},"reject":"context.window must be positive"} +{"name":"negative tokens","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000,"tokens":-1}},"reject":"context.tokens must not be negative"} +{"name":"percent past 100","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000,"percent":101}},"reject":"context.percent must be 0 to 100"} +{"name":"fractional percent","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000,"percent":21.5}},"reject":"context.percent must be an integer"} +{"name":"rate limit without kind","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000},"rate_limits":[{"percent_used":3}]},"reject":"a rate limit's kind is required"} +{"name":"negative rate limit","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000},"rate_limits":[{"kind":"five_hour","percent_used":-1}]},"reject":"percent_used must not be negative"} +{"name":"reset that is not a timestamp","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000},"rate_limits":[{"kind":"five_hour","percent_used":3,"resets_at":"soon"}]},"reject":"resets_at must be RFC 3339"} +{"name":"negative cost","body":{"session_id":"s1","conversation_id":"c1","context":{"window":200000},"cost_usd":-0.01},"reject":"cost_usd must not be negative"} +{"name":"malformed json","raw":"{","reject":"body must be a JSON object"} diff --git a/tests/mod-usage.engine.test.ts b/tests/mod-usage.engine.test.ts new file mode 100644 index 0000000..3b46632 --- /dev/null +++ b/tests/mod-usage.engine.test.ts @@ -0,0 +1,65 @@ +// The usage report under Claude Code's own engine: `claude plugin test .` loads +// the plugin's hooks module (hooks/lich.js, which registers hooks/mod-usage.js) +// from hooks/hooks.json the way a session does, the test's hooks standing for +// lich (`http.fetch`). tests/mod-usage.test.mjs is the suite CI runs against +// the contract fixtures; this one proves the module loads and its hooks chain +// in the engine that runs it. + +import { expect, mock, test } from 'claude-code/testing' +import type { On, SessionMeasureInput } from 'claude-code' + +const ENV = { LICH_PORT: '47999', LICH_TOKEN: 'tok', LICH_SESSION_ID: 'lich-1' } + +const MEASURED: SessionMeasureInput = { + context: { tokens: 43592, window: 200000, percent: 22 }, + rateLimits: [{ kind: 'five_hour', percentUsed: 14, resetsAt: '2026-10-05T00:30:00.000Z' }], + cost: { usd: 0.0977631 }, + changed: ['context', 'cost'], +} + +/** + * Records every usage report and parks every poll, as lich does with nothing + * queued; also stands for the engine's own session start, measurement and id, + * which the kit leaves to the test. + */ +function lich(on: On) { + const reports: Record[] = [] + on('session.start', async (_$, e) => ({ cwd: e.cwd })) + on('session.measure', async (_$, e) => ({ changed: e.changed })) + on('session.id', async () => ({ value: 'conv-1' })) + on('http.fetch', async (_$, e) => { + if (e.url.includes('/mod/usage')) { + reports.push(JSON.parse(e.init?.body ?? '{}')) + return { value: { status: 204, ok: true, headers: {}, text: '' } } + } + return new Promise(() => {}) + }) + return { reports } +} + +test('a measurement is reported to lich', async ($, on) => { + mock.env(on, ENV) + const clock = mock.clock(on) + const world = lich(on) + await $.session.start({ cwd: '/w', surface: 'terminal', isInteractive: true }) + await $.session.measure(MEASURED) + await clock.settle() + expect(world.reports).toHaveLength(1) + expect(world.reports[0]).toMatchObject({ + session_id: 'lich-1', + conversation_id: 'conv-1', + context: { window: 200000, tokens: 43592, percent: 22 }, + rate_limits: [{ kind: 'five_hour', percent_used: 14, resets_at: '2026-10-05T00:30:00.000Z' }], + cost_usd: 0.0977631, + }) +}) + +test('a non-interactive run reports nothing', async ($, on) => { + mock.env(on, ENV) + const clock = mock.clock(on) + const world = lich(on) + await $.session.start({ cwd: '/w', surface: 'terminal', isInteractive: false }) + await $.session.measure(MEASURED) + await clock.settle() + expect(world.reports).toHaveLength(0) +}) diff --git a/tests/mod-usage.test.mjs b/tests/mod-usage.test.mjs new file mode 100644 index 0000000..4ef1368 --- /dev/null +++ b/tests/mod-usage.test.mjs @@ -0,0 +1,223 @@ +// Pins the Claude Code mod's usage report (hooks/mod-usage.js) to lich's +// mod-usage contract. +// +// Like mod-control.test.mjs, it imports the module and hands it a fake engine, +// recording every fetch as contract.mjs reads a request, so the reports answer +// to the same fixtures lich asserts (mod-usage.jsonl). The `session.measure` +// payloads below are what Claude Code 2.1.289 delivered against a stub lich. +// +// Run: node --test tests/ + +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import path from 'node:path' + +import { LICH_SESSION_ID, ROOT, assertContractHonoured, lichEnv } from './contract.mjs' +import { register } from '../hooks/mod-usage.js' +import { register as registerEntry } from '../hooks/lich.js' + +const PORT = 47998 +const CONVERSATION = '3fb3ad05-5b58-45ea-a1e0-0d1d4b0a89b9' + +const AT_START = { + context: { window: 200000 }, + rateLimits: [ + { kind: 'five_hour', percentUsed: 14, resetsAt: '2026-10-05T00:30:00.000Z' }, + { kind: 'seven_day', percentUsed: 80, resetsAt: '2026-10-05T15:00:00.000Z' }, + ], + cost: { usd: 0 }, + changed: ['context', 'rateLimits', 'cost'], +} + +const AT_TURN_END = { + context: { tokens: 43592, window: 200000, percent: 22 }, + rateLimits: AT_START.rateLimits, + cost: { usd: 0.0977631 }, + changed: ['context', 'cost'], +} + +const NO_CONTENT = { status: 204, ok: true, headers: {}, text: '' } + +/** Waits for `predicate` to hold, failing after a second of real time. */ +async function until(predicate, what) { + const deadline = Date.now() + 1000 + while (!predicate()) { + if (Date.now() > deadline) assert.fail(`timed out waiting for ${what}`) + await new Promise((r) => setImmediate(r)) + } +} + +const settle = () => new Promise((r) => setTimeout(r, 20)) + +/** + * Registers the module against a fake engine. A hook registered with a + * matcher runs only for an event that matches it, as the engine runs it. + * `answer` answers each report. + */ +function load({ env = lichEnv(PORT), answer = () => Promise.resolve(NO_CONTENT) } = {}) { + const hooks = new Map() + register((event, ...rest) => { + const hook = rest.pop() + const matcher = rest[0] ?? {} + hooks.set(event, { matcher, hook }) + }) + const requests = [] + const $ = { + env: { get: async (name) => env[name] }, + session: { id: async () => CONVERSATION }, + http: { + fetch: (url, init = {}) => { + const { pathname, search } = new URL(url) + const headers = Object.fromEntries(Object.entries(init.headers ?? {}).map(([k, v]) => [k.toLowerCase(), v])) + const request = { method: init.method ?? 'GET', url: pathname + search, headers, raw: init.body ?? '' } + requests.push(request) + return answer(request) + }, + }, + } + const passOn = async (e) => e + const fire = (event, e) => { + const { matcher, hook } = hooks.get(event) + const matches = Object.entries(matcher).every(([key, value]) => e[key] === value) + return matches ? hook($, e, passOn) : passOn(e) + } + return { + requests, + reports: () => requests.map((r) => JSON.parse(r.raw)), + start: (isInteractive = true) => fire('session.start', { cwd: '/w', surface: 'terminal', isInteractive }), + measure: (e) => fire('session.measure', e), + } +} + +test('a measurement reaches lich as the contract spells it', async () => { + const mod = load() + await mod.start() + + await mod.measure(AT_TURN_END) + await until(() => mod.requests.length === 1, 'the report') + + assertContractHonoured('/mod/usage', mod.requests[0]) + assert.deepEqual(mod.reports()[0], { + session_id: LICH_SESSION_ID, + conversation_id: CONVERSATION, + context: { window: 200000, tokens: 43592, percent: 22 }, + rate_limits: [ + { kind: 'five_hour', percent_used: 14, resets_at: '2026-10-05T00:30:00.000Z' }, + { kind: 'seven_day', percent_used: 80, resets_at: '2026-10-05T15:00:00.000Z' }, + ], + cost_usd: 0.0977631, + }) +}) + +test('a figure Claude Code does not have is left out, never zeroed', async () => { + const mod = load() + await mod.start() + + await mod.measure({ context: { window: 200000 }, rateLimits: [{ kind: 'five_hour', percentUsed: 3 }], changed: ['context'] }) + await until(() => mod.requests.length === 1, 'the report') + + assertContractHonoured('/mod/usage', mod.requests[0]) + const body = mod.reports()[0] + assert.deepEqual(body.context, { window: 200000 }) + assert.deepEqual(body.rate_limits, [{ kind: 'five_hour', percent_used: 3 }]) + assert.ok(!('cost_usd' in body)) +}) + +// Claude Code measures the session as it starts, which can land before the +// session.start hook has read the link. +test('a measurement taken before the session started is sent once it does', async () => { + const mod = load() + + await mod.measure(AT_START) + await settle() + assert.equal(mod.requests.length, 0) + await mod.start() + await until(() => mod.requests.length === 1, 'the held report') + + assertContractHonoured('/mod/usage', mod.requests[0]) +}) + +test('the event is passed on without waiting for lich', async () => { + const mod = load({ answer: () => new Promise(() => {}) }) + await mod.start() + + const passed = await mod.measure(AT_START) + + assert.equal(passed, AT_START) +}) + +test('outside lich nothing is sent', async () => { + const mod = load({ env: {} }) + await mod.start() + + await mod.measure(AT_TURN_END) + await settle() + + assert.equal(mod.requests.length, 0) +}) + +test('a non-interactive run never reports onto the card that started it', async () => { + const mod = load() + await mod.start(false) + + await mod.measure(AT_TURN_END) + await settle() + + assert.equal(mod.requests.length, 0) +}) + +test('reports go out one at a time, in the order they were measured', async () => { + let release + const first = new Promise((r) => { + release = () => r(NO_CONTENT) + }) + const mod = load({ answer: () => (mod.requests.length === 1 ? first : Promise.resolve(NO_CONTENT)) }) + await mod.start() + + await mod.measure(AT_START) + await mod.measure(AT_TURN_END) + await settle() + assert.equal(mod.requests.length, 1, 'the second report went out before the first was answered') + release() + await until(() => mod.requests.length === 2, 'the second report') + + assert.deepEqual(mod.reports().map((r) => r.cost_usd), [0, 0.0977631]) +}) + +test('a report that fails is dropped and the next one still goes out', async () => { + const mod = load({ + answer: () => + mod.requests.length === 1 ? Promise.reject(new Error('connect ECONNREFUSED')) : Promise.resolve(NO_CONTENT), + }) + await mod.start() + + await mod.measure(AT_START) + await mod.measure(AT_TURN_END) + await until(() => mod.requests.length === 2, 'the report after the failure') + await settle() + + assert.equal(mod.requests.length, 2, 'the failed report was retried') +}) + +test('a lich that predates the contract is not asked again', async () => { + const mod = load({ answer: () => Promise.resolve({ status: 404, ok: false, headers: {}, text: '' }) }) + await mod.start() + + await mod.measure(AT_START) + await mod.measure(AT_TURN_END) + await settle() + + assert.equal(mod.requests.length, 1) +}) + +// Claude Code takes one hooks module per plugin, so hooks.json names the entry +// that registers every mod of the plugin, this one included. +test('the plugin entry module registers the usage report beside control', () => { + const hooksJson = JSON.parse(readFileSync(path.join(ROOT, 'hooks', 'hooks.json'), 'utf8')) + assert.deepEqual(hooksJson.modules, ['./lich.js']) + const events = [] + registerEntry((event) => events.push(event)) + assert.ok(events.includes('session.measure'), 'mod-usage is not registered') + assert.ok(events.includes('turn.step'), 'mod-control is not registered') +}) diff --git a/tests/refresh-fixtures.sh b/tests/refresh-fixtures.sh index 2094ff6..c9f8bad 100755 --- a/tests/refresh-fixtures.sh +++ b/tests/refresh-fixtures.sh @@ -23,6 +23,14 @@ for file in session-state.jsonl session-start.jsonl session-title.jsonl \ echo "refreshed $file at $ref" done +# The mod-usage contract is in no lich release yet either: its fixture is read +# at the lich commit that adds it (branch feat/mod-usage), on the same terms as +# the block below, and folds into the first list once a release ships it. +usage=445ee9e90b7b434135bc369f41486a37f0b738f6 +curl -fsSL "https://raw.githubusercontent.com/omartelo/lich/$usage/docs/hooks/fixtures/mod-usage.jsonl" \ + -o "$here/fixtures/mod-usage.jsonl" +echo "refreshed mod-usage.jsonl at $usage (unreleased)" + # The mod-control `command` and `ask` kinds are in no lich release yet: these two files are # read at the lich commit that carries them (branch feat/ask-session), so the # module is held to the contract lich is about to ship rather than to the one From d28bae7dad275ce720ae0ef2e33cb3a2e2b709ba Mon Sep 17 00:00:00 2001 From: omartelo Date: Sun, 4 Oct 2026 19:29:10 -0300 Subject: [PATCH 2/2] test: read the unreleased fixtures at the lich commits that are pushed --- tests/refresh-fixtures.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/refresh-fixtures.sh b/tests/refresh-fixtures.sh index c9f8bad..db1f6ab 100755 --- a/tests/refresh-fixtures.sh +++ b/tests/refresh-fixtures.sh @@ -24,19 +24,19 @@ for file in session-state.jsonl session-start.jsonl session-title.jsonl \ done # The mod-usage contract is in no lich release yet either: its fixture is read -# at the lich commit that adds it (branch feat/mod-usage), on the same terms as +# at a lich main commit that carries it, on the same terms as # the block below, and folds into the first list once a release ships it. -usage=445ee9e90b7b434135bc369f41486a37f0b738f6 +usage=6476d2d27ebdb5716fd369647d383dbd13a00936 curl -fsSL "https://raw.githubusercontent.com/omartelo/lich/$usage/docs/hooks/fixtures/mod-usage.jsonl" \ -o "$here/fixtures/mod-usage.jsonl" echo "refreshed mod-usage.jsonl at $usage (unreleased)" # The mod-control `command` and `ask` kinds are in no lich release yet: these two files are -# read at the lich commit that carries them (branch feat/ask-session), so the +# read at a lich main commit that carries them, so the # module is held to the contract lich is about to ship rather than to the one # it replaces. Once a release ships it, move tests/lich-ref there, fold these # files back into the list above and delete this block. -pending=f6db5f95675413ec7029658d04dbb96f0bb375c0 +pending=6476d2d27ebdb5716fd369647d383dbd13a00936 for file in mod-control.jsonl mod-commands.json; do curl -fsSL "https://raw.githubusercontent.com/omartelo/lich/$pending/docs/hooks/fixtures/$file" \ -o "$here/fixtures/$file"