From d802dd3654948c6b95bf18ba3d633841e874cb20 Mon Sep 17 00:00:00 2001 From: omartelo Date: Mon, 10 Aug 2026 10:08:42 -0300 Subject: [PATCH] test: assert hook payloads against lich's contract fixtures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two sides of every hook live in different repositories: lich owns the endpoints, this repository owns the scripts that build the payloads. A renamed field went red in neither. lich now publishes each contract as bytes (docs/hooks/fixtures/*.jsonl there, one case per line: the payloads it accepts and the ones it refuses) and asserts its endpoints against them; this is the other half. node --test tests/*.test.mjs runs every hook script as a real subprocess — the command line taken from the registration each harness actually reads, quoting included — against a stub HTTP server, and asserts the body it POSTs: its key set and value classes match some accept case, it matches no reject case, it lands on the right endpoint with the right ?token=, and it never sends the deprecated claude_session_id. The accepted states and providers are derived from the fixtures rather than copied beside them, so a value lich stops accepting stops being accepted here. A reject case with no client-side rule fails its own test, so a contract that grows a refusal cannot slip through unmodelled. The client rules are pinned too: no report without the full lich environment, and exit 0 with an empty stdout when lich answers 500 or refuses the connection. Silence is load-bearing on Codex, where a hook's own exit code answers a PermissionRequest. The suite found one payload that disagreed. report-title.sh checked that it had extracted something before posting, which a title of nothing but spaces passes, and lich rejects a blank title with a 400. Codex made it reachable: it names a thread after its first user message, so a prompt whose first line is indentation produced exactly that payload. The title is trimmed before the check now, which also matches what lich stores. The fixtures are vendored in tests/fixtures/ so the suite never needs the network — network in a test is flake, drift is not. tests/refresh-fixtures.sh re-pulls them and CI diffs the copies against lich on every run, so a contract that moves upstream goes red here without any test reaching out. --- .github/workflows/ci.yml | 31 ++ CHANGELOG.md | 26 ++ CLAUDE.md | 24 ++ README.md | 9 + hooks/report-title.sh | 4 + tests/fixtures/session-start.jsonl | 9 + tests/fixtures/session-state.jsonl | 10 + tests/fixtures/session-title.jsonl | 7 + tests/fixtures/session-touched.jsonl | 5 + tests/hooks.test.mjs | 595 +++++++++++++++++++++++++++ tests/refresh-fixtures.sh | 19 + 11 files changed, 739 insertions(+) create mode 100644 .github/workflows/ci.yml create mode 100644 tests/fixtures/session-start.jsonl create mode 100644 tests/fixtures/session-state.jsonl create mode 100644 tests/fixtures/session-title.jsonl create mode 100644 tests/fixtures/session-touched.jsonl create mode 100644 tests/hooks.test.mjs create mode 100755 tests/refresh-fixtures.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..d2aa777 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,31 @@ +name: ci + +on: + push: + branches: [main] + pull_request: + +jobs: + test: + name: hook contract tests + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: '22' + - run: node --test tests/*.test.mjs + + fixtures-drift: + name: fixtures match lich + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Refresh from omartelo/lich@main + run: tests/refresh-fixtures.sh + - name: Fail if the vendored copies drifted + run: | + git diff --exit-code -- tests/fixtures || { + echo "::error::A hook contract moved in lich. Run tests/refresh-fixtures.sh, then make the scripts match." + exit 1 + } diff --git a/CHANGELOG.md b/CHANGELOG.md index db4f0ab..56b28f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **The hook payloads are asserted against lich's contract fixtures.** lich + publishes every hook contract as bytes (`docs/hooks/fixtures/*.jsonl` there, + one case per line: the payloads it accepts and the ones it refuses) and + asserts its endpoints against them. This repository now asserts the other + half: `node --test tests/*.test.mjs` runs every hook script as a real + subprocess — from the command line its registration actually spells, for both + harnesses — against a stub HTTP server, and checks the body it POSTs matches + an accepted shape, matches no rejected one, lands on the right endpoint with + the right `?token=`, and never sends the deprecated `claude_session_id`. It + also pins the client rules: no report without the full lich environment, exit + 0 and silence when lich answers 500 or refuses the connection. The fixtures + are vendored (`tests/fixtures/`, refreshed by `tests/refresh-fixtures.sh`) so + the suite never needs the network, and CI diffs the copies against lich on + every run. A field that moves in either repository now goes red in both. + +### Fixed + +- **A blank session title is no longer reported.** `report-title.sh` checked + that it had extracted *something* before posting, which a title of nothing but + spaces passes — and lich rejects a blank title with a 400. Codex made it + reachable: it names a thread after its first user message, so a prompt whose + first line is indentation produced exactly that payload. The title is trimmed + before the check now, which also matches what lich stores. + ## [0.6.0] - 2026-08-09 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index cfa46e0..d752c65 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,6 +14,7 @@ hooks/codex-hooks.json # hook registration, Codex hooks/ # hook scripts ($CLAUDE_PLUGIN_ROOT/hooks/