Skip to content

feat(cli): check skill downloads against the published tree hash - #274

Open
dirtybits wants to merge 1 commit into
mainfrom
feat/cli-tree-hash-verify
Open

dirtybits wants to merge 1 commit into
mainfrom
feat/cli-tree-hash-verify

Conversation

@dirtybits

Copy link
Copy Markdown
Owner

What this changes

Phase 1, PR 2 of .agents/plans/skill-evals-and-packages.plan.md. Follows #262.

Do not merge before the production sweep shows zero mismatches. After this PR, the CLI refuses any download that does not match its published hash. If stored data has a mismatch, those installs stop working. Run the sweep first (command below).

CLI (packages/agentvouch-cli)

  • skill install and skills update check every download before they write anything (new src/lib/integrity.ts, algorithm agentvouch-tree-v1):
    • A skill folder: recompute the tree hash and compare it with tree_hash.
    • A single SKILL.md: compare its SHA-256 with the published file list, and check that list against tree_hash.
  • On a mismatch, the CLI re-reads the skill once, because a new version can be published during the download. Then it stops and writes nothing.
  • If the skill has no published tree_hash, the CLI stops before it downloads or pays. Without this, a paid install could take the payment and then refuse to write.
  • The CLI hashes and writes the exact /raw bytes. Before, response.text() dropped a leading byte order mark.
  • The CLI reads downloads with the mirror limits (512 files, 8 MB), so large mirrored skills install.
  • <path>.agentvouch.json records tree_hash_algorithm and tree_hash_verified_at. The install output prints tree_hash_verified.
  • The CLI passes the shared tree-hash vectors from feat(web): add skill tree hash spec, shared vectors, and read-only sweep #262.

Web

  • New GET /api/skills/{id}/versions/{version}/archive: one stored version, with the same access rules as /archive. Like /archive, it records a download event.
  • Upload rejects a tree where one path is both a file and a folder (docs and docs/a.md) with a 400. putSkillTree rejects it too.
  • Mirror sync skips such a skill with a path conflict detail, like a size-limit violation. So one bad upstream skill cannot make the daily mirror cron return 500.
  • Stored trees are read with the mirror limits. Before, /raw?path= and /zip failed for a mirrored tree above the upload limits (200 files, 5 MB).

Docs: docs/SKILL_TREE_HASH.md (client steps, Known gaps), web/public/skill.md (2.3.0: the version route and the download check), the CLI README, and the plan (status updates and one dated note).

Run the sweep before merge

From web/ on the Mac, with the production values in .env.local:

npm ci
npx tsx --env-file=.env.local ./scripts/tree-hash-sweep.ts --json /tmp/tree-hash-sweep.json
  • Exit code 0: every stored version matches. Exit code 1: findings. The JSON lists each finding with a code from docs/SKILL_TREE_HASH.md.
  • The sweep only reads (one SELECT plus Blob reads). It prints the database host first.
  • If you run it on main, the sweep still reads stored trees with the upload limits. A mirrored tree above 200 files or 5 MB then shows as files_unavailable with an "exceeds cap" message. This PR fixes that reader, so on this branch such a tree is checked normally.

Verified

  • npm run format:check, npm run lint, npm run typecheck, and CLI tsc --noEmit.
  • npm test: web 166 files, 1,613 tests; CLI 18 files, 173 tests.
  • New tests: a tampered single file, an archive with an extra file, a missing tree hash (free and paid: no download and no payment), a version published during the download, a kept byte order mark, a dry run, path conflicts at upload (JSON and multipart), the mirror write and the mirror sync skip, read limits on a 250-file stored tree, and the version route (bad version, access denied, served version, 404).
  • The new mirror-sync test fails without the fix.
  • Python reference: 10 vectors and 9 invalid paths match.
  • next build --webpack compiled and ran TypeScript, then stopped at the /sitemap.xml prerender because this workspace has no DATABASE_URL. The Vercel preview is the real build check.

Not verified

  • The production sweep has not run.
  • No live request to the new version route yet. After deploy: download version N of a free skill and compare it with that version's tree_hash.
  • Chain-only skills (chain-… ids) have no tree hash, so the CLI cannot check them. This is unchanged.


Generated by Claude Code

Phase 1, PR 2 of .agents/plans/skill-evals-and-packages.plan.md.
Follows #262 (merged).

CLI:
- New src/lib/integrity.ts (agentvouch-tree-v1). skill install and
  skills update check every download before they write: a tree against
  tree_hash, and a single SKILL.md against its manifest entry and the
  manifest against tree_hash. On a mismatch the CLI re-reads the skill
  once (a new version can be published during the download), then stops
  and writes nothing.
- Stop before any download or payment when the skill has no published
  tree_hash, so a paid install cannot take the payment and then refuse
  to write.
- Hash and write the exact /raw bytes. response.text() dropped a
  leading byte order mark.
- Read downloads with the mirror limits (512 files, 8 MB).
- Install metadata records tree_hash_algorithm and tree_hash_verified_at.
- The CLI passes the shared tree-hash vectors.

Web:
- GET /api/skills/{id}/versions/{version}/archive serves one stored
  version, with the same access rules as /archive.
- Upload rejects a file/folder path conflict ("docs" and "docs/a.md")
  with a 400, and putSkillTree rejects it too. Mirror sync skips such a
  skill, so it cannot fail the daily mirror cron.
- Read stored trees with the mirror limits, so /raw?path= and /zip can
  serve a mirrored tree above the upload limits.

Docs: docs/SKILL_TREE_HASH.md, web/public/skill.md (2.3.0), the CLI
README, and the plan.

Verified: format:check, lint, web and CLI typecheck, web vitest (166
files, 1613 tests), CLI vitest (18 files, 173 tests), Python vectors.
next build --webpack compiled and ran TypeScript, then stopped at the
/sitemap.xml prerender without DATABASE_URL.

Not verified: the production sweep, and a live fetch from the new
version route. Merge only after the sweep shows zero mismatches.

This branch was successfully deployed

1 active deployment
Preview — 7ccea445 Deployed Nov 4, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant